[must-gather ] OUT 2026-01-27T19:01:17.852504865Z Using must-gather plug-in image: quay.io/openstack-k8s-operators/openstack-must-gather:latest When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304 ClientVersion: 4.20.10 ClusterVersion: Stable at "4.16.0" ClusterOperators: clusteroperator/kube-apiserver is progressing: NodeInstallerProgressing: 1 node is at revision 12; 0 nodes have achieved new revision 13 clusteroperator/machine-config is not upgradeable because One or more machine config pools are degraded, please see `oc get mcp` for further details and resolve before upgrading clusteroperator/network is degraded because Error while updating operator configuration: could not apply (apps/v1, Kind=DaemonSet) openshift-network-operator/iptables-alerter: failed to apply / update (apps/v1, Kind=DaemonSet) openshift-network-operator/iptables-alerter: Patch "https://api-int.crc.testing:6443/apis/apps/v1/namespaces/openshift-network-operator/daemonsets/iptables-alerter?fieldManager=cluster-network-operator%2Foperconfig&force=true": dial tcp 38.102.83.217:6443: connect: connection refused clusteroperator/openshift-samples is degraded because Samples installation in error at 4.16.0: &errors.errorString{s:"global openshift image configuration prevents the creation of imagestreams using the registry "} clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing [must-gather ] OUT 2026-01-27T19:01:17.925947261Z namespace/openshift-must-gather-9xrqh created [must-gather ] OUT 2026-01-27T19:01:17.930408578Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-bdwzh created [must-gather ] OUT 2026-01-27T19:01:18.051166553Z pod for plug-in image quay.io/openstack-k8s-operators/openstack-must-gather:latest created [must-gather-g98xq] OUT 2026-01-27T19:03:56.863346159Z waiting for gather to complete [must-gather-g98xq] OUT 2026-01-27T19:03:56.866414707Z downloading gather output [must-gather-g98xq] OUT 2026-01-27T19:03:57.134778083Z receiving incremental file list [must-gather-g98xq] OUT 2026-01-27T19:03:57.143818981Z ./ [must-gather-g98xq] OUT 2026-01-27T19:03:57.144107129Z must-gather.tar.xz [must-gather-g98xq] OUT 2026-01-27T19:03:57.190439591Z packagemanifests [must-gather-g98xq] OUT 2026-01-27T19:03:58.048779869Z run.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.049105648Z version [must-gather-g98xq] OUT 2026-01-27T19:03:58.049331445Z crd/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049346655Z csv/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049350885Z ctlplane/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049641364Z ctlplane/endpoints [must-gather-g98xq] OUT 2026-01-27T19:03:58.049899251Z ctlplane/services [must-gather-g98xq] OUT 2026-01-27T19:03:58.049910351Z ctlplane/ovn/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049915572Z ctlplane/rabbitmq/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049920302Z dbs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049924952Z namespaces/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.049929612Z namespaces/openshift-machine-api/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.050469317Z namespaces/openshift-machine-api/all_resources.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.050697534Z namespaces/openshift-machine-api/events.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.051066904Z namespaces/openshift-machine-api/nad.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.05127485Z namespaces/openshift-machine-api/pvc.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.051429095Z namespaces/openshift-machine-api/buildconfig/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.051437025Z namespaces/openshift-machine-api/cert/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.051441185Z namespaces/openshift-machine-api/certificaterequests/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.051445205Z namespaces/openshift-machine-api/configmaps/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.051460646Z namespaces/openshift-machine-api/cronjobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.051464566Z namespaces/openshift-machine-api/daemonset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.051468236Z namespaces/openshift-machine-api/deployments/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.052113334Z namespaces/openshift-machine-api/deployments/control-plane-machine-set-operator.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.052439124Z namespaces/openshift-machine-api/deployments/machine-api-operator.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.0526577Z namespaces/openshift-machine-api/imagestream/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05266656Z namespaces/openshift-machine-api/installplans/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05267092Z namespaces/openshift-machine-api/issuer/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.052957088Z namespaces/openshift-machine-api/jobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.052968099Z namespaces/openshift-machine-api/poddisruptionbudgets/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.052972259Z namespaces/openshift-machine-api/pods/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.052976239Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05336979Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/control-plane-machine-set-operator-649bd778b4-tt5tw-describe [must-gather-g98xq] OUT 2026-01-27T19:03:58.053616437Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.053831083Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/control-plane-machine-set-operator.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.05405462Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.054245565Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/machine-api-operator-788b7c6b6c-ctdmb-describe [must-gather-g98xq] OUT 2026-01-27T19:03:58.054609255Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.054785711Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/kube-rbac-proxy.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.055090889Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/machine-api-operator.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.055302875Z namespaces/openshift-machine-api/replicaset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05546365Z namespaces/openshift-machine-api/replicaset/control-plane-machine-set-operator-649bd778b4.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.055672516Z namespaces/openshift-machine-api/replicaset/machine-api-operator-788b7c6b6c.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.055865311Z namespaces/openshift-machine-api/routes/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.055874322Z namespaces/openshift-machine-api/services/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05615993Z namespaces/openshift-machine-api/services/cluster-autoscaler-operator.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.056387746Z namespaces/openshift-machine-api/services/control-plane-machine-set-operator.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.056634173Z namespaces/openshift-machine-api/services/machine-api-controllers.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.056897981Z namespaces/openshift-machine-api/services/machine-api-operator-machine-webhook.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.057346924Z namespaces/openshift-machine-api/services/machine-api-operator-webhook.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.057360664Z namespaces/openshift-machine-api/services/machine-api-operator.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.057611351Z namespaces/openshift-machine-api/statefulset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.057620091Z namespaces/openshift-machine-api/subscriptions/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.057639752Z namespaces/openshift-operators/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05793926Z namespaces/openshift-operators/all_resources.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.058185307Z namespaces/openshift-operators/events.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.058407314Z namespaces/openshift-operators/nad.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.058653821Z namespaces/openshift-operators/pvc.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.058846706Z namespaces/openshift-operators/buildconfig/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.058855927Z namespaces/openshift-operators/cert/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.058860167Z namespaces/openshift-operators/certificaterequests/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.058864167Z namespaces/openshift-operators/configmaps/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.058867997Z namespaces/openshift-operators/cronjobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.05932208Z namespaces/openshift-operators/daemonset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059351151Z namespaces/openshift-operators/deployments/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059355701Z namespaces/openshift-operators/imagestream/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059359481Z namespaces/openshift-operators/installplans/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059363211Z namespaces/openshift-operators/issuer/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059370171Z namespaces/openshift-operators/jobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059374241Z namespaces/openshift-operators/poddisruptionbudgets/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059378082Z namespaces/openshift-operators/pods/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.059929307Z namespaces/openshift-operators/pods/-describe [must-gather-g98xq] OUT 2026-01-27T19:03:58.060197035Z namespaces/openshift-operators/pods/logs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060221216Z namespaces/openshift-operators/replicaset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060225976Z namespaces/openshift-operators/routes/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060230646Z namespaces/openshift-operators/services/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060235036Z namespaces/openshift-operators/statefulset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060238956Z namespaces/openshift-operators/subscriptions/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060242746Z namespaces/openstack-operators/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.060246566Z namespaces/openstack-operators/all_resources.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.060954046Z namespaces/openstack-operators/events.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.061169423Z namespaces/openstack-operators/nad.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.06141331Z namespaces/openstack-operators/pvc.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.061600405Z namespaces/openstack-operators/buildconfig/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.061742489Z namespaces/openstack-operators/cert/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.061884193Z namespaces/openstack-operators/certificaterequests/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.061894033Z namespaces/openstack-operators/configmaps/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.061897913Z namespaces/openstack-operators/cronjobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.061901944Z namespaces/openstack-operators/daemonset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.061905684Z namespaces/openstack-operators/deployments/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.062349846Z namespaces/openstack-operators/imagestream/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.062502151Z namespaces/openstack-operators/installplans/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.062623784Z namespaces/openstack-operators/issuer/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.062751458Z namespaces/openstack-operators/jobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.062764398Z namespaces/openstack-operators/poddisruptionbudgets/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.062780839Z namespaces/openstack-operators/pods/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.063062807Z namespaces/openstack-operators/pods/-describe [must-gather-g98xq] OUT 2026-01-27T19:03:58.063251952Z namespaces/openstack-operators/pods/logs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.063420447Z namespaces/openstack-operators/replicaset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.063547711Z namespaces/openstack-operators/routes/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.063714765Z namespaces/openstack-operators/services/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.063848529Z namespaces/openstack-operators/statefulset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.064015524Z namespaces/openstack-operators/subscriptions/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.064160428Z namespaces/openstack/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.064375064Z namespaces/openstack/all_resources.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.064620521Z namespaces/openstack/events.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.064863458Z namespaces/openstack/nad.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.065115435Z namespaces/openstack/pvc.log [must-gather-g98xq] OUT 2026-01-27T19:03:58.065319941Z namespaces/openstack/buildconfig/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.065334601Z namespaces/openstack/cert/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.065339782Z namespaces/openstack/certificaterequests/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.065345012Z namespaces/openstack/configmaps/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.065349952Z namespaces/openstack/cronjobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.065744593Z namespaces/openstack/daemonset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.065899558Z namespaces/openstack/deployments/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066043822Z namespaces/openstack/imagestream/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066193946Z namespaces/openstack/installplans/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066354631Z namespaces/openstack/issuer/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066486014Z namespaces/openstack/jobs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066506775Z namespaces/openstack/poddisruptionbudgets/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066511395Z namespaces/openstack/pods/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.066865855Z namespaces/openstack/pods/-describe [must-gather-g98xq] OUT 2026-01-27T19:03:58.06703228Z namespaces/openstack/pods/logs/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.06703921Z namespaces/openstack/replicaset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.06704375Z namespaces/openstack/routes/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.06704762Z namespaces/openstack/services/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.06705154Z namespaces/openstack/statefulset/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067055661Z namespaces/openstack/subscriptions/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067059871Z network/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067675338Z network/l2advertisement.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.067889214Z network/ipaddresspools/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067899815Z network/nnce/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067904615Z network/nncp/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067909505Z network/nns/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.067914375Z nodes/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.06843996Z nodes/crc.describe [must-gather-g98xq] OUT 2026-01-27T19:03:58.068817191Z nodes/crc.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.069088139Z sos-reports/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.069100449Z sos-reports/sos-report- -> ../_all_nodes/sosreport- [must-gather-g98xq] OUT 2026-01-27T19:03:58.069369467Z sos-reports/_all_nodes/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.069382937Z webhooks/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.069387297Z webhooks/mutating/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.069391567Z webhooks/validating/ [must-gather-g98xq] OUT 2026-01-27T19:03:58.069753088Z webhooks/validating/controlplanemachineset.machine.openshift.io.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.070029715Z webhooks/validating/multus.openshift.io.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.070326454Z webhooks/validating/network-node-identity.openshift.io.yaml [must-gather-g98xq] OUT 2026-01-27T19:03:58.073739201Z [must-gather-g98xq] OUT 2026-01-27T19:03:58.073994208Z sent 1,316 bytes received 30,485,382 bytes 12,194,679.20 bytes/sec [must-gather-g98xq] OUT 2026-01-27T19:03:58.074153433Z total size is 212,693,098 speedup is 6.98 [must-gather ] OUT 2026-01-27T19:03:58.101561255Z namespace/openshift-must-gather-9xrqh deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304 ClientVersion: 4.20.10 ClusterVersion: Stable at "4.16.0" ClusterOperators: clusteroperator/machine-config is not upgradeable because One or more machine config pools are degraded, please see `oc get mcp` for further details and resolve before upgrading clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing