[must-gather ] OUT 2025-10-01T07:14:52.078105623Z Using must-gather plug-in image: quay.io/openstack-k8s-operators/openstack-must-gather:latest When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304 ClientVersion: 4.19.13 ClusterVersion: Stable at "4.16.0" ClusterOperators: clusteroperator/authentication is not available (WellKnownAvailable: The well-known endpoint is not yet available: failed to get API server IPs: unable to find kube api server endpointLister port: &v1.Endpoints{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kubernetes", GenerateName:"", Namespace:"default", SelfLink:"", UID:"9374e9d0-f290-4faa-94c7-262a199a1d45", ResourceVersion:"42886", Generation:0, CreationTimestamp:time.Date(2024, time.June, 26, 12, 38, 3, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"endpointslice.kubernetes.io/skip-mirror":"true"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kube-apiserver", Operation:"Update", APIVersion:"v1", Time:time.Date(2025, time.August, 13, 20, 8, 43, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0027c48e8), Subresource:""}}}, Subsets:[]v1.EndpointSubset(nil)} (check kube-apiserver that it deploys correctly)) because All is well clusteroperator/kube-apiserver is progressing: NodeInstallerProgressing: 1 node is at revision 12; 0 nodes have achieved new revision 13 clusteroperator/machine-config is not upgradeable because One or more machine config pools are degraded, please see `oc get mcp` for further details and resolve before upgrading clusteroperator/network is progressing: DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 1 nodes) clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing [must-gather ] OUT 2025-10-01T07:14:52.146057358Z namespace/openshift-must-gather-xbknc created [must-gather ] OUT 2025-10-01T07:14:52.151079984Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-smp82 created [must-gather ] OUT 2025-10-01T07:14:52.327030864Z pod for plug-in image quay.io/openstack-k8s-operators/openstack-must-gather:latest created [must-gather-wvcz8] POD 2025-10-01T07:14:59.882727170Z volume percentage checker started..... [must-gather-wvcz8] POD 2025-10-01T07:14:59.904722812Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:00.086111490Z Error from server (NotFound): pods "openstackclient" not found [must-gather-wvcz8] POD 2025-10-01T07:15:00.247659917Z Will retrieve SOS reports from nodes [must-gather-wvcz8] POD 2025-10-01T07:15:00.408002850Z error: the server doesn't have a resource type "openstackdataplanenodesets" [must-gather-wvcz8] POD 2025-10-01T07:15:01.085669329Z Gathering CRDs [must-gather-wvcz8] POD 2025-10-01T07:15:01.452840466Z Gathering CRs [must-gather-wvcz8] POD 2025-10-01T07:15:01.647784431Z Writing validating webhooks [must-gather-wvcz8] POD 2025-10-01T07:15:01.812996315Z Writing mutating webhooks [must-gather-wvcz8] POD 2025-10-01T07:15:04.916342427Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:07.600105792Z Gathering secrets in namespace openstack [must-gather-wvcz8] POD 2025-10-01T07:15:09.944838236Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:12.101021209Z Gathering csv [must-gather-wvcz8] POD 2025-10-01T07:15:12.385254905Z Gathering subscription [must-gather-wvcz8] POD 2025-10-01T07:15:12.559894693Z Gathering installplan [must-gather-wvcz8] POD 2025-10-01T07:15:13.012446686Z No resources found in openstack namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:13.077931828Z No resources found in openstack namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:13.243076751Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-wvcz8] POD 2025-10-01T07:15:13.265456053Z No resources found in openstack namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:13.281063133Z No resources found in openstack namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:14.962573916Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:19.414850535Z Gathering secrets in namespace openstack-operators [must-gather-wvcz8] POD 2025-10-01T07:15:19.976799732Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:23.408627338Z Gathering csv [must-gather-wvcz8] POD 2025-10-01T07:15:23.598512397Z Gathering subscription [must-gather-wvcz8] POD 2025-10-01T07:15:23.735995165Z Gathering installplan [must-gather-wvcz8] POD 2025-10-01T07:15:24.130279382Z No resources found in openstack-operators namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:24.153967649Z No resources found in openstack-operators namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:24.297770237Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-wvcz8] POD 2025-10-01T07:15:24.323627613Z No resources found in openstack-operators namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:24.344072212Z No resources found in openstack-operators namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:24.992929297Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:26.646505989Z Gathering secrets in namespace baremetal-operator-system [must-gather-wvcz8] POD 2025-10-01T07:15:30.015402134Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:31.214078689Z Gathering secrets in namespace openshift-machine-api [must-gather-wvcz8] POD 2025-10-01T07:15:35.030436592Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:35.516658151Z Gathering csv [must-gather-wvcz8] POD 2025-10-01T07:15:35.669315828Z Gathering subscription [must-gather-wvcz8] POD 2025-10-01T07:15:35.804879575Z Gathering installplan [must-gather-wvcz8] POD 2025-10-01T07:15:36.397189708Z Dump logs for control-plane-machine-set-operator from control-plane-machine-set-operator-649bd778b4-tt5tw pod [must-gather-wvcz8] POD 2025-10-01T07:15:36.415845470Z No resources found in openshift-machine-api namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:36.433685660Z Dump logs for kube-rbac-proxy from machine-api-operator-788b7c6b6c-ctdmb pod [must-gather-wvcz8] POD 2025-10-01T07:15:36.510043545Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-wvcz8] POD 2025-10-01T07:15:36.548341884Z Dump logs for machine-api-operator from machine-api-operator-788b7c6b6c-ctdmb pod [must-gather-wvcz8] POD 2025-10-01T07:15:36.709462459Z Dump services: cluster-autoscaler-operator [must-gather-wvcz8] POD 2025-10-01T07:15:36.713752664Z Dump services: control-plane-machine-set-operator [must-gather-wvcz8] POD 2025-10-01T07:15:36.723811624Z Dump services: machine-api-controllers [must-gather-wvcz8] POD 2025-10-01T07:15:36.729015125Z Dump services: machine-api-operator [must-gather-wvcz8] POD 2025-10-01T07:15:36.733672100Z Dump services: machine-api-operator-machine-webhook [must-gather-wvcz8] POD 2025-10-01T07:15:36.867454559Z Dump services: machine-api-operator-webhook [must-gather-wvcz8] POD 2025-10-01T07:15:37.307319421Z Dump replicaset: control-plane-machine-set-operator-649bd778b4 [must-gather-wvcz8] POD 2025-10-01T07:15:37.311384181Z Dump replicaset: machine-api-operator-788b7c6b6c [must-gather-wvcz8] POD 2025-10-01T07:15:37.515380149Z Dump deployments: control-plane-machine-set-operator [must-gather-wvcz8] POD 2025-10-01T07:15:37.519210501Z Dump deployments: machine-api-operator [must-gather-wvcz8] POD 2025-10-01T07:15:40.041364839Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:42.666630425Z Gathering secrets in namespace cert-manager [must-gather-wvcz8] POD 2025-10-01T07:15:45.055364749Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:46.660879169Z Gathering csv [must-gather-wvcz8] POD 2025-10-01T07:15:46.800896584Z Gathering subscription [must-gather-wvcz8] POD 2025-10-01T07:15:46.951840504Z Gathering installplan [must-gather-wvcz8] POD 2025-10-01T07:15:47.402262109Z No resources found in cert-manager namespace. [must-gather-wvcz8] POD 2025-10-01T07:15:47.441377791Z Dump logs for cert-manager-controller from cert-manager-67c98b89c8-kw5xf pod [must-gather-wvcz8] POD 2025-10-01T07:15:47.480461902Z Dump logs for cert-manager-cainjector from cert-manager-cainjector-5c5695d979-nqkzk pod [must-gather-wvcz8] POD 2025-10-01T07:15:47.620421666Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather-wvcz8] POD 2025-10-01T07:15:47.688409714Z Dump logs for cert-manager-webhook from cert-manager-webhook-7f9f8648b9-vwwr6 pod [must-gather-wvcz8] POD 2025-10-01T07:15:48.059891376Z Dump services: cert-manager [must-gather-wvcz8] POD 2025-10-01T07:15:48.064369836Z Dump services: cert-manager-webhook [must-gather-wvcz8] POD 2025-10-01T07:15:48.622760154Z Dump replicaset: cert-manager-67c98b89c8 [must-gather-wvcz8] POD 2025-10-01T07:15:48.626244007Z Dump replicaset: cert-manager-cainjector-5c5695d979 [must-gather-wvcz8] POD 2025-10-01T07:15:48.630933123Z Dump replicaset: cert-manager-webhook-7f9f8648b9 [must-gather-wvcz8] POD 2025-10-01T07:15:48.807324657Z Dump deployments: cert-manager [must-gather-wvcz8] POD 2025-10-01T07:15:48.815255540Z Dump deployments: cert-manager-cainjector [must-gather-wvcz8] POD 2025-10-01T07:15:48.819093274Z Dump deployments: cert-manager-webhook [must-gather-wvcz8] POD 2025-10-01T07:15:50.082219025Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:50.363619784Z Gathering secrets in namespace openshift-nmstate [must-gather-wvcz8] POD 2025-10-01T07:15:51.194306404Z Gathering secrets in namespace openshift-operators-redhat [must-gather-wvcz8] POD 2025-10-01T07:15:52.113308151Z Gathering secrets in namespace openshift-logging [must-gather-wvcz8] POD 2025-10-01T07:15:53.390287915Z Gathering secrets in namespace metallb-system [must-gather-wvcz8] POD 2025-10-01T07:15:54.275557084Z error: the server doesn't have a resource type "nncp" [must-gather-wvcz8] POD 2025-10-01T07:15:54.519696170Z error: the server doesn't have a resource type "nnce" [must-gather-wvcz8] POD 2025-10-01T07:15:55.058905882Z error: the server doesn't have a resource type "nns" [must-gather-wvcz8] POD 2025-10-01T07:15:55.094016416Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:15:55.347141264Z error: the server doesn't have a resource type "ipaddresspools" [must-gather-wvcz8] POD 2025-10-01T07:15:55.599767869Z error: the server doesn't have a resource type "l2advertisement" [must-gather-wvcz8] POD 2025-10-01T07:15:55.617639059Z Gather ctlplane service info: openstack [must-gather-wvcz8] POD 2025-10-01T07:15:55.635003516Z Gather ctlplane service info: ovn [must-gather-wvcz8] POD 2025-10-01T07:15:55.820697880Z Error from server (NotFound): pods "openstackclient" not found [must-gather-wvcz8] POD 2025-10-01T07:15:55.849208597Z Pod ovsdbserver-nb-0 not found in namespace openstack, skipping OVN NB database collection [must-gather-wvcz8] POD 2025-10-01T07:15:55.861459577Z Error from server (NotFound): pods "openstackclient" not found [must-gather-wvcz8] POD 2025-10-01T07:15:56.054315563Z Pod ovsdbserver-sb-0 not found in namespace openstack, skipping OVN SB database collection [must-gather-wvcz8] POD 2025-10-01T07:15:56.065076223Z Gather ctlplane service info: rabbitmq [must-gather-wvcz8] POD 2025-10-01T07:15:56.375206054Z Error from server (NotFound): pods "openstackclient" not found [must-gather-wvcz8] POD 2025-10-01T07:15:56.614461628Z error: the server doesn't have a resource type "openstackcontrolplane" [must-gather-wvcz8] POD 2025-10-01T07:15:58.696809043Z tar: Removing leading `/' from member names [must-gather-wvcz8] POD 2025-10-01T07:16:00.143264795Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:05.160172353Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:10.174999705Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:15.210649458Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:20.221467553Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:25.237753204Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:30.255314481Z volume usage percentage 0 [must-gather-wvcz8] POD 2025-10-01T07:16:33.150184918Z The /must-gather/must-gather.tar.xz now can be attached to the support case. [must-gather-wvcz8] OUT 2025-10-01T07:16:34.177939548Z waiting for gather to complete [must-gather-wvcz8] OUT 2025-10-01T07:16:34.180851436Z downloading gather output [must-gather-wvcz8] OUT 2025-10-01T07:16:34.455520974Z receiving incremental file list [must-gather-wvcz8] OUT 2025-10-01T07:16:34.469193172Z ./ [must-gather-wvcz8] OUT 2025-10-01T07:16:34.469706656Z must-gather.tar.xz [must-gather-wvcz8] OUT 2025-10-01T07:16:34.56579177Z packagemanifests [must-gather-wvcz8] OUT 2025-10-01T07:16:35.358894439Z run.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359005942Z version [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359077554Z crd/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359106584Z csv/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359113064Z ctlplane/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359186086Z ctlplane/endpoints [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359293629Z ctlplane/services [must-gather-wvcz8] OUT 2025-10-01T07:16:35.35932835Z ctlplane/ovn/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359350711Z ctlplane/rabbitmq/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359369081Z dbs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359386772Z namespaces/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359420563Z namespaces/cert-manager/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359498555Z namespaces/cert-manager/all_resources.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359606698Z namespaces/cert-manager/events.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359787323Z namespaces/cert-manager/nad.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359943337Z namespaces/cert-manager/pvc.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.359994378Z namespaces/cert-manager/buildconfig/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360017519Z namespaces/cert-manager/cert/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360035709Z namespaces/cert-manager/certificaterequests/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36005726Z namespaces/cert-manager/configmaps/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360081371Z namespaces/cert-manager/cronjobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360087841Z namespaces/cert-manager/daemonset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360092851Z namespaces/cert-manager/deployments/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360149242Z namespaces/cert-manager/deployments/cert-manager-cainjector.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360329607Z namespaces/cert-manager/deployments/cert-manager-webhook.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36044662Z namespaces/cert-manager/deployments/cert-manager.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360533513Z namespaces/cert-manager/imagestream/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360558423Z namespaces/cert-manager/installplans/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360564834Z namespaces/cert-manager/issuer/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360569844Z namespaces/cert-manager/jobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360700777Z namespaces/cert-manager/pods/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360708248Z namespaces/cert-manager/pods/cert-manager-67c98b89c8-kw5xf/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360715738Z namespaces/cert-manager/pods/cert-manager-67c98b89c8-kw5xf/cert-manager-67c98b89c8-kw5xf-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360770799Z namespaces/cert-manager/pods/cert-manager-67c98b89c8-kw5xf/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.360820081Z namespaces/cert-manager/pods/cert-manager-67c98b89c8-kw5xf/logs/cert-manager-controller.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361224481Z namespaces/cert-manager/pods/cert-manager-cainjector-5c5695d979-nqkzk/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361277333Z namespaces/cert-manager/pods/cert-manager-cainjector-5c5695d979-nqkzk/cert-manager-cainjector-5c5695d979-nqkzk-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361377906Z namespaces/cert-manager/pods/cert-manager-cainjector-5c5695d979-nqkzk/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361451548Z namespaces/cert-manager/pods/cert-manager-cainjector-5c5695d979-nqkzk/logs/cert-manager-cainjector-previous.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361631743Z namespaces/cert-manager/pods/cert-manager-cainjector-5c5695d979-nqkzk/logs/cert-manager-cainjector.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361702384Z namespaces/cert-manager/pods/cert-manager-webhook-7f9f8648b9-vwwr6/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361819188Z namespaces/cert-manager/pods/cert-manager-webhook-7f9f8648b9-vwwr6/cert-manager-webhook-7f9f8648b9-vwwr6-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.361939531Z namespaces/cert-manager/pods/cert-manager-webhook-7f9f8648b9-vwwr6/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362040534Z namespaces/cert-manager/pods/cert-manager-webhook-7f9f8648b9-vwwr6/logs/cert-manager-webhook-previous.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362191568Z namespaces/cert-manager/pods/cert-manager-webhook-7f9f8648b9-vwwr6/logs/cert-manager-webhook.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36229464Z namespaces/cert-manager/replicaset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362373383Z namespaces/cert-manager/replicaset/cert-manager-67c98b89c8.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362509316Z namespaces/cert-manager/replicaset/cert-manager-cainjector-5c5695d979.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362613309Z namespaces/cert-manager/replicaset/cert-manager-webhook-7f9f8648b9.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362699411Z namespaces/cert-manager/routes/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362726312Z namespaces/cert-manager/services/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362806084Z namespaces/cert-manager/services/cert-manager-webhook.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362959598Z namespaces/cert-manager/services/cert-manager.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.362994989Z namespaces/cert-manager/statefulset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36301657Z namespaces/cert-manager/subscriptions/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36303949Z namespaces/openshift-machine-api/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363126753Z namespaces/openshift-machine-api/all_resources.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363246116Z namespaces/openshift-machine-api/events.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363467832Z namespaces/openshift-machine-api/nad.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363582035Z namespaces/openshift-machine-api/pvc.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363657397Z namespaces/openshift-machine-api/buildconfig/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363679058Z namespaces/openshift-machine-api/cert/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363696888Z namespaces/openshift-machine-api/certificaterequests/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363719549Z namespaces/openshift-machine-api/configmaps/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36374321Z namespaces/openshift-machine-api/cronjobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36375041Z namespaces/openshift-machine-api/daemonset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.3637554Z namespaces/openshift-machine-api/deployments/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363812171Z namespaces/openshift-machine-api/deployments/control-plane-machine-set-operator.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.363976536Z namespaces/openshift-machine-api/deployments/machine-api-operator.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364105979Z namespaces/openshift-machine-api/imagestream/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36411352Z namespaces/openshift-machine-api/installplans/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36412399Z namespaces/openshift-machine-api/issuer/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.3641295Z namespaces/openshift-machine-api/jobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.3641349Z namespaces/openshift-machine-api/pods/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36413988Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36414776Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/control-plane-machine-set-operator-649bd778b4-tt5tw-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364283314Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364289984Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/control-plane-machine-set-operator.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364427538Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364465389Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/machine-api-operator-788b7c6b6c-ctdmb-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364578522Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364668504Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/kube-rbac-proxy.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364801298Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/machine-api-operator.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.364945732Z namespaces/openshift-machine-api/replicaset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.365044925Z namespaces/openshift-machine-api/replicaset/control-plane-machine-set-operator-649bd778b4.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.365196569Z namespaces/openshift-machine-api/replicaset/machine-api-operator-788b7c6b6c.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36523947Z namespaces/openshift-machine-api/routes/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36526156Z namespaces/openshift-machine-api/services/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.365365563Z namespaces/openshift-machine-api/services/cluster-autoscaler-operator.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.365434375Z namespaces/openshift-machine-api/services/control-plane-machine-set-operator.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.365560399Z namespaces/openshift-machine-api/services/machine-api-controllers.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.365736123Z namespaces/openshift-machine-api/services/machine-api-operator-machine-webhook.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.3659964Z namespaces/openshift-machine-api/services/machine-api-operator-webhook.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366091223Z namespaces/openshift-machine-api/services/machine-api-operator.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366194536Z namespaces/openshift-machine-api/statefulset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366202496Z namespaces/openshift-machine-api/subscriptions/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366207716Z namespaces/openstack-operators/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366233137Z namespaces/openstack-operators/all_resources.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36634298Z namespaces/openstack-operators/events.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366452013Z namespaces/openstack-operators/nad.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366536615Z namespaces/openstack-operators/pvc.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366599287Z namespaces/openstack-operators/buildconfig/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366625307Z namespaces/openstack-operators/cert/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366632307Z namespaces/openstack-operators/certificaterequests/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366642448Z namespaces/openstack-operators/configmaps/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366657438Z namespaces/openstack-operators/cronjobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366662628Z namespaces/openstack-operators/daemonset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366693899Z namespaces/openstack-operators/deployments/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366699409Z namespaces/openstack-operators/imagestream/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366704939Z namespaces/openstack-operators/installplans/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36671007Z namespaces/openstack-operators/issuer/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36671502Z namespaces/openstack-operators/jobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36673739Z namespaces/openstack-operators/pods/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366761621Z namespaces/openstack-operators/pods/-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366785572Z namespaces/openstack-operators/pods/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366791002Z namespaces/openstack-operators/replicaset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366796152Z namespaces/openstack-operators/routes/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366801242Z namespaces/openstack-operators/services/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366808982Z namespaces/openstack-operators/statefulset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366814332Z namespaces/openstack-operators/subscriptions/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366819293Z namespaces/openstack/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366859534Z namespaces/openstack/all_resources.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366938096Z namespaces/openstack/events.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.366989847Z namespaces/openstack/nad.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36708696Z namespaces/openstack/pvc.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367117561Z namespaces/openstack/buildconfig/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367128481Z namespaces/openstack/cert/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367133581Z namespaces/openstack/certificaterequests/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367141301Z namespaces/openstack/configmaps/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367146461Z namespaces/openstack/cronjobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367151712Z namespaces/openstack/daemonset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367156992Z namespaces/openstack/deployments/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367162102Z namespaces/openstack/imagestream/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367177622Z namespaces/openstack/installplans/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367183272Z namespaces/openstack/issuer/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367188693Z namespaces/openstack/jobs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367193813Z namespaces/openstack/pods/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367201723Z namespaces/openstack/pods/-describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367210943Z namespaces/openstack/pods/logs/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367215963Z namespaces/openstack/replicaset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367223113Z namespaces/openstack/routes/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367228154Z namespaces/openstack/services/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367233094Z namespaces/openstack/statefulset/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367238184Z namespaces/openstack/subscriptions/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367244264Z network/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367266545Z network/l2advertisement.log [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367302216Z network/ipaddresspools/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367307906Z network/nnce/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367313016Z network/nncp/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367318026Z network/nns/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367335817Z nodes/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367344417Z nodes/crc.describe [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367498361Z nodes/crc.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367599894Z sos-reports/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367605934Z sos-reports/sos-report- -> ../_all_nodes/sosreport- [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367611054Z sos-reports/_all_nodes/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367616114Z webhooks/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367621084Z webhooks/mutating/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367642725Z webhooks/mutating/cert-manager-webhook.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367691126Z webhooks/validating/ [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367741067Z webhooks/validating/cert-manager-webhook.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.36784119Z webhooks/validating/controlplanemachineset.machine.openshift.io.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.367948903Z webhooks/validating/multus.openshift.io.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.368039986Z webhooks/validating/network-node-identity.openshift.io.yaml [must-gather-wvcz8] OUT 2025-10-01T07:16:35.372178397Z [must-gather-wvcz8] OUT 2025-10-01T07:16:35.372195918Z sent 1,643 bytes received 25,327,888 bytes 16,886,354.00 bytes/sec [must-gather-wvcz8] OUT 2025-10-01T07:16:35.372201218Z total size is 174,726,996 speedup is 6.90 [must-gather ] OUT 2025-10-01T07:16:35.402610979Z namespace/openshift-must-gather-xbknc deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304 ClientVersion: 4.19.13 ClusterVersion: Stable at "4.16.0" ClusterOperators: clusteroperator/machine-config is not upgradeable because One or more machine config pools are degraded, please see `oc get mcp` for further details and resolve before upgrading clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing