[must-gather      ] OUT 2025-10-11T00:42:38.834580219Z Using must-gather plug-in image: quay.io/openstack-k8s-operators/openstack-must-gather:latest
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information:
ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304
ClientVersion: 4.19.14
ClusterVersion: Stable at "4.16.0"
ClusterOperators:
	clusteroperator/authentication is not available (WellKnownAvailable: The well-known endpoint is not yet available: failed to get API server IPs: unable to find kube api server endpointLister port: &v1.Endpoints{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kubernetes", GenerateName:"", Namespace:"default", SelfLink:"", UID:"9374e9d0-f290-4faa-94c7-262a199a1d45", ResourceVersion:"43063", Generation:0, CreationTimestamp:time.Date(2024, time.June, 26, 12, 38, 3, 0, time.Local), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"endpointslice.kubernetes.io/skip-mirror":"true"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kube-apiserver", Operation:"Update", APIVersion:"v1", Time:time.Date(2025, time.August, 13, 20, 8, 43, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001f09638), Subresource:""}}}, Subsets:[]v1.EndpointSubset(nil)} (check kube-apiserver that it deploys correctly)) because All is well
	clusteroperator/kube-apiserver is progressing: NodeInstallerProgressing: 1 node is at revision 12; 0 nodes have achieved new revision 13
	clusteroperator/machine-config is degraded because Failed to resync 4.16.0 because: error required MachineConfigPool master is paused and cannot sync until it is unpaused
	clusteroperator/cloud-credential is missing
	clusteroperator/cluster-autoscaler is missing
	clusteroperator/insights is missing
	clusteroperator/monitoring is missing
	clusteroperator/storage is missing


[must-gather      ] OUT 2025-10-11T00:42:38.918146136Z namespace/openshift-must-gather-jc6hm created
[must-gather      ] OUT 2025-10-11T00:42:38.922509791Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-jx985 created
[must-gather      ] OUT 2025-10-11T00:42:39.086331498Z pod for plug-in image quay.io/openstack-k8s-operators/openstack-must-gather:latest created
[must-gather-zpjrp] OUT 2025-10-11T00:44:12.719344865Z waiting for gather to complete
[must-gather-zpjrp] OUT 2025-10-11T00:44:12.721767519Z downloading gather output
[must-gather-zpjrp] OUT 2025-10-11T00:44:12.974609476Z receiving incremental file list
[must-gather-zpjrp] OUT 2025-10-11T00:44:12.982415421Z ./
[must-gather-zpjrp] OUT 2025-10-11T00:44:12.982525004Z must-gather.tar.xz
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.070524517Z packagemanifests
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816451737Z run.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.81656003Z version
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816636872Z crd/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816644202Z csv/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816649562Z ctlplane/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816685133Z ctlplane/endpoints
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816838057Z ctlplane/services
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816876948Z ctlplane/ovn/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816882818Z ctlplane/rabbitmq/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816888218Z dbs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816893508Z namespaces/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.816899038Z namespaces/openshift-machine-api/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.81694567Z namespaces/openshift-machine-api/all_resources.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817160366Z namespaces/openshift-machine-api/events.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817407493Z namespaces/openshift-machine-api/nad.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817518415Z namespaces/openshift-machine-api/pvc.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817582067Z namespaces/openshift-machine-api/buildconfig/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817589017Z namespaces/openshift-machine-api/cert/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817594577Z namespaces/openshift-machine-api/certificaterequests/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817599908Z namespaces/openshift-machine-api/configmaps/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817605228Z namespaces/openshift-machine-api/cronjobs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817610528Z namespaces/openshift-machine-api/daemonset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817615948Z namespaces/openshift-machine-api/deployments/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817643269Z namespaces/openshift-machine-api/deployments/control-plane-machine-set-operator.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817782172Z namespaces/openshift-machine-api/deployments/machine-api-operator.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817862674Z namespaces/openshift-machine-api/imagestream/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817870125Z namespaces/openshift-machine-api/installplans/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817875365Z namespaces/openshift-machine-api/issuer/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817880715Z namespaces/openshift-machine-api/jobs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817886075Z namespaces/openshift-machine-api/pods/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817891555Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.817927606Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/control-plane-machine-set-operator-649bd778b4-tt5tw-describe
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818020538Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.81809126Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/control-plane-machine-set-operator.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818208043Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818269585Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/machine-api-operator-788b7c6b6c-ctdmb-describe
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818388868Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.81845464Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/kube-rbac-proxy.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818591193Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/machine-api-operator.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818666255Z namespaces/openshift-machine-api/replicaset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818730587Z namespaces/openshift-machine-api/replicaset/control-plane-machine-set-operator-649bd778b4.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818817939Z namespaces/openshift-machine-api/replicaset/machine-api-operator-788b7c6b6c.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818942372Z namespaces/openshift-machine-api/routes/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818950903Z namespaces/openshift-machine-api/services/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.818956783Z namespaces/openshift-machine-api/services/cluster-autoscaler-operator.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819090556Z namespaces/openshift-machine-api/services/control-plane-machine-set-operator.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819160898Z namespaces/openshift-machine-api/services/machine-api-controllers.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819268581Z namespaces/openshift-machine-api/services/machine-api-operator-machine-webhook.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819450526Z namespaces/openshift-machine-api/services/machine-api-operator-webhook.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819555068Z namespaces/openshift-machine-api/services/machine-api-operator.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819595409Z namespaces/openshift-machine-api/statefulset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819601469Z namespaces/openshift-machine-api/subscriptions/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.81960677Z namespaces/openstack-operators/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819667511Z namespaces/openstack-operators/all_resources.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819727053Z namespaces/openstack-operators/events.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819814485Z namespaces/openstack-operators/nad.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819938189Z namespaces/openstack-operators/pvc.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.81998638Z namespaces/openstack-operators/buildconfig/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819992951Z namespaces/openstack-operators/cert/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.819998481Z namespaces/openstack-operators/certificaterequests/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820003771Z namespaces/openstack-operators/configmaps/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820008921Z namespaces/openstack-operators/cronjobs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820014081Z namespaces/openstack-operators/daemonset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820019551Z namespaces/openstack-operators/deployments/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820028922Z namespaces/openstack-operators/imagestream/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820045672Z namespaces/openstack-operators/installplans/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820051262Z namespaces/openstack-operators/issuer/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820056492Z namespaces/openstack-operators/jobs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820061672Z namespaces/openstack-operators/pods/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820089473Z namespaces/openstack-operators/pods/-describe
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820127934Z namespaces/openstack-operators/pods/logs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820133794Z namespaces/openstack-operators/replicaset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820138994Z namespaces/openstack-operators/routes/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820144245Z namespaces/openstack-operators/services/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820149905Z namespaces/openstack-operators/statefulset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820155165Z namespaces/openstack-operators/subscriptions/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820160485Z namespaces/openstack/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820169135Z namespaces/openstack/all_resources.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820446082Z namespaces/openstack/events.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820577036Z namespaces/openstack/nad.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820690029Z namespaces/openstack/pvc.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.82072822Z namespaces/openstack/buildconfig/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.82073455Z namespaces/openstack/cert/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.8207403Z namespaces/openstack/certificaterequests/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.82074901Z namespaces/openstack/configmaps/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.82075456Z namespaces/openstack/cronjobs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820759921Z namespaces/openstack/daemonset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820769411Z namespaces/openstack/deployments/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820774941Z namespaces/openstack/imagestream/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820791261Z namespaces/openstack/installplans/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820796861Z namespaces/openstack/issuer/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820802212Z namespaces/openstack/jobs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820807592Z namespaces/openstack/pods/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820823122Z namespaces/openstack/pods/-describe
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820868463Z namespaces/openstack/pods/logs/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820874814Z namespaces/openstack/replicaset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820880174Z namespaces/openstack/routes/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820885554Z namespaces/openstack/services/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820890944Z namespaces/openstack/statefulset/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820896304Z namespaces/openstack/subscriptions/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820901594Z network/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820910514Z network/l2advertisement.log
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820930575Z network/ipaddresspools/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820939455Z network/nnce/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820945005Z network/nncp/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820950335Z network/nns/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820955616Z nodes/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.820968666Z nodes/crc.describe
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821195642Z nodes/crc.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821354006Z sos-reports/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821361596Z sos-reports/sos-report- -> ../_all_nodes/sosreport-
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821370786Z sos-reports/_all_nodes/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821376247Z webhooks/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821381397Z webhooks/mutating/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821386987Z webhooks/validating/
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821434458Z webhooks/validating/controlplanemachineset.machine.openshift.io.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821582532Z webhooks/validating/multus.openshift.io.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.821681654Z webhooks/validating/network-node-identity.openshift.io.yaml
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.831001779Z 
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.83102504Z sent 1,133 bytes  received 26,331,903 bytes  17,555,357.33 bytes/sec
[must-gather-zpjrp] OUT 2025-10-11T00:44:13.83103122Z total size is 177,385,815  speedup is 6.74
[must-gather      ] OUT 2025-10-11T00:44:13.862286492Z namespace/openshift-must-gather-jc6hm deleted


Reprinting Cluster State:
When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information:
ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304
ClientVersion: 4.19.14
ClusterVersion: Stable at "4.16.0"
ClusterOperators:
	clusteroperator/machine-config is degraded because Failed to resync 4.16.0 because: error required MachineConfigPool master is paused and cannot sync until it is unpaused
	clusteroperator/cloud-credential is missing
	clusteroperator/cluster-autoscaler is missing
	clusteroperator/insights is missing
	clusteroperator/monitoring is missing
	clusteroperator/storage is missing