[must-gather ] OUT 2025-10-07T11:15:07.464237617Z Using must-gather plug-in image: quay.io/openstack-k8s-operators/openstack-must-gather:latest When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 1f554581-b381-451f-a07a-fe0586b6d650 ClientVersion: 4.18.1 ClusterVersion: Stable at "4.18.1" ClusterOperators: clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing [must-gather ] OUT 2025-10-07T11:15:07.671617328Z namespace/openshift-must-gather-66rph created [must-gather ] OUT 2025-10-07T11:15:07.774559172Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-8phnc created [must-gather ] OUT 2025-10-07T11:15:08.610884966Z pod for plug-in image quay.io/openstack-k8s-operators/openstack-must-gather:latest created [must-gather-9jnrv] OUT 2025-10-07T11:17:44.043870414Z waiting for gather to complete [must-gather-9jnrv] OUT 2025-10-07T11:17:44.049314821Z downloading gather output [must-gather-9jnrv] OUT 2025-10-07T11:17:44.432928468Z receiving incremental file list [must-gather-9jnrv] OUT 2025-10-07T11:17:44.442472713Z ./ [must-gather-9jnrv] OUT 2025-10-07T11:17:44.44271037Z must-gather.tar.xz [must-gather-9jnrv] OUT 2025-10-07T11:17:44.593305625Z packagemanifests [must-gather-9jnrv] OUT 2025-10-07T11:17:45.593881099Z run.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594105376Z version [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594232479Z crd/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594268871Z csv/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594308082Z ctlplane/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594414705Z ctlplane/endpoints [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594567949Z ctlplane/services [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594641811Z ctlplane/ovn/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594689363Z ctlplane/rabbitmq/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594720064Z dbs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594751114Z namespaces/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.594792486Z namespaces/openshift-machine-api/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.59493462Z namespaces/openshift-machine-api/all_resources.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595146116Z namespaces/openshift-machine-api/events.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595542407Z namespaces/openshift-machine-api/nad.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595719632Z namespaces/openshift-machine-api/pvc.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595833476Z namespaces/openshift-machine-api/buildconfig/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595876247Z namespaces/openshift-machine-api/cert/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595915628Z namespaces/openshift-machine-api/certificaterequests/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.595963349Z namespaces/openshift-machine-api/configmaps/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.59599999Z namespaces/openshift-machine-api/cronjobs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596046462Z namespaces/openshift-machine-api/daemonset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596088583Z namespaces/openshift-machine-api/deployments/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596190986Z namespaces/openshift-machine-api/deployments/control-plane-machine-set-operator.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596381061Z namespaces/openshift-machine-api/deployments/machine-api-operator.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596513185Z namespaces/openshift-machine-api/imagestream/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596555326Z namespaces/openshift-machine-api/installplans/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596580057Z namespaces/openshift-machine-api/issuer/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596625668Z namespaces/openshift-machine-api/jobs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596655479Z namespaces/openshift-machine-api/pods/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596697021Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-78cbb6b69f-hwgbz/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596793783Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-78cbb6b69f-hwgbz/control-plane-machine-set-operator-78cbb6b69f-hwgbz-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.596970028Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-78cbb6b69f-hwgbz/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.597073341Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-78cbb6b69f-hwgbz/logs/control-plane-machine-set-operator-previous.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.597762731Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-78cbb6b69f-hwgbz/logs/control-plane-machine-set-operator.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.597906785Z namespaces/openshift-machine-api/pods/machine-api-operator-5694c8668f-w7rtg/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598017649Z namespaces/openshift-machine-api/pods/machine-api-operator-5694c8668f-w7rtg/machine-api-operator-5694c8668f-w7rtg-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598157003Z namespaces/openshift-machine-api/pods/machine-api-operator-5694c8668f-w7rtg/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598252265Z namespaces/openshift-machine-api/pods/machine-api-operator-5694c8668f-w7rtg/logs/kube-rbac-proxy.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598393169Z namespaces/openshift-machine-api/pods/machine-api-operator-5694c8668f-w7rtg/logs/machine-api-operator-previous.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598577005Z namespaces/openshift-machine-api/pods/machine-api-operator-5694c8668f-w7rtg/logs/machine-api-operator.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598711169Z namespaces/openshift-machine-api/replicaset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598818212Z namespaces/openshift-machine-api/replicaset/control-plane-machine-set-operator-78cbb6b69f.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.598962956Z namespaces/openshift-machine-api/replicaset/machine-api-operator-5694c8668f.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.59911717Z namespaces/openshift-machine-api/routes/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.599151041Z namespaces/openshift-machine-api/services/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.599246554Z namespaces/openshift-machine-api/services/cluster-autoscaler-operator.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.599389388Z namespaces/openshift-machine-api/services/control-plane-machine-set-operator.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.602155388Z namespaces/openshift-machine-api/services/machine-api-controllers.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.602325393Z namespaces/openshift-machine-api/services/machine-api-operator-machine-webhook.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.60255912Z namespaces/openshift-machine-api/services/machine-api-operator-webhook.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.602719754Z namespaces/openshift-machine-api/services/machine-api-operator.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.602836908Z namespaces/openshift-machine-api/statefulset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.602880809Z namespaces/openshift-machine-api/subscriptions/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.60291627Z namespaces/openshift-marketplace/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.603038783Z namespaces/openshift-marketplace/all_resources.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.603199428Z namespaces/openshift-marketplace/events.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.603751624Z namespaces/openshift-marketplace/nad.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.603910129Z namespaces/openshift-marketplace/pvc.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.603996261Z namespaces/openshift-marketplace/buildconfig/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604045873Z namespaces/openshift-marketplace/cert/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604083964Z namespaces/openshift-marketplace/certificaterequests/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604120865Z namespaces/openshift-marketplace/configmaps/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604159256Z namespaces/openshift-marketplace/cronjobs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604196797Z namespaces/openshift-marketplace/daemonset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604220918Z namespaces/openshift-marketplace/deployments/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.60431652Z namespaces/openshift-marketplace/deployments/marketplace-operator.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604438004Z namespaces/openshift-marketplace/imagestream/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604477635Z namespaces/openshift-marketplace/installplans/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604501836Z namespaces/openshift-marketplace/issuer/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604545677Z namespaces/openshift-marketplace/jobs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604575368Z namespaces/openshift-marketplace/pods/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604605999Z namespaces/openshift-marketplace/pods/certified-operators-t4pqh/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604695461Z namespaces/openshift-marketplace/pods/certified-operators-t4pqh/certified-operators-t4pqh-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604849136Z namespaces/openshift-marketplace/pods/certified-operators-t4pqh/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.604933168Z namespaces/openshift-marketplace/pods/certified-operators-t4pqh/logs/extract-content.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.605055622Z namespaces/openshift-marketplace/pods/certified-operators-t4pqh/logs/extract-utilities.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.605178655Z namespaces/openshift-marketplace/pods/certified-operators-t4pqh/logs/registry-server.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.609920182Z namespaces/openshift-marketplace/pods/community-operators-cpblc/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.610057566Z namespaces/openshift-marketplace/pods/community-operators-cpblc/community-operators-cpblc-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.610401646Z namespaces/openshift-marketplace/pods/community-operators-cpblc/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.610510399Z namespaces/openshift-marketplace/pods/community-operators-cpblc/logs/extract-content.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.610684094Z namespaces/openshift-marketplace/pods/community-operators-cpblc/logs/extract-utilities.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.610865939Z namespaces/openshift-marketplace/pods/community-operators-cpblc/logs/registry-server.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.61644483Z namespaces/openshift-marketplace/pods/marketplace-operator-79b997595-zkg7m/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.616552593Z namespaces/openshift-marketplace/pods/marketplace-operator-79b997595-zkg7m/marketplace-operator-79b997595-zkg7m-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.616672377Z namespaces/openshift-marketplace/pods/marketplace-operator-79b997595-zkg7m/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.616751679Z namespaces/openshift-marketplace/pods/marketplace-operator-79b997595-zkg7m/logs/marketplace-operator-previous.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.616927334Z namespaces/openshift-marketplace/pods/marketplace-operator-79b997595-zkg7m/logs/marketplace-operator.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.617053818Z namespaces/openshift-marketplace/pods/redhat-marketplace-g4s9t/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.617151661Z namespaces/openshift-marketplace/pods/redhat-marketplace-g4s9t/redhat-marketplace-g4s9t-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.617280604Z namespaces/openshift-marketplace/pods/redhat-marketplace-g4s9t/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.617380127Z namespaces/openshift-marketplace/pods/redhat-marketplace-g4s9t/logs/extract-content.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.617500361Z namespaces/openshift-marketplace/pods/redhat-marketplace-g4s9t/logs/extract-utilities.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.617647325Z namespaces/openshift-marketplace/pods/redhat-marketplace-g4s9t/logs/registry-server.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.618745627Z namespaces/openshift-marketplace/pods/redhat-operators-vclwv/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.61887546Z namespaces/openshift-marketplace/pods/redhat-operators-vclwv/redhat-operators-vclwv-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.619029955Z namespaces/openshift-marketplace/pods/redhat-operators-vclwv/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.619119547Z namespaces/openshift-marketplace/pods/redhat-operators-vclwv/logs/extract-content.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.619242851Z namespaces/openshift-marketplace/pods/redhat-operators-vclwv/logs/extract-utilities.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.619394805Z namespaces/openshift-marketplace/pods/redhat-operators-vclwv/logs/registry-server.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624012469Z namespaces/openshift-marketplace/replicaset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624163013Z namespaces/openshift-marketplace/replicaset/marketplace-operator-79b997595.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624294647Z namespaces/openshift-marketplace/routes/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624339318Z namespaces/openshift-marketplace/services/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624437921Z namespaces/openshift-marketplace/services/certified-operators.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624602236Z namespaces/openshift-marketplace/services/community-operators.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.62476086Z namespaces/openshift-marketplace/services/marketplace-operator-metrics.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.624960316Z namespaces/openshift-marketplace/services/redhat-marketplace.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.62511336Z namespaces/openshift-marketplace/services/redhat-operators.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.625556993Z namespaces/openshift-marketplace/statefulset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.625594224Z namespaces/openshift-marketplace/subscriptions/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.625633815Z namespaces/openshift-operators/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.625734918Z namespaces/openshift-operators/all_resources.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.625903733Z namespaces/openshift-operators/events.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626042797Z namespaces/openshift-operators/nad.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626201992Z namespaces/openshift-operators/pvc.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626287544Z namespaces/openshift-operators/buildconfig/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626335916Z namespaces/openshift-operators/cert/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626377177Z namespaces/openshift-operators/certificaterequests/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626420118Z namespaces/openshift-operators/configmaps/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626451909Z namespaces/openshift-operators/cronjobs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.62649085Z namespaces/openshift-operators/daemonset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626519691Z namespaces/openshift-operators/deployments/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626561412Z namespaces/openshift-operators/imagestream/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626592193Z namespaces/openshift-operators/installplans/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626628314Z namespaces/openshift-operators/issuer/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626661975Z namespaces/openshift-operators/jobs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626697616Z namespaces/openshift-operators/pods/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626781848Z namespaces/openshift-operators/pods/-describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626890802Z namespaces/openshift-operators/pods/logs/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626933063Z namespaces/openshift-operators/replicaset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.626971664Z namespaces/openshift-operators/routes/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627027556Z namespaces/openshift-operators/services/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627057846Z namespaces/openshift-operators/statefulset/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627084377Z namespaces/openshift-operators/subscriptions/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627116328Z network/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.62719724Z network/l2advertisement.log [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627269382Z network/ipaddresspools/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627297713Z network/nnce/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627328984Z network/nncp/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627353145Z network/nns/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627385696Z nodes/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627460408Z nodes/crc.describe [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627658544Z nodes/crc.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627826449Z sos-reports/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.62788344Z sos-reports/sos-report- -> ../_all_nodes/sosreport- [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627912161Z sos-reports/_all_nodes/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627951482Z webhooks/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.627974683Z webhooks/mutating/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.628007554Z webhooks/validating/ [must-gather-9jnrv] OUT 2025-10-07T11:17:45.628102447Z webhooks/validating/controlplanemachineset.machine.openshift.io.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.628343833Z webhooks/validating/monitoringconfigmaps.openshift.io.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.628495928Z webhooks/validating/multus.openshift.io.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.628699614Z webhooks/validating/network-node-identity.openshift.io.yaml [must-gather-9jnrv] OUT 2025-10-07T11:17:45.632249796Z [must-gather-9jnrv] OUT 2025-10-07T11:17:45.632308488Z sent 1,713 bytes received 30,281,281 bytes 20,188,662.67 bytes/sec [must-gather-9jnrv] OUT 2025-10-07T11:17:45.632335759Z total size is 202,391,503 speedup is 6.68 [must-gather ] OUT 2025-10-07T11:17:45.665386702Z namespace/openshift-must-gather-66rph deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 1f554581-b381-451f-a07a-fe0586b6d650 ClientVersion: 4.18.1 ClusterVersion: Stable at "4.18.1" ClusterOperators: clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing