[must-gather ] OUT 2025-11-24T00:31:35.006040785Z Using must-gather plug-in image: quay.io/openstack-k8s-operators/openstack-must-gather:latest When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304 ClientVersion: 4.20.3 ClusterVersion: Stable at "4.16.0" ClusterOperators: clusteroperator/machine-config is degraded because Failed to resync 4.16.0 because: error required MachineConfigPool master is paused and cannot sync until it is unpaused clusteroperator/openshift-controller-manager is progressing: Progressing: deployment/controller-manager: observed generation is 18, desired generation is 19. clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing [must-gather ] OUT 2025-11-24T00:31:35.069555115Z namespace/openshift-must-gather-qfszc created [must-gather ] OUT 2025-11-24T00:31:35.073818229Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-rlhvw created [must-gather ] OUT 2025-11-24T00:31:35.243904357Z pod for plug-in image quay.io/openstack-k8s-operators/openstack-must-gather:latest created [must-gather-8bjtx] OUT 2025-11-24T00:34:35.474266538Z waiting for gather to complete [must-gather-8bjtx] OUT 2025-11-24T00:34:35.477139709Z downloading gather output [must-gather-8bjtx] OUT 2025-11-24T00:34:35.715081453Z receiving incremental file list [must-gather-8bjtx] OUT 2025-11-24T00:34:35.721637754Z ./ [must-gather-8bjtx] OUT 2025-11-24T00:34:35.721785878Z must-gather.tar.xz [must-gather-8bjtx] OUT 2025-11-24T00:34:35.82527482Z packagemanifests [must-gather-8bjtx] OUT 2025-11-24T00:34:36.695174728Z run.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.695329281Z version [must-gather-8bjtx] OUT 2025-11-24T00:34:36.695390783Z crd/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.695400193Z csv/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.695467635Z csv/cluster-observability-operator.v1.3.0 [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696443199Z ctlplane/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.69649527Z ctlplane/endpoints [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696612773Z ctlplane/services [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696648114Z ctlplane/ovn/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696653384Z ctlplane/rabbitmq/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696657454Z dbs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696661454Z namespaces/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696665384Z namespaces/cert-manager/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696727756Z namespaces/cert-manager/all_resources.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696855229Z namespaces/cert-manager/events.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.696992412Z namespaces/cert-manager/nad.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697112465Z namespaces/cert-manager/pvc.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697165976Z namespaces/cert-manager/buildconfig/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697171497Z namespaces/cert-manager/cert/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697175877Z namespaces/cert-manager/certificaterequests/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697180997Z namespaces/cert-manager/configmaps/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697184687Z namespaces/cert-manager/cronjobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697188657Z namespaces/cert-manager/daemonset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697192727Z namespaces/cert-manager/deployments/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697256419Z namespaces/cert-manager/deployments/cert-manager-cainjector.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697403772Z namespaces/cert-manager/deployments/cert-manager-webhook.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697672179Z namespaces/cert-manager/deployments/cert-manager.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697756581Z namespaces/cert-manager/imagestream/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697805972Z namespaces/cert-manager/installplans/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697811052Z namespaces/cert-manager/issuer/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697815582Z namespaces/cert-manager/jobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697819812Z namespaces/cert-manager/pods/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697824173Z namespaces/cert-manager/pods/cert-manager-755d7666d5-mjtlq/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697839303Z namespaces/cert-manager/pods/cert-manager-755d7666d5-mjtlq/cert-manager-755d7666d5-mjtlq-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697932715Z namespaces/cert-manager/pods/cert-manager-755d7666d5-mjtlq/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.697997447Z namespaces/cert-manager/pods/cert-manager-755d7666d5-mjtlq/logs/cert-manager-controller.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698155881Z namespaces/cert-manager/pods/cert-manager-cainjector-6dcc74f67d-27qfj/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698211712Z namespaces/cert-manager/pods/cert-manager-cainjector-6dcc74f67d-27qfj/cert-manager-cainjector-6dcc74f67d-27qfj-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698299254Z namespaces/cert-manager/pods/cert-manager-cainjector-6dcc74f67d-27qfj/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698354946Z namespaces/cert-manager/pods/cert-manager-cainjector-6dcc74f67d-27qfj/logs/cert-manager-cainjector.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698469218Z namespaces/cert-manager/pods/cert-manager-webhook-58ffc98b58-pkhdf/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698560001Z namespaces/cert-manager/pods/cert-manager-webhook-58ffc98b58-pkhdf/cert-manager-webhook-58ffc98b58-pkhdf-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698629182Z namespaces/cert-manager/pods/cert-manager-webhook-58ffc98b58-pkhdf/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698697444Z namespaces/cert-manager/pods/cert-manager-webhook-58ffc98b58-pkhdf/logs/cert-manager-webhook.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698777566Z namespaces/cert-manager/replicaset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698835757Z namespaces/cert-manager/replicaset/cert-manager-755d7666d5.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.698963731Z namespaces/cert-manager/replicaset/cert-manager-cainjector-6dcc74f67d.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699074033Z namespaces/cert-manager/replicaset/cert-manager-webhook-58ffc98b58.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699160055Z namespaces/cert-manager/routes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699166726Z namespaces/cert-manager/services/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699230237Z namespaces/cert-manager/services/cert-manager-cainjector.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.69935013Z namespaces/cert-manager/services/cert-manager-webhook.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699475353Z namespaces/cert-manager/services/cert-manager.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699523004Z namespaces/cert-manager/statefulset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699557965Z namespaces/cert-manager/subscriptions/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699575036Z namespaces/openshift-machine-api/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699611856Z namespaces/openshift-machine-api/all_resources.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.69974562Z namespaces/openshift-machine-api/events.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.699954305Z namespaces/openshift-machine-api/nad.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700057677Z namespaces/openshift-machine-api/pvc.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700113439Z namespaces/openshift-machine-api/buildconfig/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700124479Z namespaces/openshift-machine-api/cert/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700129139Z namespaces/openshift-machine-api/certificaterequests/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700132969Z namespaces/openshift-machine-api/configmaps/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70014299Z namespaces/openshift-machine-api/cronjobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70014846Z namespaces/openshift-machine-api/daemonset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70015235Z namespaces/openshift-machine-api/deployments/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700274913Z namespaces/openshift-machine-api/deployments/control-plane-machine-set-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700334784Z namespaces/openshift-machine-api/deployments/machine-api-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700421076Z namespaces/openshift-machine-api/imagestream/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700432787Z namespaces/openshift-machine-api/installplans/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700437457Z namespaces/openshift-machine-api/issuer/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700441627Z namespaces/openshift-machine-api/jobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700445597Z namespaces/openshift-machine-api/pods/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700450867Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70056133Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/control-plane-machine-set-operator-649bd778b4-tt5tw-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700690703Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700752154Z namespaces/openshift-machine-api/pods/control-plane-machine-set-operator-649bd778b4-tt5tw/logs/control-plane-machine-set-operator.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700878108Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.700921879Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/machine-api-operator-788b7c6b6c-ctdmb-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701032171Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701092483Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/kube-rbac-proxy.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701311098Z namespaces/openshift-machine-api/pods/machine-api-operator-788b7c6b6c-ctdmb/logs/machine-api-operator.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701438721Z namespaces/openshift-machine-api/replicaset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701590495Z namespaces/openshift-machine-api/replicaset/control-plane-machine-set-operator-649bd778b4.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701734479Z namespaces/openshift-machine-api/replicaset/machine-api-operator-788b7c6b6c.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70181125Z namespaces/openshift-machine-api/routes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701822791Z namespaces/openshift-machine-api/services/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701918113Z namespaces/openshift-machine-api/services/cluster-autoscaler-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.701988515Z namespaces/openshift-machine-api/services/control-plane-machine-set-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702131198Z namespaces/openshift-machine-api/services/machine-api-controllers.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702255711Z namespaces/openshift-machine-api/services/machine-api-operator-machine-webhook.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702434916Z namespaces/openshift-machine-api/services/machine-api-operator-webhook.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702575719Z namespaces/openshift-machine-api/services/machine-api-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702635301Z namespaces/openshift-machine-api/statefulset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702643731Z namespaces/openshift-machine-api/subscriptions/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702647851Z namespaces/openshift-marketplace/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702709683Z namespaces/openshift-marketplace/all_resources.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.702821465Z namespaces/openshift-marketplace/events.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703456041Z namespaces/openshift-marketplace/nad.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703583024Z namespaces/openshift-marketplace/pvc.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703621355Z namespaces/openshift-marketplace/buildconfig/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703627125Z namespaces/openshift-marketplace/cert/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703631125Z namespaces/openshift-marketplace/certificaterequests/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703638125Z namespaces/openshift-marketplace/configmaps/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703642375Z namespaces/openshift-marketplace/cronjobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703646386Z namespaces/openshift-marketplace/daemonset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703651666Z namespaces/openshift-marketplace/deployments/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703703877Z namespaces/openshift-marketplace/deployments/marketplace-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703799299Z namespaces/openshift-marketplace/imagestream/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703805849Z namespaces/openshift-marketplace/installplans/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70381035Z namespaces/openshift-marketplace/issuer/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.7038141Z namespaces/openshift-marketplace/jobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.703852791Z namespaces/openshift-marketplace/jobs/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb692b16c.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704020325Z namespaces/openshift-marketplace/jobs/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92100b6b5.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704203519Z namespaces/openshift-marketplace/jobs/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8f8c0cf.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704352063Z namespaces/openshift-marketplace/jobs/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5e559b8.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704476256Z namespaces/openshift-marketplace/pods/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704482166Z namespaces/openshift-marketplace/pods/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704553058Z namespaces/openshift-marketplace/pods/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704683451Z namespaces/openshift-marketplace/pods/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704733802Z namespaces/openshift-marketplace/pods/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8/logs/extract.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.704840175Z namespaces/openshift-marketplace/pods/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8/logs/pull.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705136532Z namespaces/openshift-marketplace/pods/695e9552c02c72940c72621f824780f00ca58086c3badc308bf0a2eb69j8sv8/logs/util.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705204144Z namespaces/openshift-marketplace/pods/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705247655Z namespaces/openshift-marketplace/pods/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705356218Z namespaces/openshift-marketplace/pods/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705410279Z namespaces/openshift-marketplace/pods/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f/logs/extract.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705583233Z namespaces/openshift-marketplace/pods/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f/logs/pull.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705713646Z namespaces/openshift-marketplace/pods/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c92107hb5f/logs/util.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705778718Z namespaces/openshift-marketplace/pods/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705838239Z namespaces/openshift-marketplace/pods/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705941632Z namespaces/openshift-marketplace/pods/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.705991803Z namespaces/openshift-marketplace/pods/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z/logs/extract.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706145987Z namespaces/openshift-marketplace/pods/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z/logs/pull.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70625518Z namespaces/openshift-marketplace/pods/6e3e74c24700cc2bb66271d960117ff0976dc779e6a3bc37905b952e8fh2l7z/logs/util.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706315561Z namespaces/openshift-marketplace/pods/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706373853Z namespaces/openshift-marketplace/pods/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706467835Z namespaces/openshift-marketplace/pods/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706517256Z namespaces/openshift-marketplace/pods/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz/logs/extract.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70666149Z namespaces/openshift-marketplace/pods/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz/logs/pull.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706799793Z namespaces/openshift-marketplace/pods/8ed862a309935d5a1c8012df79b93f7fb46e029d4689f7f6ddcb9e7f5elmpmz/logs/util.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706863765Z namespaces/openshift-marketplace/pods/certified-operators-hmc4t/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.706925636Z namespaces/openshift-marketplace/pods/certified-operators-hmc4t/certified-operators-hmc4t-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707026609Z namespaces/openshift-marketplace/pods/certified-operators-hmc4t/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70708106Z namespaces/openshift-marketplace/pods/certified-operators-hmc4t/logs/extract-content.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707172502Z namespaces/openshift-marketplace/pods/certified-operators-hmc4t/logs/extract-utilities.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707256144Z namespaces/openshift-marketplace/pods/certified-operators-hmc4t/logs/registry-server.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707330206Z namespaces/openshift-marketplace/pods/community-operators-mmfmw/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707392478Z namespaces/openshift-marketplace/pods/community-operators-mmfmw/community-operators-mmfmw-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70748991Z namespaces/openshift-marketplace/pods/community-operators-mmfmw/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707557382Z namespaces/openshift-marketplace/pods/community-operators-mmfmw/logs/extract-content.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707655754Z namespaces/openshift-marketplace/pods/community-operators-mmfmw/logs/extract-utilities.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707739066Z namespaces/openshift-marketplace/pods/community-operators-mmfmw/logs/registry-server.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707806508Z namespaces/openshift-marketplace/pods/marketplace-operator-8b455464d-w8sgs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707868439Z namespaces/openshift-marketplace/pods/marketplace-operator-8b455464d-w8sgs/marketplace-operator-8b455464d-w8sgs-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.707964632Z namespaces/openshift-marketplace/pods/marketplace-operator-8b455464d-w8sgs/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708082255Z namespaces/openshift-marketplace/pods/marketplace-operator-8b455464d-w8sgs/logs/marketplace-operator-previous.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708240288Z namespaces/openshift-marketplace/pods/marketplace-operator-8b455464d-w8sgs/logs/marketplace-operator.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70831635Z namespaces/openshift-marketplace/pods/redhat-operators-l8lwg/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708378922Z namespaces/openshift-marketplace/pods/redhat-operators-l8lwg/redhat-operators-l8lwg-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708470244Z namespaces/openshift-marketplace/pods/redhat-operators-l8lwg/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708572797Z namespaces/openshift-marketplace/pods/redhat-operators-l8lwg/logs/extract-content.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708657729Z namespaces/openshift-marketplace/pods/redhat-operators-l8lwg/logs/extract-utilities.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708769851Z namespaces/openshift-marketplace/pods/redhat-operators-l8lwg/logs/registry-server.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708835193Z namespaces/openshift-marketplace/replicaset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708896555Z namespaces/openshift-marketplace/replicaset/marketplace-operator-8b455464d.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708986937Z namespaces/openshift-marketplace/routes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.708995257Z namespaces/openshift-marketplace/services/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709049888Z namespaces/openshift-marketplace/services/certified-operators.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709141721Z namespaces/openshift-marketplace/services/community-operators.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709277014Z namespaces/openshift-marketplace/services/marketplace-operator-metrics.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709392437Z namespaces/openshift-marketplace/services/redhat-operators.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709439488Z namespaces/openshift-marketplace/statefulset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709444628Z namespaces/openshift-marketplace/subscriptions/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709451418Z namespaces/openshift-operators/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70950908Z namespaces/openshift-operators/all_resources.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709675394Z namespaces/openshift-operators/events.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709825747Z namespaces/openshift-operators/nad.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.70993763Z namespaces/openshift-operators/pvc.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709985491Z namespaces/openshift-operators/buildconfig/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709991001Z namespaces/openshift-operators/cert/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.709996432Z namespaces/openshift-operators/certificaterequests/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710000342Z namespaces/openshift-operators/configmaps/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710004672Z namespaces/openshift-operators/cronjobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710015762Z namespaces/openshift-operators/daemonset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710019662Z namespaces/openshift-operators/deployments/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710155966Z namespaces/openshift-operators/deployments/obo-prometheus-operator-admission-webhook.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710235637Z namespaces/openshift-operators/deployments/obo-prometheus-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710382011Z namespaces/openshift-operators/deployments/observability-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710542665Z namespaces/openshift-operators/deployments/perses-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710705519Z namespaces/openshift-operators/imagestream/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710715529Z namespaces/openshift-operators/installplans/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.710773011Z namespaces/openshift-operators/installplans/install-znchq [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71114055Z namespaces/openshift-operators/issuer/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71115644Z namespaces/openshift-operators/jobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71116172Z namespaces/openshift-operators/pods/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711173351Z namespaces/openshift-operators/pods/obo-prometheus-operator-864b67f9b9-7dxvw/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711183391Z namespaces/openshift-operators/pods/obo-prometheus-operator-864b67f9b9-7dxvw/obo-prometheus-operator-864b67f9b9-7dxvw-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711306444Z namespaces/openshift-operators/pods/obo-prometheus-operator-864b67f9b9-7dxvw/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711342165Z namespaces/openshift-operators/pods/obo-prometheus-operator-864b67f9b9-7dxvw/logs/prometheus-operator.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711479418Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-dv42s/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71154792Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-dv42s/obo-prometheus-operator-admission-webhook-85c87846f8-dv42s-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711688813Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-dv42s/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711733084Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-dv42s/logs/prometheus-operator-admission-webhook.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711813286Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-phh8f/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.711846907Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-phh8f/obo-prometheus-operator-admission-webhook-85c87846f8-phh8f-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71195758Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-phh8f/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712000091Z namespaces/openshift-operators/pods/obo-prometheus-operator-admission-webhook-85c87846f8-phh8f/logs/prometheus-operator-admission-webhook.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712069493Z namespaces/openshift-operators/pods/observability-operator-65df589ff7-vmjst/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712112644Z namespaces/openshift-operators/pods/observability-operator-65df589ff7-vmjst/observability-operator-65df589ff7-vmjst-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712253817Z namespaces/openshift-operators/pods/observability-operator-65df589ff7-vmjst/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712314019Z namespaces/openshift-operators/pods/observability-operator-65df589ff7-vmjst/logs/operator.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712449192Z namespaces/openshift-operators/pods/perses-operator-574fd8d65d-x2rfs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712500783Z namespaces/openshift-operators/pods/perses-operator-574fd8d65d-x2rfs/perses-operator-574fd8d65d-x2rfs-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712606276Z namespaces/openshift-operators/pods/perses-operator-574fd8d65d-x2rfs/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712668977Z namespaces/openshift-operators/pods/perses-operator-574fd8d65d-x2rfs/logs/perses-operator.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712746809Z namespaces/openshift-operators/replicaset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712802501Z namespaces/openshift-operators/replicaset/obo-prometheus-operator-864b67f9b9.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.712924584Z namespaces/openshift-operators/replicaset/obo-prometheus-operator-admission-webhook-85c87846f8.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713099988Z namespaces/openshift-operators/replicaset/observability-operator-65df589ff7.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713333524Z namespaces/openshift-operators/replicaset/perses-operator-574fd8d65d.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713453286Z namespaces/openshift-operators/routes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713460667Z namespaces/openshift-operators/services/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713505838Z namespaces/openshift-operators/services/obo-prometheus-operator-admission-webhook-service.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713622861Z namespaces/openshift-operators/services/obo-prometheus-operator-admission-webhook.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713742294Z namespaces/openshift-operators/services/obo-prometheus-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713903078Z namespaces/openshift-operators/services/observability-operator.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713970979Z namespaces/openshift-operators/statefulset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.713976419Z namespaces/openshift-operators/subscriptions/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714053761Z namespaces/openshift-operators/subscriptions/cluster-observability-operator [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714126603Z namespaces/openstack-operators/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714190285Z namespaces/openstack-operators/all_resources.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714283427Z namespaces/openstack-operators/events.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714370169Z namespaces/openstack-operators/nad.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714491852Z namespaces/openstack-operators/pvc.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714551713Z namespaces/openstack-operators/buildconfig/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714599875Z namespaces/openstack-operators/cert/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714604965Z namespaces/openstack-operators/certificaterequests/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714608805Z namespaces/openstack-operators/configmaps/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714612545Z namespaces/openstack-operators/cronjobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714618985Z namespaces/openstack-operators/daemonset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714622975Z namespaces/openstack-operators/deployments/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714626735Z namespaces/openstack-operators/imagestream/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714630955Z namespaces/openstack-operators/installplans/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714634796Z namespaces/openstack-operators/issuer/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714641016Z namespaces/openstack-operators/jobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714644786Z namespaces/openstack-operators/pods/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714713227Z namespaces/openstack-operators/pods/-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714738608Z namespaces/openstack-operators/pods/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714744788Z namespaces/openstack-operators/replicaset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714753628Z namespaces/openstack-operators/routes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714759249Z namespaces/openstack-operators/services/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714764339Z namespaces/openstack-operators/statefulset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714775389Z namespaces/openstack-operators/subscriptions/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714779309Z namespaces/openstack/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71482905Z namespaces/openstack/all_resources.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.714934333Z namespaces/openstack/events.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715048766Z namespaces/openstack/nad.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715165549Z namespaces/openstack/pvc.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.7152158Z namespaces/openstack/buildconfig/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71522349Z namespaces/openstack/cert/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71523007Z namespaces/openstack/certificaterequests/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71523439Z namespaces/openstack/configmaps/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715245391Z namespaces/openstack/cronjobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715258761Z namespaces/openstack/daemonset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715264191Z namespaces/openstack/deployments/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715269361Z namespaces/openstack/imagestream/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715273601Z namespaces/openstack/installplans/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715278201Z namespaces/openstack/issuer/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715282391Z namespaces/openstack/jobs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715286872Z namespaces/openstack/pods/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715310212Z namespaces/openstack/pods/-describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715363993Z namespaces/openstack/pods/logs/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715368474Z namespaces/openstack/replicaset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715372254Z namespaces/openstack/routes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715376494Z namespaces/openstack/services/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715380204Z namespaces/openstack/statefulset/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715384074Z namespaces/openstack/subscriptions/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715397314Z network/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715461416Z network/l2advertisement.log [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715538718Z network/ipaddresspools/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715549688Z network/nnce/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715554058Z network/nncp/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715558388Z network/nns/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715562348Z nodes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715603449Z nodes/crc.describe [must-gather-8bjtx] OUT 2025-11-24T00:34:36.715850825Z nodes/crc.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.716008379Z sos-reports/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.716017079Z sos-reports/sos-report- -> ../_all_nodes/sosreport- [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71602182Z sos-reports/_all_nodes/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71602634Z webhooks/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71603023Z webhooks/mutating/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71603486Z webhooks/validating/ [must-gather-8bjtx] OUT 2025-11-24T00:34:36.716145013Z webhooks/validating/controlplanemachineset.machine.openshift.io.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.716331157Z webhooks/validating/multus.openshift.io.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.716507451Z webhooks/validating/network-node-identity.openshift.io.yaml [must-gather-8bjtx] OUT 2025-11-24T00:34:36.71929003Z [must-gather-8bjtx] OUT 2025-11-24T00:34:36.719351411Z sent 3,248 bytes received 31,528,343 bytes 21,021,060.67 bytes/sec [must-gather-8bjtx] OUT 2025-11-24T00:34:36.719361232Z total size is 214,384,453 speedup is 6.80 [must-gather ] OUT 2025-11-24T00:34:36.753040349Z namespace/openshift-must-gather-qfszc deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a84dabf3-edcf-4828-b6a1-f9d3a6f02304 ClientVersion: 4.20.3 ClusterVersion: Stable at "4.16.0" ClusterOperators: clusteroperator/machine-config is degraded because Failed to resync 4.16.0 because: error required MachineConfigPool master is paused and cannot sync until it is unpaused clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/insights is missing clusteroperator/monitoring is missing clusteroperator/storage is missing