2026-03-18T13:38:51.444969955Z volume percentage checker started.....
2026-03-18T13:38:51.467313699Z volume usage percentage 0
2026-03-18T13:38:51.812348343Z Gathering data for ns/openshift-cluster-version...
2026-03-18T13:38:54.132689646Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T13:38:54.450993651Z Gathering data for ns/default...
2026-03-18T13:38:54.886126971Z Gathering data for ns/openshift...
2026-03-18T13:38:55.776877329Z Waiting on subprocesses to finish execution.
2026-03-18T13:38:55.782430949Z INFO: Gathering HAProxy config files
2026-03-18T13:38:55.785414714Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-03-18T13:38:55.788430031Z INFO: Collecting host service logs for crio
2026-03-18T13:38:55.788846083Z INFO: Collecting host service logs for kubelet
2026-03-18T13:38:55.789366828Z INFO: Collecting host service logs for rpm-ostreed
2026-03-18T13:38:55.789446631Z INFO: Collecting host service logs for ostree-finalize-staged
2026-03-18T13:38:55.789735479Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-03-18T13:38:55.790025627Z INFO: Collecting host service logs for machine-config-daemon-host
2026-03-18T13:38:55.790310865Z INFO: Collecting host service logs for NetworkManager
2026-03-18T13:38:55.790578733Z INFO: Collecting host service logs for openvswitch
2026-03-18T13:38:55.790851241Z INFO: Collecting host service logs for ovs-configuration
2026-03-18T13:38:55.791197011Z INFO: Collecting host service logs for ovsdb-server
2026-03-18T13:38:55.791489329Z INFO: Collecting host service logs for ovs-vswitchd
2026-03-18T13:38:55.791832879Z INFO: Waiting for worker host service log collection to complete ...
2026-03-18T13:38:55.794007102Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-03-18T13:38:55.803613849Z Gathering data for ns/kube-system...
2026-03-18T13:38:55.843023523Z INFO: Waiting for node performance related collection to complete ...
2026-03-18T13:38:56.280150040Z No resources found
2026-03-18T13:38:56.339871640Z INFO: "sriov-network-operator" not detected. Skipping.
2026-03-18T13:38:56.408185377Z INFO: Found 1 replicas - prometheus-k8s-0
2026-03-18T13:38:56.499024862Z INFO: Collecting Insights Archives from insights-operator-68bf6ff9d6-ckwz8
2026-03-18T13:38:56.669845911Z INFO: Started  downloading kube-apiserver/termination.log from master-0
2026-03-18T13:38:56.696340754Z volume usage percentage 0
2026-03-18T13:38:56.920473328Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-03-18T13:38:56.920473328Z INFO: on-disk MachineConfig config collection complete.
2026-03-18T13:38:57.088482126Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-03-18T13:38:57.207776711Z Getting information from pod "etcd-master-0", container "etcdctl"
2026-03-18T13:38:57.207776711Z Using endpoints: https://192.168.32.10:2379
2026-03-18T13:38:57.211851318Z INFO: Starting getting etcd information
2026-03-18T13:38:57.211851318Z INFO: Getting etcdctl member list
2026-03-18T13:38:57.212045554Z INFO: Getting etcdctl endpoint status
2026-03-18T13:38:57.212223829Z INFO: Getting etcdctl endpoint health
2026-03-18T13:38:57.212478966Z INFO: Getting etcdctl alarm list
2026-03-18T13:38:57.212803805Z INFO: Getting object count
2026-03-18T13:38:57.214359990Z INFO: Waiting for etcd info collection to complete ...
2026-03-18T13:38:57.265033509Z No resources found
2026-03-18T13:38:57.336213629Z INFO: Finished downloading kube-apiserver/termination.log from master-0
2026-03-18T13:38:57.367034036Z Wrote inspect data to must-gather.
2026-03-18T13:38:57.528210247Z Wrote inspect data to must-gather.
2026-03-18T13:38:57.551643562Z Gathering data for ns/metallb-system...
2026-03-18T13:38:57.631546673Z Gathering data for ns/openshift-etcd...
2026-03-18T13:38:57.633343975Z INFO: Worker host service log collection to complete.
2026-03-18T13:38:57.659025184Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-03-18T13:38:57.763917904Z INFO: Waiting for HAProxy config collection to complete ...
2026-03-18T13:38:57.890256932Z Gathering data for ns/openshift-nmstate...
2026-03-18T13:38:57.931704576Z Wrote inspect data to must-gather.
2026-03-18T13:38:57.948567881Z receiving incremental file list
2026-03-18T13:38:57.970466952Z insights-operator/
2026-03-18T13:38:57.970541684Z insights-operator/insights-2026-03-18-131158.tar.gz
2026-03-18T13:38:58.013274724Z 
2026-03-18T13:38:58.013274724Z sent 47 bytes  received 169,645 bytes  113,128.00 bytes/sec
2026-03-18T13:38:58.013274724Z total size is 169,447  speedup is 1.00
2026-03-18T13:38:58.039256523Z Wrote inspect data to must-gather.
2026-03-18T13:38:58.092560717Z Wrote inspect data to must-gather.
2026-03-18T13:38:58.227895274Z Wrote inspect data to must-gather.
2026-03-18T13:38:58.280140089Z Wrote inspect data to must-gather.
2026-03-18T13:38:58.427222994Z INFO: Done collecting etcd information
2026-03-18T13:38:58.458003190Z tar: Removing leading `/' from member names
2026-03-18T13:38:58.468572514Z INFO: HAProxy config collection complete.
2026-03-18T13:38:58.474678910Z INFO: Getting rules from prometheus-k8s-0
2026-03-18T13:38:58.669612393Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T13:38:58.773682869Z INFO: Collecting /debug/api_priority_and_fairness endpoint from kube-apiserver-master-0
2026-03-18T13:38:58.980750991Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T13:38:59.040561134Z W0318 13:38:59.039861    1264 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-03-18T13:38:59.049920043Z Wrote inspect data to must-gather.
2026-03-18T13:38:59.080792812Z Gathering data for ns/openshift-config...
2026-03-18T13:38:59.119095515Z Wrote inspect data to must-gather.
2026-03-18T13:38:59.274614483Z INFO: Getting status/config from prometheus-k8s-0
2026-03-18T13:38:59.347716708Z W0318 13:38:59.347364     946 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-03-18T13:38:59.392198299Z Wrote inspect data to must-gather.
2026-03-18T13:38:59.508903229Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T13:38:59.743316709Z Gathering data for ns/assisted-installer...
2026-03-18T13:38:59.744983997Z Wrote inspect data to must-gather.
2026-03-18T13:38:59.765802367Z Wrote inspect data to must-gather.
2026-03-18T13:38:59.821150760Z Wrote inspect data to must-gather.
2026-03-18T13:39:00.226134532Z Gathering data for ns/openshift-config-managed...
2026-03-18T13:39:00.369179651Z Wrote inspect data to must-gather.
2026-03-18T13:39:00.646711472Z INFO: Getting status/flags from prometheus-k8s-0
2026-03-18T13:39:00.770064804Z Wrote inspect data to must-gather.
2026-03-18T13:39:00.902331052Z Wrote inspect data to must-gather.
2026-03-18T13:39:00.997147563Z INFO: INTERCONNECT MODE
2026-03-18T13:39:00.997147563Z INFO: Gathering ovn-kubernetes DBs
2026-03-18T13:39:01.024839110Z Wrote inspect data to must-gather.
2026-03-18T13:39:01.087998639Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-03-18T13:39:01.169426413Z Wrote inspect data to must-gather.
2026-03-18T13:39:01.361515634Z INFO: Gathering OVN_Northbound from ovnkube-node-pfs29...
2026-03-18T13:39:01.361515634Z INFO: Gathering OVN_Southbound from ovnkube-node-pfs29...
2026-03-18T13:39:01.374998553Z Wrote inspect data to must-gather.
2026-03-18T13:39:01.449543339Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-03-18T13:39:01.573119087Z Gathering data for ns/openshift-authentication...
2026-03-18T13:39:01.625780774Z tar: Removing leading `/' from member names
2026-03-18T13:39:01.653125181Z Wrote inspect data to must-gather.
2026-03-18T13:39:01.662402468Z tar: Removing leading `/' from member names
2026-03-18T13:39:01.745556493Z Wrote inspect data to must-gather.
2026-03-18T13:39:01.767515695Z volume usage percentage 0
2026-03-18T13:39:01.866356001Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-03-18T13:39:01.947439126Z INFO: Waiting for network log collection to complete ...
2026-03-18T13:39:01.947439126Z INFO: Waiting for ovnk database copies to complete ...
2026-03-18T13:39:01.947968671Z INFO: Copying ovnk databases complete.
2026-03-18T13:39:01.950381161Z 2.1M	must-gather/network_logs/ovnk_database_store
2026-03-18T13:39:01.954749686Z ovnk_database_store/
2026-03-18T13:39:01.954749686Z ovnk_database_store/ovnkube-node-pfs29_nbdb
2026-03-18T13:39:01.992666598Z Wrote inspect data to must-gather.
2026-03-18T13:39:02.013476967Z ovnk_database_store/ovnkube-node-pfs29_sbdb
2026-03-18T13:39:02.098239458Z INFO: Network log collection complete.
2026-03-18T13:39:02.602540668Z Wrote inspect data to must-gather.
2026-03-18T13:39:02.669705832Z Wrote inspect data to must-gather.
2026-03-18T13:39:02.735075684Z INFO: Getting status from alertmanager-main-0
2026-03-18T13:39:02.909776965Z Wrote inspect data to must-gather.
2026-03-18T13:39:02.963907913Z Gathering data for ns/openshift-authentication-operator...
2026-03-18T13:39:03.184530666Z Wrote inspect data to must-gather.
2026-03-18T13:39:03.307840777Z Wrote inspect data to must-gather.
2026-03-18T13:39:03.376700039Z Wrote inspect data to must-gather.
2026-03-18T13:39:03.530689664Z Wrote inspect data to must-gather.
2026-03-18T13:39:03.785795669Z Gathering data for ns/openshift-ingress...
2026-03-18T13:39:03.849044510Z Wrote inspect data to must-gather.
2026-03-18T13:39:03.956681280Z Wrote inspect data to must-gather.
2026-03-18T13:39:04.240837002Z Wrote inspect data to must-gather.
2026-03-18T13:39:04.311110415Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa5e782406f71c048b1ac3a4bf5d1227ff4be81111114083ad4c7a209c6bfb5a
2026-03-18T13:39:04.382499871Z Gathering data for ns/openshift-oauth-apiserver...
2026-03-18T13:39:04.520345730Z daemonset.apps/perf-node-gather-daemonset created
2026-03-18T13:39:04.766683693Z Wrote inspect data to must-gather.
2026-03-18T13:39:04.812416450Z Waiting for performance profile collector pods to become ready: 1
2026-03-18T13:39:05.070724208Z Gathering data for ns/openshift-machine-api...
2026-03-18T13:39:05.420617143Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/frr.conf: No such file or directory
2026-03-18T13:39:05.420705176Z /usr/bin/gather_metallb: line 53: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/reloader/reloader/logs/reloader.pid: No such file or directory
2026-03-18T13:39:05.421311883Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/frr.log: No such file or directory
2026-03-18T13:39:05.421907800Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/daemons: No such file or directory
2026-03-18T13:39:05.422601800Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/vtysh.conf: No such file or directory
2026-03-18T13:39:05.422891389Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:05.664180296Z Error from server (BadRequest): container frr is not valid for pod speaker-qbx7x
2026-03-18T13:39:05.673333390Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:05.673472344Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:05.899352387Z Error from server (BadRequest): container frr is not valid for pod speaker-qbx7x
2026-03-18T13:39:05.913174175Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:05.915163452Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:06.110431225Z Waiting for performance profile collector pods to become ready: 2
2026-03-18T13:39:06.204043290Z Error from server (BadRequest): container frr is not valid for pod speaker-qbx7x
2026-03-18T13:39:06.218667872Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:06.218789165Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:06.485998149Z Error from server (BadRequest): container frr is not valid for pod speaker-qbx7x
2026-03-18T13:39:06.493364801Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:06.493364801Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:06.608549968Z Gathering data for ns/openshift-cloud-controller-manager-operator...
2026-03-18T13:39:06.768007270Z Error from server (BadRequest): container frr is not valid for pod speaker-qbx7x
2026-03-18T13:39:06.786657027Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-qbx7x/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T13:39:06.812118730Z volume usage percentage 0
2026-03-18T13:39:07.677065625Z Waiting for performance profile collector pods to become ready: 3
2026-03-18T13:39:07.677065625Z Gathering data for ns/openshift-cloud-controller-manager...
2026-03-18T13:39:08.318355361Z Gathering data for ns/openshift-cloud-credential-operator...
2026-03-18T13:39:08.914638701Z Waiting for performance profile collector pods to become ready: 4
2026-03-18T13:39:09.675744665Z Gathering data for ns/openshift-config-operator...
2026-03-18T13:39:10.094444991Z Waiting for performance profile collector pods to become ready: 5
2026-03-18T13:39:10.386167151Z Gathering data for ns/openshift-console-operator...
2026-03-18T13:39:11.019355337Z Gathering data for ns/openshift-console...
2026-03-18T13:39:11.298514984Z Waiting for performance profile collector pods to become ready: 6
2026-03-18T13:39:11.855209970Z volume usage percentage 0
2026-03-18T13:39:11.892272876Z Gathering data for ns/openshift-cluster-storage-operator...
2026-03-18T13:39:12.455347276Z Waiting for performance profile collector pods to become ready: 7
2026-03-18T13:39:12.501632857Z Gathering data for ns/openshift-dns-operator...
2026-03-18T13:39:13.157188885Z Gathering data for ns/openshift-dns...
2026-03-18T13:39:13.708344892Z Waiting for performance profile collector pods to become ready: 8
2026-03-18T13:39:13.788331261Z Gathering data for ns/openshift-etcd-operator...
2026-03-18T13:39:14.339992813Z Gathering data for ns/openshift-etcd...
2026-03-18T13:39:14.888104463Z Waiting for performance profile collector pods to become ready: 9
2026-03-18T13:39:15.537883305Z Gathering data for ns/openshift-image-registry...
2026-03-18T13:39:16.114382391Z Gathering data for ns/openshift-ingress-operator...
2026-03-18T13:39:16.120718483Z Daemonset perf-node-gather-daemonset ready 1 out of 1
2026-03-18T13:39:16.549467301Z Collecting performance related data for node master-0
2026-03-18T13:39:16.853970675Z Gathering data for ns/openshift-ingress-canary...
2026-03-18T13:39:16.903652604Z volume usage percentage 0
2026-03-18T13:39:17.334574314Z Gathering data for ns/openshift-insights...
2026-03-18T13:39:18.982487306Z Gathering data for ns/openshift-monitoring...
2026-03-18T13:39:21.784233452Z Gathering data for ns/metallb-system...
2026-03-18T13:39:21.965205215Z volume usage percentage 0
2026-03-18T13:39:23.600856724Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-03-18T13:39:24.150179558Z Gathering data for ns/openshift-operators...
2026-03-18T13:39:25.619195215Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-03-18T13:39:26.459956010Z Gathering data for ns/openshift-kube-apiserver...
2026-03-18T13:39:27.005742964Z volume usage percentage 0
2026-03-18T13:39:27.511526526Z Gathering data for ns/openshift-catalogd...
2026-03-18T13:39:28.076106019Z Gathering data for ns/cert-manager...
2026-03-18T13:39:28.612034398Z Gathering data for ns/openshift-nmstate...
2026-03-18T13:39:29.418160267Z Gathering data for ns/openshift-multus...
2026-03-18T13:39:30.289975704Z Gathering data for ns/openshift-storage...
2026-03-18T13:39:30.883162819Z Gathering data for ns/openshift-kube-controller-manager...
2026-03-18T13:39:31.464993987Z Collecting kubelet logs for node master-0
2026-03-18T13:39:32.083473140Z volume usage percentage 0
2026-03-18T13:39:32.200033732Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-03-18T13:39:32.351106385Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-03-18T13:39:32.368925568Z INFO: Node performance data collection complete.
2026-03-18T13:39:32.977267099Z Gathering data for ns/kube-system...
2026-03-18T13:39:33.562066823Z Gathering data for ns/openshift-kube-scheduler...
2026-03-18T13:39:34.181033090Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-03-18T13:39:34.741982668Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-03-18T13:39:35.255647337Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-03-18T13:39:35.846179456Z Gathering data for ns/openshift-cluster-machine-approver...
2026-03-18T13:39:36.318691122Z Gathering data for ns/openshift-machine-config-operator...
2026-03-18T13:39:37.129128774Z volume usage percentage 0
2026-03-18T13:39:37.274005650Z Gathering data for ns/openshift-kni-infra...
2026-03-18T13:39:37.787891735Z Gathering data for ns/openshift-openstack-infra...
2026-03-18T13:39:38.310409628Z Gathering data for ns/openshift-ovirt-infra...
2026-03-18T13:39:38.827065583Z Gathering data for ns/openshift-vsphere-infra...
2026-03-18T13:39:39.234069056Z Gathering data for ns/openshift-nutanix-infra...
2026-03-18T13:39:39.612198998Z Gathering data for ns/openshift-cloud-platform-infra...
2026-03-18T13:39:39.974443063Z Gathering data for ns/openshift-marketplace...
2026-03-18T13:39:42.231884039Z volume usage percentage 0
2026-03-18T13:39:43.043329270Z Gathering data for ns/openshift-user-workload-monitoring...
2026-03-18T13:39:44.768681379Z Gathering data for ns/openshift-ovn-kubernetes...
2026-03-18T13:39:46.219703728Z Gathering data for ns/openshift-host-network...
2026-03-18T13:39:46.705158846Z Gathering data for ns/openshift-network-diagnostics...
2026-03-18T13:39:47.288723005Z volume usage percentage 0
2026-03-18T13:39:47.400084467Z Gathering data for ns/openshift-network-node-identity...
2026-03-18T13:39:48.024976664Z Gathering data for ns/openshift-network-console...
2026-03-18T13:39:48.427716254Z Gathering data for ns/openshift-network-operator...
2026-03-18T13:39:49.078048182Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-03-18T13:39:49.892352605Z Gathering data for ns/openshift-operator-controller...
2026-03-18T13:39:51.042404522Z Gathering data for ns/openshift-cluster-olm-operator...
2026-03-18T13:39:51.826604620Z Gathering data for ns/openshift-apiserver-operator...
2026-03-18T13:39:52.364630969Z Gathering data for ns/openshift-apiserver...
2026-03-18T13:39:52.371533628Z volume usage percentage 0
2026-03-18T13:39:53.320208084Z Gathering data for ns/openshift-controller-manager-operator...
2026-03-18T13:39:53.774442084Z Gathering data for ns/openshift-controller-manager...
2026-03-18T13:39:54.470643081Z Gathering data for ns/openshift-route-controller-manager...
2026-03-18T13:39:54.928664381Z Gathering data for ns/openshift-cluster-samples-operator...
2026-03-18T13:39:55.967246833Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-03-18T13:39:56.943275725Z Gathering data for ns/openshift-service-ca-operator...
2026-03-18T13:39:57.506081837Z volume usage percentage 0
2026-03-18T13:39:57.615025019Z Gathering data for ns/openshift-service-ca...
2026-03-18T13:39:58.221829586Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-03-18T13:39:59.955627867Z Wrote inspect data to must-gather.
2026-03-18T13:39:59.955727389Z error: inspection completed with the errors occurred while gathering data:
2026-03-18T13:39:59.955727389Z     [skipping gathering clusterroles.rbac.authorization.k8s.io/system:registry due to error: clusterroles.rbac.authorization.k8s.io "system:registry" not found, skipping gathering clusterrolebindings.rbac.authorization.k8s.io/registry-registry-role due to error: clusterrolebindings.rbac.authorization.k8s.io "registry-registry-role" not found, skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering endpoints/host-etcd-2 due to error: endpoints "host-etcd-2" not found]
2026-03-18T13:40:00.184345563Z error: the server doesn't have a resource type "clusters"