2026-03-18T10:27:40.552158093Z volume percentage checker started.....
2026-03-18T10:27:40.569053178Z volume usage percentage 0
2026-03-18T10:27:41.071020214Z Gathering data for ns/openshift-cluster-version...
2026-03-18T10:27:44.369872810Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T10:27:44.636893775Z Gathering data for ns/default...
2026-03-18T10:27:45.086870386Z Gathering data for ns/openshift...
2026-03-18T10:27:45.196631563Z INFO: Gathering HAProxy config files
2026-03-18T10:27:45.198112740Z Waiting on subprocesses to finish execution.
2026-03-18T10:27:45.204721466Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-03-18T10:27:45.213447715Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-03-18T10:27:45.221437055Z INFO: Collecting host service logs for crio
2026-03-18T10:27:45.221651801Z INFO: Collecting host service logs for kubelet
2026-03-18T10:27:45.221895127Z INFO: Collecting host service logs for rpm-ostreed
2026-03-18T10:27:45.222118643Z INFO: Collecting host service logs for ostree-finalize-staged
2026-03-18T10:27:45.222355649Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-03-18T10:27:45.222584804Z INFO: Collecting host service logs for machine-config-daemon-host
2026-03-18T10:27:45.222841061Z INFO: Collecting host service logs for NetworkManager
2026-03-18T10:27:45.223051336Z INFO: Collecting host service logs for openvswitch
2026-03-18T10:27:45.223271022Z INFO: Collecting host service logs for ovs-configuration
2026-03-18T10:27:45.223511438Z INFO: Collecting host service logs for ovsdb-server
2026-03-18T10:27:45.223752424Z INFO: Collecting host service logs for ovs-vswitchd
2026-03-18T10:27:45.224043261Z INFO: Waiting for worker host service log collection to complete ...
2026-03-18T10:27:45.246119095Z INFO: Waiting for node performance related collection to complete ...
2026-03-18T10:27:45.650009458Z volume usage percentage 0
2026-03-18T10:27:45.819246718Z INFO: Found 1 replicas - prometheus-k8s-0
2026-03-18T10:27:46.711653949Z Wrote inspect data to must-gather.
2026-03-18T10:27:47.585055053Z INFO: Collecting Insights Archives from insights-operator-68bf6ff9d6-bdcw7
2026-03-18T10:27:47.589405903Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-03-18T10:27:47.589405903Z INFO: on-disk MachineConfig config collection complete.
2026-03-18T10:27:47.597366143Z No resources found
2026-03-18T10:27:47.605221180Z INFO: Waiting for HAProxy config collection to complete ...
2026-03-18T10:27:47.638214269Z INFO: "sriov-network-operator" not detected. Skipping.
2026-03-18T10:27:47.651299527Z INFO: Started  downloading kube-apiserver/termination.log from master-0
2026-03-18T10:27:47.690296367Z Gathering data for ns/metallb-system...
2026-03-18T10:27:47.700153364Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-03-18T10:27:47.896683930Z Wrote inspect data to must-gather.
2026-03-18T10:27:47.985553561Z INFO: Worker host service log collection to complete.
2026-03-18T10:27:48.572435569Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-03-18T10:27:48.611596402Z No resources found
2026-03-18T10:27:48.695690404Z INFO: Finished downloading kube-apiserver/termination.log from master-0
2026-03-18T10:27:48.819204946Z Gathering data for ns/openshift-nmstate...
2026-03-18T10:27:48.824708924Z Wrote inspect data to must-gather.
2026-03-18T10:27:48.825670578Z tar: Removing leading `/' from member names
2026-03-18T10:27:48.838163822Z INFO: HAProxy config collection complete.
2026-03-18T10:27:48.864715589Z Wrote inspect data to must-gather.
2026-03-18T10:27:48.950284268Z Wrote inspect data to must-gather.
2026-03-18T10:27:48.978557408Z Wrote inspect data to must-gather.
2026-03-18T10:27:48.978557408Z Wrote inspect data to must-gather.
2026-03-18T10:27:48.992612651Z Getting information from pod "etcd-master-0", container "etcdctl"
2026-03-18T10:27:48.992612651Z Using endpoints: https://192.168.32.10:2379
2026-03-18T10:27:49.002881769Z INFO: Starting getting etcd information
2026-03-18T10:27:49.002881769Z INFO: Getting etcdctl member list
2026-03-18T10:27:49.005076344Z INFO: Getting etcdctl endpoint status
2026-03-18T10:27:49.005220588Z INFO: Getting etcdctl endpoint health
2026-03-18T10:27:49.005453933Z INFO: Getting etcdctl alarm list
2026-03-18T10:27:49.005706610Z INFO: Getting object count
2026-03-18T10:27:49.007546516Z INFO: Waiting for etcd info collection to complete ...
2026-03-18T10:27:49.020888211Z Gathering data for ns/kube-system...
2026-03-18T10:27:49.148404052Z INFO: Getting rules from prometheus-k8s-0
2026-03-18T10:27:49.199346612Z receiving incremental file list
2026-03-18T10:27:49.226108324Z insights-operator/
2026-03-18T10:27:49.226237987Z insights-operator/insights-2026-03-18-095854.tar.gz
2026-03-18T10:27:49.235591392Z 
2026-03-18T10:27:49.235591392Z sent 47 bytes  received 170,723 bytes  113,846.67 bytes/sec
2026-03-18T10:27:49.235662704Z total size is 170,524  speedup is 1.00
2026-03-18T10:27:49.310081663Z INFO: Collecting /debug/api_priority_and_fairness endpoint from kube-apiserver-master-0
2026-03-18T10:27:49.496005682Z Wrote inspect data to must-gather.
2026-03-18T10:27:49.643844344Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T10:27:49.691302876Z Gathering data for ns/openshift-config...
2026-03-18T10:27:49.855946241Z INFO: Done collecting etcd information
2026-03-18T10:27:49.890357065Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T10:27:49.992683905Z W0318 10:27:49.992610    1225 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-03-18T10:27:50.037578752Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-18T10:27:50.050587909Z Wrote inspect data to must-gather.
2026-03-18T10:27:50.133550453Z Wrote inspect data to must-gather.
2026-03-18T10:27:50.374866173Z Gathering data for ns/openshift-etcd...
2026-03-18T10:27:50.464779061Z INFO: Getting status/config from prometheus-k8s-0
2026-03-18T10:27:50.546777780Z Wrote inspect data to must-gather.
2026-03-18T10:27:50.574894356Z Gathering data for ns/openshift-config-managed...
2026-03-18T10:27:50.669298727Z Wrote inspect data to must-gather.
2026-03-18T10:27:50.675420720Z volume usage percentage 0
2026-03-18T10:27:50.692026308Z INFO: INTERCONNECT MODE
2026-03-18T10:27:50.692026308Z INFO: Gathering ovn-kubernetes DBs
2026-03-18T10:27:50.704915461Z Wrote inspect data to must-gather.
2026-03-18T10:27:50.983864837Z INFO: Gathering OVN_Northbound from ovnkube-node-frnfl...
2026-03-18T10:27:50.984092232Z INFO: Gathering OVN_Southbound from ovnkube-node-frnfl...
2026-03-18T10:27:50.997053768Z Wrote inspect data to must-gather.
2026-03-18T10:27:51.172951645Z Wrote inspect data to must-gather.
2026-03-18T10:27:51.320248935Z tar: Removing leading `/' from member names
2026-03-18T10:27:51.322691586Z INFO: Getting status/flags from prometheus-k8s-0
2026-03-18T10:27:51.354925845Z tar: Removing leading `/' from member names
2026-03-18T10:27:51.385083543Z Wrote inspect data to must-gather.
2026-03-18T10:27:51.577591827Z Wrote inspect data to must-gather.
2026-03-18T10:27:51.678392359Z INFO: Waiting for network log collection to complete ...
2026-03-18T10:27:51.678392359Z INFO: Waiting for ovnk database copies to complete ...
2026-03-18T10:27:51.678898351Z INFO: Copying ovnk databases complete.
2026-03-18T10:27:51.695202781Z 2.1M	must-gather/network_logs/ovnk_database_store
2026-03-18T10:27:51.700397691Z ovnk_database_store/
2026-03-18T10:27:51.700397691Z ovnk_database_store/ovnkube-node-frnfl_sbdb
2026-03-18T10:27:51.717186133Z Gathering data for ns/assisted-installer...
2026-03-18T10:27:51.740195701Z Gathering data for ns/openshift-authentication...
2026-03-18T10:27:51.759292810Z ovnk_database_store/ovnkube-node-frnfl_nbdb
2026-03-18T10:27:51.781255292Z Wrote inspect data to must-gather.
2026-03-18T10:27:51.782161704Z INFO: Network log collection complete.
2026-03-18T10:27:51.804000803Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-03-18T10:27:52.009244397Z Wrote inspect data to must-gather.
2026-03-18T10:27:52.112345757Z Wrote inspect data to must-gather.
2026-03-18T10:27:52.300560993Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-03-18T10:27:52.391770804Z Wrote inspect data to must-gather.
2026-03-18T10:27:52.659898477Z Gathering data for ns/openshift-authentication-operator...
2026-03-18T10:27:52.786746272Z Wrote inspect data to must-gather.
2026-03-18T10:27:52.807006141Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-03-18T10:27:52.831046095Z Wrote inspect data to must-gather.
2026-03-18T10:27:52.916045559Z Wrote inspect data to must-gather.
2026-03-18T10:27:52.964234460Z Wrote inspect data to must-gather.
2026-03-18T10:27:53.268564482Z W0318 10:27:53.268124    1803 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-03-18T10:27:53.311597263Z INFO: Getting status from alertmanager-main-0
2026-03-18T10:27:53.327612845Z Wrote inspect data to must-gather.
2026-03-18T10:27:53.447408244Z Wrote inspect data to must-gather.
2026-03-18T10:27:53.485340246Z Gathering data for ns/openshift-ingress...
2026-03-18T10:27:53.705361802Z Wrote inspect data to must-gather.
2026-03-18T10:27:53.851166504Z Wrote inspect data to must-gather.
2026-03-18T10:27:54.000317339Z Wrote inspect data to must-gather.
2026-03-18T10:27:54.185592952Z Gathering data for ns/openshift-oauth-apiserver...
2026-03-18T10:27:54.304573910Z Wrote inspect data to must-gather.
2026-03-18T10:27:54.478201041Z Wrote inspect data to must-gather.
2026-03-18T10:27:54.683452745Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa5e782406f71c048b1ac3a4bf5d1227ff4be81111114083ad4c7a209c6bfb5a
2026-03-18T10:27:54.754835028Z Gathering data for ns/openshift-machine-api...
2026-03-18T10:27:54.893297755Z daemonset.apps/perf-node-gather-daemonset created
2026-03-18T10:27:54.945774243Z Wrote inspect data to must-gather.
2026-03-18T10:27:55.128359719Z Waiting for performance profile collector pods to become ready: 1
2026-03-18T10:27:55.566494712Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/frr.conf: No such file or directory
2026-03-18T10:27:55.566494712Z /usr/bin/gather_metallb: line 53: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/reloader/reloader/logs/reloader.pid: No such file or directory
2026-03-18T10:27:55.567060976Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/frr.log: No such file or directory
2026-03-18T10:27:55.567786214Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/daemons: No such file or directory
2026-03-18T10:27:55.568385039Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/vtysh.conf: No such file or directory
2026-03-18T10:27:55.568997775Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:55.716439437Z volume usage percentage 0
2026-03-18T10:27:55.730643504Z Error from server (BadRequest): container frr is not valid for pod speaker-nv5ns
2026-03-18T10:27:55.743843006Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:55.743843006Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:55.978248952Z Error from server (BadRequest): container frr is not valid for pod speaker-nv5ns
2026-03-18T10:27:56.000121322Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:56.000214954Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:56.081392663Z Gathering data for ns/openshift-cloud-controller-manager-operator...
2026-03-18T10:27:56.183019215Z Error from server (BadRequest): container frr is not valid for pod speaker-nv5ns
2026-03-18T10:27:56.191947739Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:56.191947739Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:56.318778323Z Waiting for performance profile collector pods to become ready: 2
2026-03-18T10:27:56.456893232Z Error from server (BadRequest): container frr is not valid for pod speaker-nv5ns
2026-03-18T10:27:56.481115940Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:56.481115940Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:56.762223990Z Error from server (BadRequest): container frr is not valid for pod speaker-nv5ns
2026-03-18T10:27:56.873695859Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-nv5ns/frr/frr/logs/dump_frr: No such file or directory
2026-03-18T10:27:57.056801078Z Gathering data for ns/openshift-cloud-controller-manager...
2026-03-18T10:27:57.549238354Z Waiting for performance profile collector pods to become ready: 3
2026-03-18T10:27:58.169156403Z Gathering data for ns/openshift-cloud-credential-operator...
2026-03-18T10:27:58.711756980Z Waiting for performance profile collector pods to become ready: 4
2026-03-18T10:27:59.353794963Z Gathering data for ns/openshift-config-operator...
2026-03-18T10:27:59.931102760Z Waiting for performance profile collector pods to become ready: 5
2026-03-18T10:28:00.097049258Z Gathering data for ns/openshift-console-operator...
2026-03-18T10:28:00.596260615Z Gathering data for ns/openshift-console...
2026-03-18T10:28:00.756148330Z volume usage percentage 0
2026-03-18T10:28:01.134969364Z Waiting for performance profile collector pods to become ready: 6
2026-03-18T10:28:01.278536829Z Gathering data for ns/openshift-cluster-storage-operator...
2026-03-18T10:28:01.832711217Z Gathering data for ns/openshift-dns-operator...
2026-03-18T10:28:02.315772438Z Waiting for performance profile collector pods to become ready: 7
2026-03-18T10:28:02.344553231Z Gathering data for ns/openshift-dns...
2026-03-18T10:28:02.799837905Z Gathering data for ns/openshift-etcd-operator...
2026-03-18T10:28:03.365636043Z Gathering data for ns/openshift-etcd...
2026-03-18T10:28:03.487548835Z Waiting for performance profile collector pods to become ready: 8
2026-03-18T10:28:04.371700859Z Gathering data for ns/openshift-image-registry...
2026-03-18T10:28:04.702160448Z Waiting for performance profile collector pods to become ready: 9
2026-03-18T10:28:04.890183730Z Gathering data for ns/openshift-ingress-operator...
2026-03-18T10:28:05.586735483Z Gathering data for ns/openshift-ingress-canary...
2026-03-18T10:28:05.800506271Z volume usage percentage 0
2026-03-18T10:28:05.880912081Z Daemonset perf-node-gather-daemonset ready 1 out of 1
2026-03-18T10:28:06.058922691Z Gathering data for ns/openshift-insights...
2026-03-18T10:28:06.275109460Z Collecting performance related data for node master-0
2026-03-18T10:28:07.452791705Z Gathering data for ns/openshift-monitoring...
2026-03-18T10:28:09.612358290Z Gathering data for ns/metallb-system...
2026-03-18T10:28:10.848061392Z volume usage percentage 0
2026-03-18T10:28:11.203983300Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-03-18T10:28:11.710599553Z Gathering data for ns/openshift-operators...
2026-03-18T10:28:13.123845705Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-03-18T10:28:13.784031155Z Gathering data for ns/openshift-kube-apiserver...
2026-03-18T10:28:14.779407051Z Gathering data for ns/openshift-catalogd...
2026-03-18T10:28:15.427874817Z Gathering data for ns/cert-manager...
2026-03-18T10:28:15.892052524Z volume usage percentage 0
2026-03-18T10:28:15.925081573Z Gathering data for ns/openshift-nmstate...
2026-03-18T10:28:16.748063441Z Gathering data for ns/openshift-multus...
2026-03-18T10:28:17.573738107Z Gathering data for ns/openshift-storage...
2026-03-18T10:28:18.119269946Z Gathering data for ns/openshift-kube-controller-manager...
2026-03-18T10:28:19.052587185Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-03-18T10:28:19.680795202Z Gathering data for ns/kube-system...
2026-03-18T10:28:20.286507653Z Gathering data for ns/openshift-kube-scheduler...
2026-03-18T10:28:20.950258262Z volume usage percentage 0
2026-03-18T10:28:20.973103246Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-03-18T10:28:21.026081617Z Collecting kubelet logs for node master-0
2026-03-18T10:28:21.706655637Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-03-18T10:28:21.785200350Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-03-18T10:28:21.801868348Z INFO: Node performance data collection complete.
2026-03-18T10:28:22.296842289Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-03-18T10:28:22.919024714Z Gathering data for ns/openshift-cluster-machine-approver...
2026-03-18T10:28:23.419292158Z Gathering data for ns/openshift-machine-config-operator...
2026-03-18T10:28:24.351793086Z Gathering data for ns/openshift-kni-infra...
2026-03-18T10:28:24.704796281Z Gathering data for ns/openshift-openstack-infra...
2026-03-18T10:28:25.054036181Z Gathering data for ns/openshift-ovirt-infra...
2026-03-18T10:28:25.430302260Z Gathering data for ns/openshift-vsphere-infra...
2026-03-18T10:28:25.776989587Z Gathering data for ns/openshift-nutanix-infra...
2026-03-18T10:28:26.013936428Z volume usage percentage 0
2026-03-18T10:28:26.087478475Z Gathering data for ns/openshift-cloud-platform-infra...
2026-03-18T10:28:26.428215512Z Gathering data for ns/openshift-marketplace...
2026-03-18T10:28:29.183225558Z Gathering data for ns/openshift-user-workload-monitoring...
2026-03-18T10:28:30.332052639Z Gathering data for ns/openshift-ovn-kubernetes...
2026-03-18T10:28:31.059876387Z volume usage percentage 0
2026-03-18T10:28:31.689114300Z Gathering data for ns/openshift-host-network...
2026-03-18T10:28:32.192516461Z Gathering data for ns/openshift-network-diagnostics...
2026-03-18T10:28:32.801044733Z Gathering data for ns/openshift-network-node-identity...
2026-03-18T10:28:33.445045946Z Gathering data for ns/openshift-network-console...
2026-03-18T10:28:33.831974354Z Gathering data for ns/openshift-network-operator...
2026-03-18T10:28:34.523858419Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-03-18T10:28:35.185287780Z Gathering data for ns/openshift-operator-controller...
2026-03-18T10:28:36.075641769Z Gathering data for ns/openshift-cluster-olm-operator...
2026-03-18T10:28:36.118732101Z volume usage percentage 0
2026-03-18T10:28:36.976097723Z Gathering data for ns/openshift-apiserver-operator...
2026-03-18T10:28:37.524009853Z Gathering data for ns/openshift-apiserver...
2026-03-18T10:28:38.366986373Z Gathering data for ns/openshift-controller-manager-operator...
2026-03-18T10:28:38.819530418Z Gathering data for ns/openshift-controller-manager...
2026-03-18T10:28:39.317152624Z Gathering data for ns/openshift-route-controller-manager...
2026-03-18T10:28:39.909403808Z Gathering data for ns/openshift-cluster-samples-operator...
2026-03-18T10:28:40.946216126Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-03-18T10:28:41.178962441Z volume usage percentage 0
2026-03-18T10:28:42.063915286Z Gathering data for ns/openshift-service-ca-operator...
2026-03-18T10:28:42.555227654Z Gathering data for ns/openshift-service-ca...
2026-03-18T10:28:43.066669837Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-03-18T10:28:44.481697534Z Wrote inspect data to must-gather.
2026-03-18T10:28:44.481748635Z error: inspection completed with the errors occurred while gathering data:
2026-03-18T10:28:44.481748635Z     [skipping gathering clusterroles.rbac.authorization.k8s.io/system:registry due to error: clusterroles.rbac.authorization.k8s.io "system:registry" not found, skipping gathering clusterrolebindings.rbac.authorization.k8s.io/registry-registry-role due to error: clusterrolebindings.rbac.authorization.k8s.io "registry-registry-role" not found, skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering endpoints/host-etcd-2 due to error: endpoints "host-etcd-2" not found]
2026-03-18T10:28:44.680553978Z error: the server doesn't have a resource type "clusters"