2026-03-20T09:04:49.001290637Z volume percentage checker started.....
2026-03-20T09:04:49.031163064Z volume usage percentage 0
2026-03-20T09:04:49.341311698Z Gathering data for ns/openshift-cluster-version...
2026-03-20T09:04:51.512334057Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-20T09:04:51.779904285Z Gathering data for ns/default...
2026-03-20T09:04:52.251925365Z Gathering data for ns/openshift...
2026-03-20T09:04:52.842278631Z Waiting on subprocesses to finish execution.
2026-03-20T09:04:52.848771549Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-03-20T09:04:52.852328476Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-03-20T09:04:52.855083522Z INFO: Gathering HAProxy config files
2026-03-20T09:04:52.862316459Z INFO: Collecting host service logs for crio
2026-03-20T09:04:52.862706400Z INFO: Collecting host service logs for kubelet
2026-03-20T09:04:52.863120372Z INFO: Collecting host service logs for rpm-ostreed
2026-03-20T09:04:52.863227114Z INFO: Collecting host service logs for ostree-finalize-staged
2026-03-20T09:04:52.863595144Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-03-20T09:04:52.863863172Z INFO: Collecting host service logs for machine-config-daemon-host
2026-03-20T09:04:52.864187110Z INFO: Collecting host service logs for NetworkManager
2026-03-20T09:04:52.864457408Z INFO: Collecting host service logs for openvswitch
2026-03-20T09:04:52.864724335Z INFO: Collecting host service logs for ovs-configuration
2026-03-20T09:04:52.865004112Z INFO: Collecting host service logs for ovsdb-server
2026-03-20T09:04:52.865206078Z INFO: Collecting host service logs for ovs-vswitchd
2026-03-20T09:04:52.865546668Z INFO: Waiting for worker host service log collection to complete ...
2026-03-20T09:04:52.867076609Z INFO: Waiting for node performance related collection to complete ...
2026-03-20T09:04:52.884955989Z Gathering data for ns/kube-system...
2026-03-20T09:04:53.415715255Z No resources found
2026-03-20T09:04:53.909023447Z INFO: Collecting Insights Archives from insights-operator-68bf6ff9d6-c7zf4
2026-03-20T09:04:53.935707968Z Wrote inspect data to must-gather.
2026-03-20T09:04:53.943000657Z INFO: Found 1 replicas - prometheus-k8s-0
2026-03-20T09:04:53.963475737Z INFO: Waiting for HAProxy config collection to complete ...
2026-03-20T09:04:53.966238573Z INFO: "sriov-network-operator" not detected. Skipping.
2026-03-20T09:04:53.977359686Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-03-20T09:04:53.977359686Z INFO: on-disk MachineConfig config collection complete.
2026-03-20T09:04:54.015304044Z INFO: Started  downloading kube-apiserver/termination.log from master-0
2026-03-20T09:04:54.085750841Z INFO: Worker host service log collection to complete.
2026-03-20T09:04:54.086116171Z volume usage percentage 0
2026-03-20T09:04:54.185633553Z Gathering data for ns/metallb-system...
2026-03-20T09:04:54.250397844Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-03-20T09:04:54.371915398Z Getting information from pod "etcd-master-0", container "etcdctl"
2026-03-20T09:04:54.371915398Z Using endpoints: https://192.168.32.10:2379
2026-03-20T09:04:54.386409174Z INFO: Starting getting etcd information
2026-03-20T09:04:54.386439115Z INFO: Getting etcdctl member list
2026-03-20T09:04:54.386731103Z INFO: Getting etcdctl endpoint status
2026-03-20T09:04:54.386914558Z INFO: Getting etcdctl endpoint health
2026-03-20T09:04:54.388347517Z INFO: Getting etcdctl alarm list
2026-03-20T09:04:54.388366138Z INFO: Getting object count
2026-03-20T09:04:54.390196408Z INFO: Waiting for etcd info collection to complete ...
2026-03-20T09:04:54.391162234Z Gathering data for ns/openshift-etcd...
2026-03-20T09:04:54.526085624Z INFO: Finished downloading kube-apiserver/termination.log from master-0
2026-03-20T09:04:54.573884581Z INFO: HAProxy config collection complete.
2026-03-20T09:04:54.800503600Z No resources found
2026-03-20T09:04:54.911585278Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-03-20T09:04:55.119658859Z Gathering data for ns/openshift-nmstate...
2026-03-20T09:04:55.232498706Z Wrote inspect data to must-gather.
2026-03-20T09:04:55.334943397Z Wrote inspect data to must-gather.
2026-03-20T09:04:55.349303850Z Wrote inspect data to must-gather.
2026-03-20T09:04:55.538091854Z Wrote inspect data to must-gather.
2026-03-20T09:04:55.576535274Z Wrote inspect data to must-gather.
2026-03-20T09:04:55.673226620Z receiving incremental file list
2026-03-20T09:04:55.677202218Z INFO: Done collecting etcd information
2026-03-20T09:04:55.694564833Z insights-operator/
2026-03-20T09:04:55.694653785Z insights-operator/insights-2026-03-20-083924.tar.gz
2026-03-20T09:04:55.710879669Z 
2026-03-20T09:04:55.710879669Z sent 47 bytes  received 187,811 bytes  125,238.67 bytes/sec
2026-03-20T09:04:55.710879669Z total size is 187,611  speedup is 1.00
2026-03-20T09:04:55.893679239Z INFO: Getting rules from prometheus-k8s-0
2026-03-20T09:04:55.961093953Z Wrote inspect data to must-gather.
2026-03-20T09:04:55.961917415Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-20T09:04:55.996394098Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-20T09:04:56.000692496Z Gathering data for ns/openshift-config...
2026-03-20T09:04:56.098931753Z INFO: Collecting /debug/api_priority_and_fairness endpoint from kube-apiserver-master-0
2026-03-20T09:04:56.210452993Z Wrote inspect data to must-gather.
2026-03-20T09:04:56.352719193Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-20T09:04:56.601320163Z Gathering data for ns/assisted-installer...
2026-03-20T09:04:56.612518449Z INFO: Getting status/config from prometheus-k8s-0
2026-03-20T09:04:56.718062766Z Wrote inspect data to must-gather.
2026-03-20T09:04:56.776544026Z Wrote inspect data to must-gather.
2026-03-20T09:04:56.892960090Z Wrote inspect data to must-gather.
2026-03-20T09:04:56.897559836Z Gathering data for ns/openshift-config-managed...
2026-03-20T09:04:56.993193931Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.370825040Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.374529061Z INFO: Getting status/flags from prometheus-k8s-0
2026-03-20T09:04:57.471043141Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.476640944Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.569144164Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.651327502Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-03-20T09:04:57.701870864Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.836153827Z Wrote inspect data to must-gather.
2026-03-20T09:04:57.845712848Z Gathering data for ns/openshift-authentication...
2026-03-20T09:04:57.939074222Z INFO: INTERCONNECT MODE
2026-03-20T09:04:57.939074222Z INFO: Gathering ovn-kubernetes DBs
2026-03-20T09:04:58.008951053Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-03-20T09:04:58.014949647Z Wrote inspect data to must-gather.
2026-03-20T09:04:58.149928429Z Wrote inspect data to must-gather.
2026-03-20T09:04:58.189231143Z INFO: Gathering OVN_Northbound from ovnkube-node-bvndl...
2026-03-20T09:04:58.189489260Z INFO: Gathering OVN_Southbound from ovnkube-node-bvndl...
2026-03-20T09:04:58.269567591Z Wrote inspect data to must-gather.
2026-03-20T09:04:58.383111106Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-03-20T09:04:58.409486368Z tar: Removing leading `/' from member names
2026-03-20T09:04:58.608861341Z Wrote inspect data to must-gather.
2026-03-20T09:04:58.617965690Z tar: Removing leading `/' from member names
2026-03-20T09:04:58.712816554Z Wrote inspect data to must-gather.
2026-03-20T09:04:58.750934276Z INFO: Waiting for network log collection to complete ...
2026-03-20T09:04:58.750934276Z INFO: Waiting for ovnk database copies to complete ...
2026-03-20T09:04:58.750934276Z INFO: Copying ovnk databases complete.
2026-03-20T09:04:58.750934276Z 2.0M	must-gather/network_logs/ovnk_database_store
2026-03-20T09:04:58.752149620Z ovnk_database_store/
2026-03-20T09:04:58.752149620Z ovnk_database_store/ovnkube-node-bvndl_nbdb
2026-03-20T09:04:58.755771668Z Gathering data for ns/openshift-authentication-operator...
2026-03-20T09:04:58.765194457Z ovnk_database_store/ovnkube-node-bvndl_sbdb
2026-03-20T09:04:58.814402552Z INFO: Network log collection complete.
2026-03-20T09:04:58.910429168Z Wrote inspect data to must-gather.
2026-03-20T09:04:58.964551679Z INFO: Getting status from alertmanager-main-0
2026-03-20T09:04:59.161340021Z Wrote inspect data to must-gather.
2026-03-20T09:04:59.161600978Z volume usage percentage 0
2026-03-20T09:04:59.276204703Z Wrote inspect data to must-gather.
2026-03-20T09:04:59.516426673Z Wrote inspect data to must-gather.
2026-03-20T09:04:59.540295606Z Gathering data for ns/openshift-ingress...
2026-03-20T09:04:59.680505440Z Wrote inspect data to must-gather.
2026-03-20T09:05:00.117398120Z Wrote inspect data to must-gather.
2026-03-20T09:05:00.120937217Z Wrote inspect data to must-gather.
2026-03-20T09:05:00.151113732Z Gathering data for ns/openshift-oauth-apiserver...
2026-03-20T09:05:00.233993449Z Wrote inspect data to must-gather.
2026-03-20T09:05:00.599360241Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa5e782406f71c048b1ac3a4bf5d1227ff4be81111114083ad4c7a209c6bfb5a
2026-03-20T09:05:00.752853700Z Gathering data for ns/openshift-machine-api...
2026-03-20T09:05:00.805342956Z Wrote inspect data to must-gather.
2026-03-20T09:05:00.806798665Z daemonset.apps/perf-node-gather-daemonset created
2026-03-20T09:05:01.035977403Z Waiting for performance profile collector pods to become ready: 1
2026-03-20T09:05:01.290325740Z Wrote inspect data to must-gather.
2026-03-20T09:05:01.894851734Z /usr/bin/gather_metallb: line 53: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/reloader/reloader/logs/reloader.pid: No such file or directory
2026-03-20T09:05:01.896012326Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/frr.conf: No such file or directory
2026-03-20T09:05:01.901600789Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/frr.log: No such file or directory
2026-03-20T09:05:01.902604447Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/daemons: No such file or directory
2026-03-20T09:05:01.903842070Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/vtysh.conf: No such file or directory
2026-03-20T09:05:01.904321763Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:01.955160613Z Gathering data for ns/openshift-cloud-controller-manager-operator...
2026-03-20T09:05:02.193300566Z Error from server (BadRequest): container frr is not valid for pod speaker-p2fhx
2026-03-20T09:05:02.205783939Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.205783939Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.224585402Z Waiting for performance profile collector pods to become ready: 2
2026-03-20T09:05:02.411734111Z Error from server (BadRequest): container frr is not valid for pod speaker-p2fhx
2026-03-20T09:05:02.419005150Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.419043571Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.505087255Z Gathering data for ns/openshift-cloud-controller-manager...
2026-03-20T09:05:02.609494540Z Error from server (BadRequest): container frr is not valid for pod speaker-p2fhx
2026-03-20T09:05:02.616836840Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.616876741Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.792809583Z Error from server (BadRequest): container frr is not valid for pod speaker-p2fhx
2026-03-20T09:05:02.803728452Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:02.803796594Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:03.043077589Z Error from server (BadRequest): container frr is not valid for pod speaker-p2fhx
2026-03-20T09:05:03.050471571Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-p2fhx/frr/frr/logs/dump_frr: No such file or directory
2026-03-20T09:05:03.188669101Z Gathering data for ns/openshift-cloud-credential-operator...
2026-03-20T09:05:03.548617565Z Waiting for performance profile collector pods to become ready: 3
2026-03-20T09:05:04.198091549Z volume usage percentage 0
2026-03-20T09:05:04.702829854Z Waiting for performance profile collector pods to become ready: 4
2026-03-20T09:05:05.081972093Z Gathering data for ns/openshift-config-operator...
2026-03-20T09:05:05.833420936Z Gathering data for ns/openshift-console-operator...
2026-03-20T09:05:05.866928132Z Waiting for performance profile collector pods to become ready: 5
2026-03-20T09:05:06.370844605Z Gathering data for ns/openshift-console...
2026-03-20T09:05:07.094450676Z Gathering data for ns/openshift-cluster-storage-operator...
2026-03-20T09:05:07.112121139Z Waiting for performance profile collector pods to become ready: 6
2026-03-20T09:05:07.627241798Z Gathering data for ns/openshift-dns-operator...
2026-03-20T09:05:08.067870829Z Gathering data for ns/openshift-dns...
2026-03-20T09:05:08.277906523Z Waiting for performance profile collector pods to become ready: 7
2026-03-20T09:05:08.520940691Z Gathering data for ns/openshift-etcd-operator...
2026-03-20T09:05:08.989441204Z Gathering data for ns/openshift-etcd...
2026-03-20T09:05:09.229223983Z volume usage percentage 0
2026-03-20T09:05:09.480762723Z Waiting for performance profile collector pods to become ready: 8
2026-03-20T09:05:09.979260387Z Gathering data for ns/openshift-image-registry...
2026-03-20T09:05:10.547298222Z Gathering data for ns/openshift-ingress-operator...
2026-03-20T09:05:10.678420369Z Waiting for performance profile collector pods to become ready: 9
2026-03-20T09:05:11.029275565Z Gathering data for ns/openshift-ingress-canary...
2026-03-20T09:05:11.477927046Z Gathering data for ns/openshift-insights...
2026-03-20T09:05:11.855839511Z Daemonset perf-node-gather-daemonset ready 1 out of 1
2026-03-20T09:05:12.310243160Z Collecting performance related data for node master-0
2026-03-20T09:05:12.936056076Z Gathering data for ns/openshift-monitoring...
2026-03-20T09:05:14.274440812Z volume usage percentage 0
2026-03-20T09:05:14.875884112Z Gathering data for ns/metallb-system...
2026-03-20T09:05:16.390820105Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-03-20T09:05:17.156763085Z Gathering data for ns/openshift-operators...
2026-03-20T09:05:18.506632655Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-03-20T09:05:19.210302190Z Gathering data for ns/openshift-kube-apiserver...
2026-03-20T09:05:19.310794689Z volume usage percentage 0
2026-03-20T09:05:20.142887727Z Gathering data for ns/openshift-catalogd...
2026-03-20T09:05:20.650958713Z Gathering data for ns/cert-manager...
2026-03-20T09:05:21.141138709Z Gathering data for ns/openshift-nmstate...
2026-03-20T09:05:21.758537895Z Gathering data for ns/openshift-multus...
2026-03-20T09:05:22.550384233Z Gathering data for ns/openshift-storage...
2026-03-20T09:05:23.186498841Z Gathering data for ns/openshift-kube-controller-manager...
2026-03-20T09:05:24.549601272Z volume usage percentage 0
2026-03-20T09:05:25.963311027Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-03-20T09:05:26.614721664Z Gathering data for ns/kube-system...
2026-03-20T09:05:27.107412220Z Gathering data for ns/openshift-kube-scheduler...
2026-03-20T09:05:27.217963333Z Collecting kubelet logs for node master-0
2026-03-20T09:05:27.727053438Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-03-20T09:05:28.054662418Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-03-20T09:05:28.090715844Z INFO: Node performance data collection complete.
2026-03-20T09:05:28.710796503Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-03-20T09:05:29.134167822Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-03-20T09:05:29.601295889Z volume usage percentage 0
2026-03-20T09:05:29.909323954Z Gathering data for ns/openshift-cluster-machine-approver...
2026-03-20T09:05:30.718653149Z Gathering data for ns/openshift-machine-config-operator...
2026-03-20T09:05:32.468690054Z Gathering data for ns/openshift-kni-infra...
2026-03-20T09:05:32.971618179Z Gathering data for ns/openshift-openstack-infra...
2026-03-20T09:05:33.318565038Z Gathering data for ns/openshift-ovirt-infra...
2026-03-20T09:05:33.754506242Z Gathering data for ns/openshift-vsphere-infra...
2026-03-20T09:05:34.112206604Z Gathering data for ns/openshift-nutanix-infra...
2026-03-20T09:05:34.424393823Z Gathering data for ns/openshift-cloud-platform-infra...
2026-03-20T09:05:34.644007559Z volume usage percentage 0
2026-03-20T09:05:34.803138472Z Gathering data for ns/openshift-marketplace...
2026-03-20T09:05:38.271561845Z Gathering data for ns/openshift-user-workload-monitoring...
2026-03-20T09:05:39.486768431Z Gathering data for ns/openshift-ovn-kubernetes...
2026-03-20T09:05:39.694470662Z volume usage percentage 0
2026-03-20T09:05:40.946603889Z Gathering data for ns/openshift-host-network...
2026-03-20T09:05:41.469335996Z Gathering data for ns/openshift-network-diagnostics...
2026-03-20T09:05:42.065238793Z Gathering data for ns/openshift-network-node-identity...
2026-03-20T09:05:42.774330927Z Gathering data for ns/openshift-network-console...
2026-03-20T09:05:43.254975884Z Gathering data for ns/openshift-network-operator...
2026-03-20T09:05:43.926371447Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-03-20T09:05:44.756364497Z volume usage percentage 0
2026-03-20T09:05:44.774308338Z Gathering data for ns/openshift-operator-controller...
2026-03-20T09:05:45.981739172Z Gathering data for ns/openshift-cluster-olm-operator...
2026-03-20T09:05:46.695683229Z Gathering data for ns/openshift-apiserver-operator...
2026-03-20T09:05:47.323544021Z Gathering data for ns/openshift-apiserver...
2026-03-20T09:05:48.161309004Z Gathering data for ns/openshift-controller-manager-operator...
2026-03-20T09:05:48.629683915Z Gathering data for ns/openshift-controller-manager...
2026-03-20T09:05:49.162967740Z Gathering data for ns/openshift-route-controller-manager...
2026-03-20T09:05:49.692908384Z Gathering data for ns/openshift-cluster-samples-operator...
2026-03-20T09:05:49.813292177Z volume usage percentage 0
2026-03-20T09:05:50.672925888Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-03-20T09:05:51.513522579Z Gathering data for ns/openshift-service-ca-operator...
2026-03-20T09:05:51.964223886Z Gathering data for ns/openshift-service-ca...
2026-03-20T09:05:52.447583936Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-03-20T09:05:53.930773882Z Wrote inspect data to must-gather.
2026-03-20T09:05:53.930864015Z error: inspection completed with the errors occurred while gathering data:
2026-03-20T09:05:53.930864015Z     [skipping gathering clusterroles.rbac.authorization.k8s.io/system:registry due to error: clusterroles.rbac.authorization.k8s.io "system:registry" not found, skipping gathering clusterrolebindings.rbac.authorization.k8s.io/registry-registry-role due to error: clusterrolebindings.rbac.authorization.k8s.io "registry-registry-role" not found, skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering endpoints/host-etcd-2 due to error: endpoints "host-etcd-2" not found]
2026-03-20T09:05:54.181295744Z error: the server doesn't have a resource type "clusters"