2026-03-13T13:10:25.046715340Z volume percentage checker started.....
2026-03-13T13:10:25.066041835Z volume usage percentage 0
2026-03-13T13:10:25.372841564Z Gathering data for ns/openshift-cluster-version...
2026-03-13T13:10:28.313637666Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-13T13:10:28.642071604Z Gathering data for ns/default...
2026-03-13T13:10:28.726410464Z Waiting on subprocesses to finish execution.
2026-03-13T13:10:28.729310156Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-03-13T13:10:28.738051524Z INFO: Collecting host service logs for crio
2026-03-13T13:10:28.738262671Z INFO: Collecting host service logs for kubelet
2026-03-13T13:10:28.739181220Z INFO: Collecting host service logs for rpm-ostreed
2026-03-13T13:10:28.740115170Z INFO: Collecting host service logs for ostree-finalize-staged
2026-03-13T13:10:28.740560773Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-03-13T13:10:28.740903334Z INFO: Collecting host service logs for machine-config-daemon-host
2026-03-13T13:10:28.741333078Z INFO: Collecting host service logs for NetworkManager
2026-03-13T13:10:28.741861135Z INFO: Collecting host service logs for openvswitch
2026-03-13T13:10:28.743802097Z INFO: Collecting host service logs for ovs-configuration
2026-03-13T13:10:28.743802097Z INFO: Collecting host service logs for ovsdb-server
2026-03-13T13:10:28.744352554Z INFO: Collecting host service logs for ovs-vswitchd
2026-03-13T13:10:28.745702808Z INFO: Waiting for worker host service log collection to complete ...
2026-03-13T13:10:28.753961210Z INFO: Gathering HAProxy config files
2026-03-13T13:10:28.763758081Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-03-13T13:10:28.838182737Z INFO: Waiting for node performance related collection to complete ...
2026-03-13T13:10:29.441064386Z INFO: "sriov-network-operator" not detected. Skipping.
2026-03-13T13:10:29.542042185Z INFO: Started  downloading kube-apiserver/termination.log from master-0
2026-03-13T13:10:29.633551923Z No resources found
2026-03-13T13:10:29.716227941Z Gathering data for ns/openshift...
2026-03-13T13:10:29.878719926Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-03-13T13:10:29.878719926Z INFO: on-disk MachineConfig config collection complete.
2026-03-13T13:10:30.031272093Z INFO: Found 1 replicas - prometheus-k8s-0
2026-03-13T13:10:30.060981698Z INFO: Collecting Insights Archives from insights-operator-8f89dfddd-s4gd8
2026-03-13T13:10:30.091524119Z volume usage percentage 0
2026-03-13T13:10:30.220840359Z INFO: Worker host service log collection to complete.
2026-03-13T13:10:30.241775693Z INFO: Waiting for HAProxy config collection to complete ...
2026-03-13T13:10:30.330385880Z Wrote inspect data to must-gather.
2026-03-13T13:10:30.613359462Z INFO: Finished downloading kube-apiserver/termination.log from master-0
2026-03-13T13:10:30.707447583Z Wrote inspect data to must-gather.
2026-03-13T13:10:31.582734850Z Gathering data for ns/metallb-system...
2026-03-13T13:10:31.666242634Z Gathering data for ns/openshift-nmstate...
2026-03-13T13:10:31.830950059Z Wrote inspect data to must-gather.
2026-03-13T13:10:31.895744608Z Wrote inspect data to must-gather.
2026-03-13T13:10:31.911370415Z INFO: HAProxy config collection complete.
2026-03-13T13:10:31.922312553Z Wrote inspect data to must-gather.
2026-03-13T13:10:31.924864744Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-03-13T13:10:32.091143708Z Getting information from pod "etcd-master-0", container "etcdctl"
2026-03-13T13:10:32.091143708Z Using endpoints: https://192.168.32.10:2379
2026-03-13T13:10:32.095214287Z INFO: Starting getting etcd information
2026-03-13T13:10:32.095214287Z INFO: Getting etcdctl member list
2026-03-13T13:10:32.095439844Z INFO: Getting etcdctl endpoint status
2026-03-13T13:10:32.095657761Z INFO: Getting etcdctl endpoint health
2026-03-13T13:10:32.095872688Z INFO: Getting etcdctl alarm list
2026-03-13T13:10:32.096124406Z INFO: Getting object count
2026-03-13T13:10:32.097681095Z INFO: Waiting for etcd info collection to complete ...
2026-03-13T13:10:32.162044581Z receiving incremental file list
2026-03-13T13:10:32.220502859Z insights-operator/
2026-03-13T13:10:32.243435838Z 
2026-03-13T13:10:32.243435838Z sent 28 bytes  received 71 bytes  39.60 bytes/sec
2026-03-13T13:10:32.243435838Z total size is 0  speedup is 0.00
2026-03-13T13:10:32.259638152Z No resources found
2026-03-13T13:10:32.387871039Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-03-13T13:10:32.480388489Z Wrote inspect data to must-gather.
2026-03-13T13:10:32.546456728Z Gathering data for ns/kube-system...
2026-03-13T13:10:32.569324185Z INFO: Collecting /debug/api_priority_and_fairness endpoint from kube-apiserver-master-0
2026-03-13T13:10:32.747028922Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-13T13:10:33.230123766Z Wrote inspect data to must-gather.
2026-03-13T13:10:33.236690604Z Wrote inspect data to must-gather.
2026-03-13T13:10:33.241096455Z INFO: Getting rules from prometheus-k8s-0
2026-03-13T13:10:33.280635041Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-13T13:10:33.428632735Z Gathering data for ns/openshift-config...
2026-03-13T13:10:33.699326937Z INFO: Done collecting etcd information
2026-03-13T13:10:33.978138719Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-03-13T13:10:34.117066124Z Gathering data for ns/openshift-etcd...
2026-03-13T13:10:34.144634490Z Wrote inspect data to must-gather.
2026-03-13T13:10:34.184528428Z INFO: Getting status/config from prometheus-k8s-0
2026-03-13T13:10:34.316504412Z Wrote inspect data to must-gather.
2026-03-13T13:10:34.352355131Z Wrote inspect data to must-gather.
2026-03-13T13:10:34.446952808Z Gathering data for ns/openshift-config-managed...
2026-03-13T13:10:34.486027259Z Wrote inspect data to must-gather.
2026-03-13T13:10:34.682569246Z Wrote inspect data to must-gather.
2026-03-13T13:10:34.904239721Z INFO: INTERCONNECT MODE
2026-03-13T13:10:34.904239721Z INFO: Gathering ovn-kubernetes DBs
2026-03-13T13:10:35.096661826Z Wrote inspect data to must-gather.
2026-03-13T13:10:35.162327723Z volume usage percentage 0
2026-03-13T13:10:35.231267554Z INFO: Getting status/flags from prometheus-k8s-0
2026-03-13T13:10:35.242074317Z INFO: Gathering OVN_Northbound from ovnkube-node-vlrf6...
2026-03-13T13:10:35.242359456Z INFO: Gathering OVN_Southbound from ovnkube-node-vlrf6...
2026-03-13T13:10:35.261134603Z W0313 13:10:35.260979    1484 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-03-13T13:10:35.297300192Z Wrote inspect data to must-gather.
2026-03-13T13:10:35.420739775Z Wrote inspect data to must-gather.
2026-03-13T13:10:35.591109149Z Gathering data for ns/assisted-installer...
2026-03-13T13:10:35.681094229Z Gathering data for ns/openshift-authentication...
2026-03-13T13:10:35.700851817Z tar: Removing leading `/' from member names
2026-03-13T13:10:35.732279756Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-03-13T13:10:35.774557160Z tar: Removing leading `/' from member names
2026-03-13T13:10:35.779213718Z Wrote inspect data to must-gather.
2026-03-13T13:10:36.027203890Z Wrote inspect data to must-gather.
2026-03-13T13:10:36.125896235Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-03-13T13:10:36.132293349Z INFO: Waiting for network log collection to complete ...
2026-03-13T13:10:36.132777645Z INFO: Waiting for ovnk database copies to complete ...
2026-03-13T13:10:36.133243310Z INFO: Copying ovnk databases complete.
2026-03-13T13:10:36.139393555Z 2.0M	must-gather/network_logs/ovnk_database_store
2026-03-13T13:10:36.141852563Z Wrote inspect data to must-gather.
2026-03-13T13:10:36.144794077Z ovnk_database_store/
2026-03-13T13:10:36.144794077Z ovnk_database_store/ovnkube-node-vlrf6_sbdb
2026-03-13T13:10:36.296327403Z ovnk_database_store/ovnkube-node-vlrf6_nbdb
2026-03-13T13:10:36.359025255Z Wrote inspect data to must-gather.
2026-03-13T13:10:36.698163773Z INFO: Network log collection complete.
2026-03-13T13:10:37.114864356Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-03-13T13:10:37.189087155Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.230220222Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.243704261Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.260057580Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.268038294Z Gathering data for ns/openshift-authentication-operator...
2026-03-13T13:10:37.725961507Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.730569653Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.748124552Z Wrote inspect data to must-gather.
2026-03-13T13:10:37.759447832Z INFO: Getting status from alertmanager-main-0
2026-03-13T13:10:38.198414792Z Gathering data for ns/openshift-ingress...
2026-03-13T13:10:38.215281438Z Wrote inspect data to must-gather.
2026-03-13T13:10:38.232519836Z Wrote inspect data to must-gather.
2026-03-13T13:10:38.722653903Z Wrote inspect data to must-gather.
2026-03-13T13:10:38.778881680Z Wrote inspect data to must-gather.
2026-03-13T13:10:38.896212419Z Gathering data for ns/openshift-oauth-apiserver...
2026-03-13T13:10:39.257167070Z Wrote inspect data to must-gather.
2026-03-13T13:10:39.349548486Z Wrote inspect data to must-gather.
2026-03-13T13:10:39.728567312Z Gathering data for ns/openshift-machine-api...
2026-03-13T13:10:39.868387126Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f34dc492c80a3dee4643cc2291044750ac51e6e919b973de8723fa8b70bde70
2026-03-13T13:10:40.160298872Z /usr/bin/gather_metallb: line 53: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/reloader/reloader/logs/reloader.pid: No such file or directory
2026-03-13T13:10:40.160298872Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/frr.conf: No such file or directory
2026-03-13T13:10:40.167011476Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/frr.log: No such file or directory
2026-03-13T13:10:40.167990127Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/daemons: No such file or directory
2026-03-13T13:10:40.170527338Z /usr/bin/gather_metallb: line 23: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/vtysh.conf: No such file or directory
2026-03-13T13:10:40.170721794Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:40.200494460Z volume usage percentage 0
2026-03-13T13:10:40.230881766Z daemonset.apps/perf-node-gather-daemonset created
2026-03-13T13:10:40.467393573Z Error from server (BadRequest): container frr is not valid for pod speaker-zlvcv
2026-03-13T13:10:40.513801387Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:40.513801387Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:40.557048142Z Waiting for performance profile collector pods to become ready: 1
2026-03-13T13:10:41.095809994Z Error from server (BadRequest): container frr is not valid for pod speaker-zlvcv
2026-03-13T13:10:41.117375639Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:41.117375639Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:41.417743045Z Error from server (BadRequest): container frr is not valid for pod speaker-zlvcv
2026-03-13T13:10:41.439674632Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:41.439674632Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:41.801709768Z Error from server (BadRequest): container frr is not valid for pod speaker-zlvcv
2026-03-13T13:10:41.807923925Z Gathering data for ns/openshift-cloud-controller-manager-operator...
2026-03-13T13:10:41.815895239Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:41.815895239Z /usr/bin/gather_metallb: line 27: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:41.850954182Z Waiting for performance profile collector pods to become ready: 2
2026-03-13T13:10:42.143049726Z Error from server (BadRequest): container frr is not valid for pod speaker-zlvcv
2026-03-13T13:10:42.159031773Z /usr/bin/gather_metallb: line 28: must-gather/namespaces/metallb-system/pods/speaker-zlvcv/frr/frr/logs/dump_frr: No such file or directory
2026-03-13T13:10:42.634452544Z Gathering data for ns/openshift-cloud-controller-manager...
2026-03-13T13:10:43.269284889Z Waiting for performance profile collector pods to become ready: 3
2026-03-13T13:10:43.647189699Z Gathering data for ns/openshift-cloud-credential-operator...
2026-03-13T13:10:44.590038483Z Waiting for performance profile collector pods to become ready: 4
2026-03-13T13:10:45.237286674Z volume usage percentage 0
2026-03-13T13:10:45.479206841Z Gathering data for ns/openshift-config-operator...
2026-03-13T13:10:45.820381325Z Waiting for performance profile collector pods to become ready: 5
2026-03-13T13:10:46.258923262Z Gathering data for ns/openshift-console-operator...
2026-03-13T13:10:47.111133435Z Waiting for performance profile collector pods to become ready: 6
2026-03-13T13:10:47.178793466Z Gathering data for ns/openshift-console...
2026-03-13T13:10:47.913369292Z Gathering data for ns/openshift-cluster-storage-operator...
2026-03-13T13:10:48.279289061Z Waiting for performance profile collector pods to become ready: 7
2026-03-13T13:10:48.595683886Z Gathering data for ns/openshift-dns-operator...
2026-03-13T13:10:49.121198647Z Gathering data for ns/openshift-dns...
2026-03-13T13:10:49.497142485Z Waiting for performance profile collector pods to become ready: 8
2026-03-13T13:10:49.903325144Z Gathering data for ns/openshift-etcd-operator...
2026-03-13T13:10:50.565935425Z volume usage percentage 0
2026-03-13T13:10:50.737429823Z Daemonset perf-node-gather-daemonset ready 1 out of 1
2026-03-13T13:10:51.066626292Z Gathering data for ns/openshift-etcd...
2026-03-13T13:10:51.181647621Z Collecting performance related data for node master-0
2026-03-13T13:10:52.101589727Z Gathering data for ns/openshift-image-registry...
2026-03-13T13:10:52.777319592Z Gathering data for ns/openshift-ingress-operator...
2026-03-13T13:10:53.856260802Z Gathering data for ns/openshift-ingress-canary...
2026-03-13T13:10:54.444206697Z Gathering data for ns/openshift-insights...
2026-03-13T13:10:55.610023698Z volume usage percentage 0
2026-03-13T13:10:56.034419265Z Gathering data for ns/openshift-monitoring...
2026-03-13T13:10:59.164559314Z Gathering data for ns/metallb-system...
2026-03-13T13:11:00.649857508Z volume usage percentage 0
2026-03-13T13:11:00.957653101Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-03-13T13:11:02.862931751Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-03-13T13:11:03.618024889Z Gathering data for ns/openshift-kube-apiserver...
2026-03-13T13:11:04.926070580Z Gathering data for ns/openshift-catalogd...
2026-03-13T13:11:05.708995541Z volume usage percentage 0
2026-03-13T13:11:05.742964321Z Gathering data for ns/cert-manager...
2026-03-13T13:11:06.939075336Z Gathering data for ns/openshift-nmstate...
2026-03-13T13:11:07.687476250Z Gathering data for ns/openshift-operators...
2026-03-13T13:11:08.438666263Z Gathering data for ns/openshift-multus...
2026-03-13T13:11:09.219705464Z Gathering data for ns/openshift-storage...
2026-03-13T13:11:10.161780805Z Gathering data for ns/openshift-kube-controller-manager...
2026-03-13T13:11:10.756838096Z volume usage percentage 0
2026-03-13T13:11:11.220734859Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-03-13T13:11:11.578286212Z Collecting kubelet logs for node master-0
2026-03-13T13:11:11.896166804Z Gathering data for ns/kube-system...
2026-03-13T13:11:12.205371052Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-03-13T13:11:12.221636888Z INFO: Node performance data collection complete.
2026-03-13T13:11:12.810430211Z Gathering data for ns/openshift-kube-scheduler...
2026-03-13T13:11:13.731044849Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-03-13T13:11:14.366543605Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-03-13T13:11:14.811492286Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-03-13T13:11:15.499282564Z Gathering data for ns/openshift-cluster-machine-approver...
2026-03-13T13:11:15.805647391Z volume usage percentage 0
2026-03-13T13:11:16.057152004Z Gathering data for ns/openshift-machine-config-operator...
2026-03-13T13:11:17.307105259Z Gathering data for ns/openshift-kni-infra...
2026-03-13T13:11:18.011602528Z Gathering data for ns/openshift-openstack-infra...
2026-03-13T13:11:18.463405007Z Gathering data for ns/openshift-ovirt-infra...
2026-03-13T13:11:18.935727178Z Gathering data for ns/openshift-vsphere-infra...
2026-03-13T13:11:19.484695185Z Gathering data for ns/openshift-nutanix-infra...
2026-03-13T13:11:19.982350610Z Gathering data for ns/openshift-cloud-platform-infra...
2026-03-13T13:11:20.385735610Z Gathering data for ns/openshift-marketplace...
2026-03-13T13:11:21.032976872Z volume usage percentage 0
2026-03-13T13:11:23.631646607Z Gathering data for ns/openshift-user-workload-monitoring...
2026-03-13T13:11:25.343525433Z Gathering data for ns/openshift-ovn-kubernetes...
2026-03-13T13:11:26.106199442Z volume usage percentage 0
2026-03-13T13:11:27.156120219Z Gathering data for ns/openshift-host-network...
2026-03-13T13:11:27.692835356Z Gathering data for ns/openshift-network-diagnostics...
2026-03-13T13:11:28.927165924Z Gathering data for ns/openshift-network-node-identity...
2026-03-13T13:11:29.566286886Z Gathering data for ns/openshift-network-console...
2026-03-13T13:11:30.079395622Z Gathering data for ns/openshift-network-operator...
2026-03-13T13:11:30.848040761Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-03-13T13:11:31.224300148Z volume usage percentage 0
2026-03-13T13:11:31.790595886Z Gathering data for ns/openshift-operator-controller...
2026-03-13T13:11:32.805456969Z Gathering data for ns/openshift-cluster-olm-operator...
2026-03-13T13:11:33.425298058Z Gathering data for ns/openshift-apiserver-operator...
2026-03-13T13:11:34.304853791Z Gathering data for ns/openshift-apiserver...
2026-03-13T13:11:35.320633223Z Gathering data for ns/openshift-controller-manager-operator...
2026-03-13T13:11:35.882157309Z Gathering data for ns/openshift-controller-manager...
2026-03-13T13:11:36.319210918Z volume usage percentage 0
2026-03-13T13:11:36.510526499Z Gathering data for ns/openshift-route-controller-manager...
2026-03-13T13:11:37.129943625Z Gathering data for ns/openshift-cluster-samples-operator...
2026-03-13T13:11:38.320693708Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-03-13T13:11:39.984170844Z Gathering data for ns/openshift-service-ca-operator...
2026-03-13T13:11:40.479192507Z Gathering data for ns/openshift-service-ca...
2026-03-13T13:11:41.007231648Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-03-13T13:11:41.395124426Z volume usage percentage 0
2026-03-13T13:11:42.997188331Z Wrote inspect data to must-gather.
2026-03-13T13:11:42.997261533Z error: inspection completed with the errors occurred while gathering data:
2026-03-13T13:11:42.997261533Z     [skipping gathering clusterroles.rbac.authorization.k8s.io/system:registry due to error: clusterroles.rbac.authorization.k8s.io "system:registry" not found, skipping gathering clusterrolebindings.rbac.authorization.k8s.io/registry-registry-role due to error: clusterrolebindings.rbac.authorization.k8s.io "registry-registry-role" not found, skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering endpoints/host-etcd-2 due to error: endpoints "host-etcd-2" not found]
2026-03-13T13:11:43.231743575Z error: the server doesn't have a resource type "clusters"