Trigger GMR for Service cinder Trigger GMR for Nova services Will retrieve SOS reports from nodes crc Generating SOS Report for crc Generating SOS Report for EDPM compute-0 Generating SOS Report for EDPM compute-1 Starting pod/crc-debug-zdp6x ... To use host binaries, run `chroot /host` Warning: Permanently added '192.168.122.101' (ED25519) to the list of known hosts. Warning: Permanently added '192.168.122.100' (ED25519) to the list of known hosts. +--------------------------------------+----------------+--------------------------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +--------------------------------------+----------------+--------------------------------+----------+---------+-------+----------------------------+ | c220010b-eaaf-4dd0-91f4-94223d2cedb1 | nova-conductor | nova-cell0-conductor-0 | internal | enabled | up | 2025-09-30T19:24:05.000000 | | 030fb935-2abc-488c-93a7-ff015320e361 | nova-scheduler | nova-scheduler-0 | internal | enabled | up | 2025-09-30T19:24:10.000000 | | 973b06b4-4d3b-4d6e-a066-c32cd20ac71b | nova-conductor | nova-cell1-conductor-0 | internal | enabled | up | 2025-09-30T19:24:08.000000 | | 69720e80-c788-4734-8854-21bb7e41d622 | nova-compute | compute-0.ctlplane.example.com | nova | enabled | up | 2025-09-30T19:24:10.000000 | | ed484dfa-033a-4a53-9327-ca60d744c18d | nova-compute | compute-1.ctlplane.example.com | nova | enabled | up | 2025-09-30T19:24:11.000000 | +--------------------------------------+----------------+--------------------------------+----------+---------+-------+----------------------------+ sos report (version 4.10.0) sos report (version 4.10.0) This command will collect diagnostic and configuration information from this CentOS Linux system and installed applications. An archive containing the collected information will be generated in /var/tmp/sos-osp/sos.d6ms_b67 and may be provided to a CentOS support representative. Any information provided to CentOS will be treated in accordance with the published support policies at: Community Website : https://www.centos.org/ The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. Setting up archive ... Setting up plugins ... This command will collect diagnostic and configuration information from this CentOS Linux system and installed applications. An archive containing the collected information will be generated in /var/tmp/sos-osp/sos.sem38znb and may be provided to a CentOS support representative. Any information provided to CentOS will be treated in accordance with the published support policies at: Community Website : https://www.centos.org/ The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. Setting up archive ... Setting up plugins ... toolbox-osp Trying to pull registry.redhat.io/rhel9/support-tools:latest... Getting image source signatures Checking if image destination supports signatures Copying blob sha256:97718391176c91bbeea71172faf017b5982e3e383b2d2a6652ade6b25910938e Copying blob sha256:74956640af323393be3edafd6bbf4d02e5336ac16c17bd052e9127b7a9f77159 Copying config sha256:a46baefd96eef0f353f3454364d157bbbd4c6e35b14a551dee466c297887355c Writing manifest to image destination Storing signatures a46baefd96eef0f353f3454364d157bbbd4c6e35b14a551dee466c297887355c .toolboxrc file detected, overriding defaults... Checking if there is a newer version of registry.redhat.io/rhel9/support-tools available... Gathering CRDs [plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec. Use '--allow-system-changes' to enable collection. [plugin:openstack_neutron] Could not open conf file /etc/neutron/plugins/ml2/ml2_conf.ini: [Errno 2] No such file or directory: '/etc/neutron/plugins/ml2/ml2_conf.ini' Not all environment variables set. Source the environment file for the user intended to connect to the OpenStack environment. [plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec. Use '--allow-system-changes' to enable collection. [plugin:openstack_neutron] Could not open conf file /etc/neutron/plugins/ml2/ml2_conf.ini: [Errno 2] No such file or directory: '/etc/neutron/plugins/ml2/ml2_conf.ini' Not all environment variables set. Source the environment file for the user intended to connect to the OpenStack environment. [plugin:systemd] skipped command 'systemd-resolve --status': required services missing: systemd-resolved. [plugin:systemd] skipped command 'systemd-resolve --statistics': required services missing: systemd-resolved. [plugin:systemd] skipped command 'systemd-resolve --status': required services missing: systemd-resolved. [plugin:systemd] skipped command 'systemd-resolve --statistics': required services missing: systemd-resolved. Running plugins. Please wait ... Starting 3/77 ata [Running: ata] Starting 1/77 anaconda [Running: anaconda] Starting 4/77 auditd [Running: anaconda auditd] Starting 5/77 block [Running: anaconda auditd block] Starting 2/77 anacron [Running: anaconda auditd block anacron] Starting 6/77 boot [Running: anaconda auditd block boot] Starting 7/77 buildah [Running: auditd block boot buildah] Starting 8/77 ceph_common [Running: block boot buildah ceph_common] Running plugins. Please wait ... Starting 1/77 anaconda [Running: anaconda] Starting 2/77 anacron [Running: anaconda anacron] Starting 4/77 auditd [Running: anaconda anacron auditd] Starting 3/77 ata [Running: anaconda anacron auditd ata] Starting 5/77 block [Running: anaconda anacron auditd block] Starting 6/77 boot [Running: anaconda auditd block boot] Starting 7/77 buildah [Running: auditd block boot buildah] Starting 9/77 cgroups [Running: block boot buildah cgroups] Starting 8/77 ceph_common [Running: block boot buildah ceph_common] Starting 9/77 cgroups [Running: block boot buildah cgroups] Starting 10/77 chrony [Running: boot buildah cgroups chrony] Starting 11/77 console [Running: boot cgroups chrony console] Starting 12/77 containers_common [Running: boot cgroups console containers_common] Starting 13/77 coredump [Running: boot cgroups console coredump] Starting 14/77 cron [Running: boot cgroups console cron] Starting 15/77 crypto [Running: boot cgroups console crypto] Starting 10/77 chrony [Running: boot buildah cgroups chrony] Starting 16/77 dbus [Running: boot cgroups crypto dbus] Spawning a container 'toolbox-osp' with image 'registry.redhat.io/rhel9/support-tools' Detected RUN label in the container image. Using that as the default... Starting 17/77 devicemapper [Running: boot cgroups crypto devicemapper] Starting 11/77 console [Running: boot cgroups chrony console] Starting 18/77 devices [Running: boot cgroups devicemapper devices] 478a943b453ba16fcc92f6a94c8cac594ec8c596941e5ad80602e321cf14695f Starting 12/77 containers_common [Running: boot cgroups console containers_common] Starting 13/77 coredump [Running: boot cgroups console coredump] Starting 19/77 dnf [Running: boot cgroups devices dnf] Starting 14/77 cron [Running: boot cgroups console cron] Starting 20/77 filesys [Running: boot cgroups dnf filesys] Starting 15/77 crypto [Running: boot cgroups console crypto] Starting 16/77 dbus [Running: boot cgroups crypto dbus] Starting 17/77 devicemapper [Running: boot cgroups crypto devicemapper] Starting 18/77 devices [Running: boot cgroups devicemapper devices] Starting 19/77 dnf [Running: boot cgroups devices dnf] toolbox-osp Starting 20/77 filesys [Running: boot cgroups dnf filesys] Starting 21/77 firewall_tables [Running: boot cgroups dnf firewall_tables] sos report --batch --all-logs --only-plugins block,cifs,crio,devicemapper,devices,iscsi,lvm2,memory,multipath,nfs,nis,nvme,podman,process,processor,selinux,scsi,udev,logs,crypto --tmp-dir=/var/tmp/sos-osp; exit Starting 22/77 hardware [Running: boot cgroups dnf hardware] Starting 21/77 firewall_tables [Running: boot cgroups dnf firewall_tables] Starting 22/77 hardware [Running: boot cgroups dnf hardware] Starting 23/77 host [Running: boot cgroups dnf host] Starting 24/77 i18n [Running: boot dnf host i18n] Starting 25/77 iscsi [Running: boot dnf host iscsi] Starting 26/77 kdump [Running: boot dnf host kdump] Starting 27/77 kernel [Running: boot dnf kdump kernel] Starting 28/77 keyutils [Running: boot dnf kernel keyutils] Starting 29/77 krb5 [Running: boot dnf kernel krb5] Starting 30/77 kvm [Running: boot dnf kernel kvm] Starting 23/77 host [Running: boot cgroups dnf host] Starting 24/77 i18n [Running: boot dnf host i18n] Starting 25/77 iscsi [Running: boot dnf host iscsi] Starting 26/77 kdump [Running: boot dnf host kdump] Starting 27/77 kernel [Running: boot dnf kdump kernel] Starting 28/77 keyutils [Running: boot dnf kernel keyutils] Starting 29/77 krb5 [Running: boot dnf kernel krb5] Starting 30/77 kvm [Running: boot dnf kernel kvm] Starting 31/77 ldap [Running: boot dnf kvm ldap] Starting 32/77 libraries [Running: boot dnf kvm libraries] Starting 33/77 libvirt [Running: boot dnf kvm libvirt] Starting 34/77 login [Running: boot dnf kvm login] [?2004l sos report (version 4.9.2) Starting 35/77 logrotate [Running: boot dnf kvm logrotate] Starting 36/77 logs [Running: boot dnf kvm logs] Starting 31/77 ldap [Running: boot dnf kvm ldap] Starting 32/77 libraries [Running: boot dnf kvm libraries] Starting 33/77 libvirt [Running: boot dnf kvm libvirt] Starting 34/77 login [Running: boot dnf kvm login] Starting 37/77 lvm2 [Running: dnf kvm logs lvm2] Starting 38/77 md [Running: dnf logs lvm2 md] Starting 35/77 logrotate [Running: dnf kvm login logrotate] Starting 39/77 memory [Running: dnf logs lvm2 memory] Starting 36/77 logs [Running: dnf kvm login logs] Starting 37/77 lvm2 [Running: dnf kvm logs lvm2] Starting 40/77 multipath [Running: dnf logs lvm2 multipath] Starting 41/77 networking [Running: dnf logs multipath networking] Starting 42/77 networkmanager [Running: dnf logs networking networkmanager] Starting 38/77 md [Running: dnf kvm logs md] Starting 39/77 memory [Running: dnf kvm logs memory] Starting 40/77 multipath [Running: dnf logs memory multipath] Starting 41/77 networking [Running: dnf logs multipath networking] Starting 42/77 networkmanager [Running: dnf logs networking networkmanager] Starting 43/77 nfs [Running: dnf networking networkmanager nfs] Starting 44/77 numa [Running: dnf networking nfs numa] Starting 45/77 nvme [Running: dnf networking numa nvme] Starting 43/77 nfs [Running: dnf logs networking nfs] Starting 46/77 openhpi [Running: dnf networking numa openhpi] Starting 47/77 openstack_edpm [Running: dnf networking numa openstack_edpm] Starting 48/77 openstack_neutron [Running: dnf networking numa openstack_neutron] Starting 49/77 openstack_nova [Running: dnf networking openstack_neutron openstack_nova] Starting 50/77 openvswitch [Running: dnf networking openstack_nova openvswitch] Starting 51/77 ovn_host [Running: dnf networking openvswitch ovn_host] Starting 52/77 pam [Running: dnf networking openvswitch pam] Gathering CRs Starting 53/77 pci [Running: dnf networking openvswitch pci] Starting 44/77 numa [Running: dnf logs networking numa] Starting 54/77 podman [Running: dnf networking openvswitch podman] Starting 45/77 nvme [Running: dnf networking numa nvme] Starting 46/77 openhpi [Running: dnf networking nvme openhpi] Starting 47/77 openstack_edpm [Running: dnf networking nvme openstack_edpm] Starting 48/77 openstack_neutron [Running: dnf networking nvme openstack_neutron] Starting 49/77 openstack_nova [Running: dnf networking openstack_neutron openstack_nova] Starting 50/77 openvswitch [Running: dnf networking openstack_nova openvswitch] Starting 51/77 ovn_host [Running: dnf networking openvswitch ovn_host] Starting 52/77 pam [Running: dnf networking openvswitch pam] Starting 53/77 pci [Running: dnf networking openvswitch pci] Starting 54/77 podman [Running: dnf networking openvswitch podman] Starting 55/77 process [Running: dnf networking openvswitch process] Starting 55/77 process [Running: dnf networking podman process] Starting 56/77 processor [Running: dnf networking process processor] Starting 56/77 processor [Running: dnf networking process processor] Starting 57/77 python [Running: networking process processor python] Starting 58/77 release [Running: networking process processor release] Starting 59/77 rpm [Running: networking process processor rpm] Starting 60/77 sar [Running: process processor rpm sar] Starting 61/77 scsi [Running: process processor rpm scsi] Starting 57/77 python [Running: dnf process processor python] Starting 62/77 selinux [Running: process processor scsi selinux] Starting 63/77 services [Running: process processor selinux services] Starting 64/77 ssh [Running: process processor selinux ssh] Starting 65/77 sudo [Running: process processor selinux sudo] Starting 58/77 release [Running: dnf process processor release] This command will collect diagnostic and configuration information from this Red Hat Enterprise Linux system and installed applications. An archive containing the collected information will be generated in /host/var/tmp/sos-osp/sos.14xxc91o and may be provided to a Red Hat support representative. Any information provided to Red Hat will be treated in accordance with the published support policies at: Distribution Website : https://www.redhat.com/ Commercial Support : https://access.redhat.com/ The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. Setting up archive ... Starting 66/77 sunrpc [Running: process processor selinux sunrpc] Setting up plugins ... Starting 67/77 system [Running: process processor selinux system] Starting 59/77 rpm [Running: dnf process processor rpm] Starting 60/77 sar [Running: dnf processor rpm sar] Starting 61/77 scsi [Running: dnf processor rpm scsi] Starting 62/77 selinux [Running: dnf processor scsi selinux] Starting 63/77 services [Running: dnf processor selinux services] Starting 64/77 ssh [Running: dnf processor selinux ssh] Starting 65/77 sudo [Running: dnf processor selinux sudo] Starting 66/77 sunrpc [Running: dnf processor selinux sunrpc] Starting 67/77 system [Running: dnf processor selinux system] Starting 68/77 systemd [Running: processor selinux system systemd] Starting 68/77 systemd [Running: processor selinux system systemd] Starting 69/77 sysvipc [Running: processor selinux systemd sysvipc] Starting 70/77 tpm2 [Running: processor selinux systemd tpm2] Starting 71/77 tuned [Running: processor selinux systemd tuned] Running plugins. Please wait ... Starting 2/20 cifs [Running: cifs] Starting 4/20 crypto [Running: cifs crypto] Starting 3/20 crio [Running: cifs crypto crio] Starting 1/20 block [Running: cifs crypto crio block] Starting 5/20 devicemapper [Running: crypto crio block devicemapper] Starting 6/20 devices [Running: crypto crio block devices] Starting 7/20 iscsi [Running: crio block devices iscsi] Starting 8/20 logs [Running: crio block logs] Starting 9/20 lvm2 [Running: crio block logs lvm2] Starting 69/77 sysvipc [Running: processor selinux systemd sysvipc] Starting 70/77 tpm2 [Running: processor selinux systemd tpm2] Starting 72/77 udev [Running: processor selinux systemd udev] Starting 73/77 unbound [Running: processor selinux systemd unbound] Starting 74/77 vhostmd [Running: processor selinux systemd vhostmd] Starting 75/77 virsh [Running: processor selinux systemd virsh] Starting 71/77 tuned [Running: processor selinux systemd tuned] Starting 10/20 memory [Running: crio logs lvm2 memory] Starting 11/20 multipath [Running: crio logs lvm2 multipath] Starting 12/20 nfs [Running: crio logs multipath nfs] Starting 13/20 nis [Running: crio logs nfs nis] Starting 14/20 nvme [Running: crio logs nfs nvme] Starting 72/77 udev [Running: processor selinux systemd udev] Starting 15/20 podman [Running: crio logs nfs podman] Starting 73/77 unbound [Running: processor selinux systemd unbound] Starting 74/77 vhostmd [Running: processor selinux systemd vhostmd] Starting 75/77 virsh [Running: processor selinux systemd virsh] Starting 16/20 process [Running: crio logs podman process] Starting 76/77 xen [Running: processor selinux systemd xen] Finishing plugins [Running: processor selinux systemd] Starting 77/77 xfs [Running: processor selinux systemd xfs] Finishing plugins [Running: processor systemd xfs] Finishing plugins [Running: processor systemd] Starting 76/77 xen [Running: processor systemd virsh xen] Finishing plugins [Running: processor systemd virsh] Starting 77/77 xfs [Running: processor systemd virsh xfs] Finishing plugins [Running: processor systemd virsh] Finishing plugins [Running: processor] Finishing plugins [Running: processor systemd] Finishing plugins [Running: processor] Finished running plugins Finished running plugins Starting 17/20 processor [Running: crio logs podman processor] Starting 18/20 scsi [Running: crio podman processor scsi] Starting 19/20 selinux [Running: crio podman processor selinux] Finishing plugins [Running: crio podman selinux] Starting 20/20 udev [Running: crio podman selinux udev] Finishing plugins [Running: crio podman selinux] Finishing plugins [Running: crio podman] Creating compressed archive... Your sos report has been generated and saved in: /var/tmp/sos-osp/sosreport-compute-1-2025-09-30-kqaqqmu.tar.xz Size 17.71MiB Owner root sha256 369974d32aa2005e26ba628860a228840f27c4c157542945dc0b68c385ba7196 Please send this file to your support representative. Retrieving SOS Report for compute-1 Finished retrieving SOS Report for compute-1 Creating compressed archive... Your sos report has been generated and saved in: /var/tmp/sos-osp/sosreport-compute-0-2025-09-30-adcarpg.tar.xz Size 18.60MiB Owner root sha256 b21a822bbbdf904e2a868d17b21afc7c71acf8865224254ab434a1c4e91a5a4b Please send this file to your support representative. Retrieving SOS Report for compute-0 Finished retrieving SOS Report for compute-0 Writing validating webhooks Writing mutating webhooks Extracting ConfigMap: rabbitmq-cell1-config-data Extracting ConfigMap: rabbitmq-cell1-plugins-conf Extracting ConfigMap: rabbitmq-cell1-server-conf Extracting ConfigMap: rabbitmq-config-data Extracting ConfigMap: rabbitmq-plugins-conf Extracting ConfigMap: rabbitmq-server-conf Extracting ConfigMap: nova-extra-config Extracting ConfigMap: horizon-config-data Extracting ConfigMap: horizon-scripts Extracting ConfigMap: neutron-edpm-ipam Extracting ConfigMap: dns-swift-storage-0 Extracting ConfigMap: swift-ring-config-data Extracting ConfigMap: swift-ring-files Extracting ConfigMap: swift-ring-scripts Extracting ConfigMap: swift-storage-config-data Extracting ConfigMap: ovncontroller-config Extracting ConfigMap: ovncontroller-extra-scripts Extracting ConfigMap: ovncontroller-metrics-config Extracting ConfigMap: ovncontroller-scripts Extracting ConfigMap: ovndbcluster-nb-config Extracting ConfigMap: ovndbcluster-nb-scripts Extracting ConfigMap: ovndbcluster-sb-config Extracting ConfigMap: ovndbcluster-sb-scripts Extracting ConfigMap: ovnnorthd-config Extracting ConfigMap: ovnnorthd-scripts Extracting ConfigMap: ovsdbserver-nb Extracting ConfigMap: ovsdbserver-sb Extracting ConfigMap: prometheus-metric-storage-rulefiles-0 Extracting ConfigMap: infra-instanceha-config Gathering secrets in namespace openstack Finishing plugins [Running: podman] Finished running plugins Creating compressed archive... Error while masking key: prometheus.yaml.gz, for file: /must-gather/namespaces/openstack/secrets/metric-storage/prometheus-metric-storage.yaml Gathering csv Gathering subscription Gathering installplan Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for barbican-api from barbican-api-7c5546bcbd-84n4q pod Dump logs for barbican-api-log from barbican-api-7c5546bcbd-84n4q pod Dump logs for barbican-keystone-listener from barbican-keystone-listener-5768c5b854-k959d pod Dump logs for barbican-keystone-listener-log from barbican-keystone-listener-5768c5b854-k959d pod Dump logs for barbican-worker from barbican-worker-7b75d759d5-6bwm5 pod Dump logs for barbican-worker-log from barbican-worker-7b75d759d5-6bwm5 pod Dump logs for bootstrap-edpm-deployment-openstack-edpm-ipam from bootstrap-edpm-deployment-openstack-edpm-ipam-v9fdg pod Dump logs for ceilometer-central-agent from ceilometer-0 pod Dump logs for ceilometer-notification-agent from ceilometer-0 pod Dump logs for proxy-httpd from ceilometer-0 pod Dump logs for sg-core from ceilometer-0 pod Dump logs for cinder-api from cinder-api-0 pod Dump logs for cinder-api-log from cinder-api-0 pod Dump logs for cinder-scheduler from cinder-scheduler-0 pod Dump logs for probe from cinder-scheduler-0 pod Dump logs for configure-network-edpm-deployment-openstack-edpm-ipam from configure-network-edpm-deployment-openstack-edpm-ipam-k9785 pod Dump logs for configure-os-edpm-deployment-openstack-edpm-ipam from configure-os-edpm-deployment-openstack-edpm-ipam-g45vj pod Dump logs for configure-os-edpm-deployment-openstack-edpm-ipam from configure-os-edpm-deployment-openstack-edpm-ipam-vn7ct pod Dump logs for init from dnsmasq-dns-7fd9f947b7-sc66n pod Dump init container logs from dnsmasq-dns-7fd9f947b7-sc66n pod Dump logs for dnsmasq-dns from dnsmasq-dns-7fd9f947b7-sc66n pod Dump logs for download-cache-edpm-deployment-openstack-edpm-ipam from download-cache-edpm-deployment-openstack-edpm-ipam-fzj8h pod Dump logs for glance-httpd from glance-default-external-api-0 pod Dump logs for glance-log from glance-default-external-api-0 pod Dump logs for glance-httpd from glance-default-internal-api-0 pod Dump logs for glance-log from glance-default-internal-api-0 pod Dump logs for horizon from horizon-6676d4ddcd-sxf6l pod Dump logs for horizon-log from horizon-6676d4ddcd-sxf6l pod Dump logs for install-certs-edpm-deployment-openstack-edpm-ipam from install-certs-edpm-deployment-openstack-edpm-ipam-65dn7 pod Dump logs for install-os-edpm-deployment-openstack-edpm-ipam from install-os-edpm-deployment-openstack-edpm-ipam-k5jg9 pod Dump logs for keystone-api from keystone-6b748fb867-znqws pod Dump logs for keystone-cron from keystone-cron-29320981-sk9mv pod Dump logs for kube-state-metrics from kube-state-metrics-0 pod Dump logs for libvirt-edpm-deployment-openstack-edpm-ipam from libvirt-edpm-deployment-openstack-edpm-ipam-vzb94 pod Dump logs for memcached from memcached-0 pod Dump logs for neutron-api from neutron-6fc4d4d55c-fzms2 pod Dump logs for neutron-httpd from neutron-6fc4d4d55c-fzms2 pod Dump logs for neutron-metadata-edpm-deployment-openstack-edpm-ipam from neutron-metadata-edpm-deployment-openstack-edpm-ipam-tx67q pod Dump logs for nova-api-api from nova-api-0 pod Dump logs for nova-api-log from nova-api-0 pod Dump logs for nova-cell0-conductor-conductor from nova-cell0-conductor-0 pod Dump logs for nova-cell1-conductor-conductor from nova-cell1-conductor-0 pod Dump logs for nova-cell1-novncproxy-novncproxy from nova-cell1-novncproxy-0 pod Dump logs for nova-edpm-deployment-openstack-edpm-ipam from nova-edpm-deployment-openstack-edpm-ipam-92w66 pod Dump logs for nova-metadata-log from nova-metadata-0 pod Dump logs for nova-metadata-metadata from nova-metadata-0 pod Dump logs for nova-scheduler-scheduler from nova-scheduler-0 pod Dump logs for mysql-bootstrap from openstack-cell1-galera-0 pod Dump init container logs from openstack-cell1-galera-0 pod Dump logs for galera from openstack-cell1-galera-0 pod Dump logs for mysql-bootstrap from openstack-galera-0 pod Dump init container logs from openstack-galera-0 pod Dump logs for galera from openstack-galera-0 pod Dump logs for openstackclient from openstackclient pod Dump logs for openstack-network-exporter from ovn-controller-metrics-8bw4c pod Dump logs for ovn-controller from ovn-controller-nf4pk pod Dump logs for ovsdb-server-init from ovn-controller-ovs-dqkv4 pod Dump init container logs from ovn-controller-ovs-dqkv4 pod Dump logs for ovs-vswitchd from ovn-controller-ovs-dqkv4 pod Dump logs for ovsdb-server from ovn-controller-ovs-dqkv4 pod Dump logs for ovn-edpm-deployment-openstack-edpm-ipam from ovn-edpm-deployment-openstack-edpm-ipam-dmdz9 pod Dump logs for openstack-network-exporter from ovn-northd-0 pod Dump logs for ovn-northd from ovn-northd-0 pod Dump logs for openstack-network-exporter from ovsdbserver-nb-0 pod Dump logs for ovsdbserver-nb from ovsdbserver-nb-0 pod Dump logs for openstack-network-exporter from ovsdbserver-sb-0 pod Dump logs for ovsdbserver-sb from ovsdbserver-sb-0 pod Dump logs for placement-api from placement-c47f4984-nxfz7 pod Dump logs for placement-log from placement-c47f4984-nxfz7 pod Dump logs for init-config-reloader from prometheus-metric-storage-0 pod Dump init container logs from prometheus-metric-storage-0 pod Dump logs for config-reloader from prometheus-metric-storage-0 pod Dump logs for prometheus from prometheus-metric-storage-0 pod Dump logs for thanos-sidecar from prometheus-metric-storage-0 pod Dump logs for setup-container from rabbitmq-cell1-server-0 pod Dump init container logs from rabbitmq-cell1-server-0 pod Dump logs for rabbitmq from rabbitmq-cell1-server-0 pod Dump logs for setup-container from rabbitmq-server-0 pod Dump init container logs from rabbitmq-server-0 pod Dump logs for rabbitmq from rabbitmq-server-0 pod Dump logs for reboot-os-edpm-deployment-openstack-edpm-ipam from reboot-os-edpm-deployment-openstack-edpm-ipam-4tsdb pod Dump logs for redhat-edpm-deployment-openstack-edpm-ipam from redhat-edpm-deployment-openstack-edpm-ipam-j5hr2 pod Dump logs for repo-setup-edpm-deployment-openstack-edpm-ipam from repo-setup-edpm-deployment-openstack-edpm-ipam-jt9bh pod Dump logs for run-os-edpm-deployment-openstack-edpm-ipam from run-os-edpm-deployment-openstack-edpm-ipam-grthd pod Dump logs for ssh-known-hosts-edpm-deployment from ssh-known-hosts-edpm-deployment-rwq96 pod Dump logs for proxy-httpd from swift-proxy-594ff6944c-p2jp5 pod Dump logs for proxy-server from swift-proxy-594ff6944c-p2jp5 pod Dump logs for swift-ring-rebalance from swift-ring-rebalance-cx5sg pod Dump logs for account-auditor from swift-storage-0 pod Dump logs for account-reaper from swift-storage-0 pod Dump logs for account-replicator from swift-storage-0 pod Dump logs for account-server from swift-storage-0 pod Dump logs for container-auditor from swift-storage-0 pod Dump logs for container-replicator from swift-storage-0 pod Dump logs for container-server from swift-storage-0 pod Dump logs for container-updater from swift-storage-0 pod Dump logs for object-auditor from swift-storage-0 pod Dump logs for object-expirer from swift-storage-0 pod Dump logs for object-replicator from swift-storage-0 pod Dump logs for object-server from swift-storage-0 pod Dump logs for object-updater from swift-storage-0 pod Dump logs for rsync from swift-storage-0 pod Dump logs for swift-recon-cron from swift-storage-0 pod Dump logs for telemetry-edpm-deployment-openstack-edpm-ipam from telemetry-edpm-deployment-openstack-edpm-ipam-9wtwm pod Dump logs for tempest-tests-tempest-tests-runner from tempest-tests-tempest pod Dump logs for test-operator-logs-container from test-operator-logs-pod-tempest-tempest-tests-tempest pod Dump logs for validate-network-edpm-deployment-openstack-edpm-ipam from validate-network-edpm-deployment-openstack-edpm-ipam-fkswn pod Dump logs for watcher-api from watcher-api-0 pod Dump logs for watcher-api-log from watcher-api-0 pod Dump logs for watcher-applier from watcher-applier-0 pod Your sos report has been generated and saved in: /host/var/tmp/sos-osp/sosreport-crc-2025-09-30-mkpoiwx.tar.xz Size 45.98MiB Owner root sha256 2f28f1cfb3fb62a9c956d5400678208daa4d5e7b9a655724bc1d0e2bf21c5f29 Please send this file to your support representative. exit Dump logs for watcher-decision-engine from watcher-decision-engine-0 pod Dump services: barbican-internal Dump services: barbican-public tar: Removing leading `/' from member names tar: /var/log/pods/*/*.log.*: Warning: Cannot stat: No such file or directory Dump services: ceilometer-internal Dump services: cinder-internal Dump services: cinder-public Dump services: dnsmasq-dns Dump services: glance-default-external-api Dump services: glance-default-internal Dump services: glance-default-internal-api Dump services: glance-default-public Dump services: horizon Dump services: keystone-internal Dump services: keystone-public Dump services: kube-state-metrics tar: Removing leading `/' from hard link targets Dump services: memcached Dump services: metric-storage-prometheus Dump services: metric-storage-thanos-sidecar Dump services: neutron-internal Dump services: neutron-public Dump services: nova-internal Dump services: nova-metadata-internal Dump services: nova-novncproxy-cell1-public Dump services: nova-public Dump services: openstack Dump services: openstack-cell1 Dump services: openstack-cell1-galera Dump services: openstack-galera Dump services: ovn-controller-metrics Dump services: ovn-northd-0 Dump services: ovsdbserver-nb Dump services: ovsdbserver-nb-0 Dump services: ovsdbserver-sb Dump services: ovsdbserver-sb-0 Dump services: placement-internal Dump services: placement-public Dump services: prometheus-operated Dump services: rabbitmq Dump services: rabbitmq-cell1 Dump services: rabbitmq-cell1-nodes Dump services: rabbitmq-nodes Dump services: swift-internal Dump services: swift-public Dump services: swift-storage Dump services: watcher-internal Dump services: watcher-public Dump routes: barbican-public Dump routes: cinder-public Dump routes: glance-default-public Dump routes: horizon Dump routes: keystone-public Dump routes: neutron-public Dump routes: nova-novncproxy-cell1-public Dump routes: nova-public Dump routes: placement-public Dump routes: swift-public Dump routes: watcher-public Dump daemonset: ovn-controller Dump daemonset: ovn-controller-metrics Dump daemonset: ovn-controller-ovs Dump replicaset: barbican-api-5bbffffb8 Dump replicaset: barbican-api-7c5546bcbd Dump replicaset: barbican-keystone-listener-5768c5b854 Dump replicaset: barbican-worker-7b75d759d5 Dump replicaset: dnsmasq-dns-55fff446b9 Dump replicaset: dnsmasq-dns-5784cf869f Dump replicaset: dnsmasq-dns-59cf4bdb65 Dump replicaset: dnsmasq-dns-67b789f86c Dump replicaset: dnsmasq-dns-75c8ddd69c Dump replicaset: dnsmasq-dns-76fcf4b695 Dump replicaset: dnsmasq-dns-77585f5f8c Dump replicaset: dnsmasq-dns-7fd9f947b7 Dump replicaset: dnsmasq-dns-845d6d6f59 Dump replicaset: dnsmasq-dns-84b966f6c9 Dump replicaset: dnsmasq-dns-8b5c85b87 Dump replicaset: horizon-549f8c559c Dump replicaset: horizon-644644d7d5 Dump replicaset: horizon-6676d4ddcd Dump replicaset: horizon-758c546b97 Dump replicaset: horizon-ff7898f76 Dump replicaset: keystone-6b748fb867 Dump replicaset: neutron-6fc4d4d55c Dump replicaset: neutron-74f97446fb Dump replicaset: placement-c47f4984 Dump replicaset: swift-proxy-594ff6944c Dump deployments: barbican-api Dump deployments: barbican-keystone-listener Dump deployments: barbican-worker Dump deployments: dnsmasq-dns Dump deployments: horizon Dump deployments: keystone Dump deployments: neutron Dump deployments: placement Dump deployments: swift-proxy Dump statefulset: ceilometer Dump statefulset: cinder-api Dump statefulset: cinder-scheduler Dump statefulset: cinder-volume-volume1 Dump statefulset: glance-default-external-api Dump statefulset: glance-default-internal-api Dump statefulset: kube-state-metrics Dump statefulset: memcached Dump statefulset: nova-api Dump statefulset: nova-cell0-conductor Dump statefulset: nova-cell1-conductor Dump statefulset: nova-cell1-novncproxy Dump statefulset: nova-metadata Dump statefulset: nova-scheduler Dump statefulset: openstack-cell1-galera Dump statefulset: openstack-galera Dump statefulset: ovn-northd Dump statefulset: ovsdbserver-nb Dump statefulset: ovsdbserver-sb Dump statefulset: prometheus-metric-storage Dump statefulset: rabbitmq-cell1-server Dump statefulset: rabbitmq-server Dump statefulset: swift-storage Dump statefulset: watcher-api Dump statefulset: watcher-applier Dump statefulset: watcher-decision-engine Dump jobs: bootstrap-edpm-deployment-openstack-edpm-ipam Dump jobs: configure-network-edpm-deployment-openstack-edpm-ipam Dump jobs: configure-os-edpm-deployment-openstack-edpm-ipam Dump jobs: download-cache-edpm-deployment-openstack-edpm-ipam Dump jobs: install-certs-edpm-deployment-openstack-edpm-ipam Dump jobs: install-os-edpm-deployment-openstack-edpm-ipam Dump jobs: keystone-cron-29320981 Dump jobs: libvirt-edpm-deployment-openstack-edpm-ipam Dump jobs: neutron-metadata-edpm-deployment-openstack-edpm-ipam Dump jobs: nova-edpm-deployment-openstack-edpm-ipam Dump jobs: ovn-edpm-deployment-openstack-edpm-ipam Dump jobs: reboot-os-edpm-deployment-openstack-edpm-ipam Dump jobs: redhat-edpm-deployment-openstack-edpm-ipam Dump jobs: repo-setup-edpm-deployment-openstack-edpm-ipam Dump jobs: run-os-edpm-deployment-openstack-edpm-ipam Dump jobs: ssh-known-hosts-edpm-deployment Dump jobs: swift-ring-rebalance Dump jobs: telemetry-edpm-deployment-openstack-edpm-ipam Dump jobs: validate-network-edpm-deployment-openstack-edpm-ipam Dump cronjobs: cinder-db-purge Dump cronjobs: glance-db-purge Dump cronjobs: keystone-cron Dump cronjobs: nova-cell0-db-purge Dump cronjobs: nova-cell1-db-purge Dump cronjobs: watcher-db-purge Dump cert: barbican-internal-svc Dump cert: barbican-public-route Dump cert: barbican-public-svc Dump cert: ceilometer-internal-svc Dump cert: cinder-internal-svc Dump cert: cinder-public-route Dump cert: cinder-public-svc Dump cert: galera-openstack-cell1-svc Dump cert: galera-openstack-svc Dump cert: glance-default-internal-svc Dump cert: glance-default-public-route Dump cert: glance-default-public-svc Dump cert: horizon-route Dump cert: horizon-svc Dump cert: keystone-internal-svc Dump cert: keystone-public-route Dump cert: keystone-public-svc Dump cert: kube-state-metrics-svc Dump cert: libvirt-default-compute-0 Dump cert: libvirt-default-compute-1 Dump cert: memcached-svc Dump cert: metric-storage-prometheus-svc Dump cert: neutron-internal-svc Dump cert: neutron-metadata-default-compute-0 Dump cert: neutron-metadata-default-compute-1 Dump cert: neutron-ovndbs Dump cert: neutron-public-route Dump cert: neutron-public-svc Dump cert: nova-internal-svc Dump cert: nova-metadata-internal-svc Dump cert: nova-novncproxy-cell1-public-route Dump cert: nova-novncproxy-cell1-public-svc Dump cert: nova-novncproxy-cell1-vencrypt Dump cert: nova-public-route Dump cert: nova-public-svc Dump cert: ovn-default-compute-0 Dump cert: ovn-default-compute-1 Dump cert: ovn-metrics Dump cert: ovncontroller-ovndbs Dump cert: ovndbcluster-nb-ovndbs Dump cert: ovndbcluster-sb-ovndbs Dump cert: ovnnorthd-ovndbs Dump cert: placement-internal-svc Dump cert: placement-public-route Dump cert: placement-public-svc Dump cert: rabbitmq-cell1-svc Dump cert: rabbitmq-svc Dump cert: rootca-internal Dump cert: rootca-libvirt Dump cert: rootca-ovn Dump cert: rootca-public Dump cert: swift-internal-svc Dump cert: swift-public-route Dump cert: swift-public-svc Dump cert: telemetry-default-compute-0 Dump cert: telemetry-default-compute-1 Dump cert: watcher-internal-svc Dump cert: watcher-public-route Dump cert: watcher-public-svc Dump issuer: rootca-internal Dump issuer: rootca-libvirt Dump issuer: rootca-ovn Dump issuer: rootca-public Dump issuer: selfsigned-issuer Dump certificaterequests: barbican-internal-svc-1 Dump certificaterequests: barbican-public-route-1 Dump certificaterequests: barbican-public-svc-1 Dump certificaterequests: ceilometer-internal-svc-1 Dump certificaterequests: cinder-internal-svc-1 Dump certificaterequests: cinder-public-route-1 Dump certificaterequests: cinder-public-svc-1 Dump certificaterequests: galera-openstack-cell1-svc-1 Dump certificaterequests: galera-openstack-svc-1 Dump certificaterequests: glance-default-internal-svc-1 Dump certificaterequests: glance-default-public-route-1 Dump certificaterequests: glance-default-public-svc-1 Dump certificaterequests: horizon-route-1 Dump certificaterequests: horizon-svc-1 Dump certificaterequests: keystone-internal-svc-1 Dump certificaterequests: keystone-public-route-1 Dump certificaterequests: keystone-public-svc-1 Dump certificaterequests: kube-state-metrics-svc-1 Dump certificaterequests: libvirt-default-compute-0-1 Dump certificaterequests: libvirt-default-compute-1-1 Dump certificaterequests: memcached-svc-1 Dump certificaterequests: metric-storage-prometheus-svc-1 Dump certificaterequests: neutron-internal-svc-1 Dump certificaterequests: neutron-metadata-default-compute-0-1 Dump certificaterequests: neutron-metadata-default-compute-1-1 Dump certificaterequests: neutron-ovndbs-1 Dump certificaterequests: neutron-public-route-1 Dump certificaterequests: neutron-public-svc-1 Dump certificaterequests: nova-internal-svc-1 Dump certificaterequests: nova-metadata-internal-svc-1 Dump certificaterequests: nova-novncproxy-cell1-public-route-1 Dump certificaterequests: nova-novncproxy-cell1-public-svc-1 Dump certificaterequests: nova-novncproxy-cell1-vencrypt-1 Dump certificaterequests: nova-public-route-1 Dump certificaterequests: nova-public-svc-1 Dump certificaterequests: ovn-default-compute-0-1 Dump certificaterequests: ovn-default-compute-1-1 Dump certificaterequests: ovn-metrics-1 Dump certificaterequests: ovncontroller-ovndbs-1 Dump certificaterequests: ovndbcluster-nb-ovndbs-1 Dump certificaterequests: ovndbcluster-sb-ovndbs-1 Dump certificaterequests: ovnnorthd-ovndbs-1 Dump certificaterequests: placement-internal-svc-1 Dump certificaterequests: placement-public-route-1 Dump certificaterequests: placement-public-svc-1 Dump certificaterequests: rabbitmq-cell1-svc-1 Dump certificaterequests: rabbitmq-svc-1 Dump certificaterequests: rootca-internal-1 Dump certificaterequests: rootca-libvirt-1 Dump certificaterequests: rootca-ovn-1 Dump certificaterequests: rootca-public-1 Dump certificaterequests: swift-internal-svc-1 Dump certificaterequests: swift-public-route-1 Dump certificaterequests: swift-public-svc-1 Dump certificaterequests: telemetry-default-compute-0-1 Dump certificaterequests: telemetry-default-compute-1-1 Dump certificaterequests: watcher-internal-svc-1 Dump certificaterequests: watcher-public-route-1 Dump certificaterequests: watcher-public-svc-1 Gathering secrets in namespace openstack-operators Removing debug pod ... Retrieving SOS Report for crc Starting pod/crc-debug-c2rx5 ... To use host binaries, run `chroot /host` Removing debug pod ... Gathering csv Gathering subscription Gathering installplan No resources found in openstack-operators namespace. Dump logs for util from 32696112cd108f810e92200d76b5367dd09d983c683f8a54ad13dc2a2asnwj6 pod Dump init container logs from 32696112cd108f810e92200d76b5367dd09d983c683f8a54ad13dc2a2asnwj6 pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for pull from 32696112cd108f810e92200d76b5367dd09d983c683f8a54ad13dc2a2asnwj6 pod Dump init container logs from 32696112cd108f810e92200d76b5367dd09d983c683f8a54ad13dc2a2asnwj6 pod Dump logs for extract from 32696112cd108f810e92200d76b5367dd09d983c683f8a54ad13dc2a2asnwj6 pod Dump logs for kube-rbac-proxy from barbican-operator-controller-manager-6ff8b75857-csf68 pod Dump logs for manager from barbican-operator-controller-manager-6ff8b75857-csf68 pod Dump logs for kube-rbac-proxy from cinder-operator-controller-manager-644bddb6d8-wvthj pod Dump logs for manager from cinder-operator-controller-manager-644bddb6d8-wvthj pod Dump logs for kube-rbac-proxy from designate-operator-controller-manager-84f4f7b77b-85wtx pod Dump logs for manager from designate-operator-controller-manager-84f4f7b77b-85wtx pod Dump logs for kube-rbac-proxy from glance-operator-controller-manager-84958c4d49-585mg pod Dump logs for manager from glance-operator-controller-manager-84958c4d49-585mg pod Dump logs for kube-rbac-proxy from heat-operator-controller-manager-5d889d78cf-h2z55 pod Dump logs for manager from heat-operator-controller-manager-5d889d78cf-h2z55 pod Dump logs for kube-rbac-proxy from horizon-operator-controller-manager-9f4696d94-d2mm8 pod Dump logs for manager from horizon-operator-controller-manager-9f4696d94-d2mm8 pod Dump logs for kube-rbac-proxy from infra-operator-controller-manager-7d857cc749-9zcc2 pod Dump logs for manager from infra-operator-controller-manager-7d857cc749-9zcc2 pod Dump logs for kube-rbac-proxy from ironic-operator-controller-manager-7975b88857-4wpww pod Dump logs for manager from ironic-operator-controller-manager-7975b88857-4wpww pod Dump logs for kube-rbac-proxy from keystone-operator-controller-manager-5bd55b4bff-slc6t pod Dump logs for manager from keystone-operator-controller-manager-5bd55b4bff-slc6t pod Dump logs for kube-rbac-proxy from manila-operator-controller-manager-6d68dbc695-vlc4g pod Dump logs for manager from manila-operator-controller-manager-6d68dbc695-vlc4g pod Dump logs for kube-rbac-proxy from mariadb-operator-controller-manager-88c7-n564q pod Dump logs for manager from mariadb-operator-controller-manager-88c7-n564q pod Dump logs for kube-rbac-proxy from neutron-operator-controller-manager-64d7b59854-qlrv8 pod Dump logs for manager from neutron-operator-controller-manager-64d7b59854-qlrv8 pod Dump logs for kube-rbac-proxy from nova-operator-controller-manager-c7c776c96-d7r9l pod Dump logs for manager from nova-operator-controller-manager-c7c776c96-d7r9l pod Dump logs for kube-rbac-proxy from octavia-operator-controller-manager-76fcc6dc7c-c58zh pod Dump logs for manager from octavia-operator-controller-manager-76fcc6dc7c-c58zh pod Dump logs for kube-rbac-proxy from openstack-baremetal-operator-controller-manager-6d776955-zj2w6 pod Dump logs for manager from openstack-baremetal-operator-controller-manager-6d776955-zj2w6 pod Dump logs for kube-rbac-proxy from openstack-operator-controller-manager-5dc7c668c-46p2f pod Dump logs for manager from openstack-operator-controller-manager-5dc7c668c-46p2f pod Dump logs for kube-rbac-proxy from openstack-operator-controller-operator-5c6649c9b9-x9zfh pod Dump logs for operator from openstack-operator-controller-operator-5c6649c9b9-x9zfh pod Dump logs for registry-server from openstack-operator-index-qkts8 pod Dump logs for kube-rbac-proxy from ovn-operator-controller-manager-9976ff44c-dbxcb pod Dump logs for manager from ovn-operator-controller-manager-9976ff44c-dbxcb pod Dump logs for kube-rbac-proxy from placement-operator-controller-manager-589c58c6c-xh5rb pod Dump logs for manager from placement-operator-controller-manager-589c58c6c-xh5rb pod Dump logs for operator from rabbitmq-cluster-operator-manager-79d8469568-gcwrs pod Dump logs for kube-rbac-proxy from swift-operator-controller-manager-bc7dc7bd9-grp8f pod Dump logs for manager from swift-operator-controller-manager-bc7dc7bd9-grp8f pod Dump logs for kube-rbac-proxy from telemetry-operator-controller-manager-b8d54b5d7-5vhf6 pod Dump logs for manager from telemetry-operator-controller-manager-b8d54b5d7-5vhf6 pod Dump logs for kube-rbac-proxy from test-operator-controller-manager-f66b554c6-66ff6 pod Dump logs for manager from test-operator-controller-manager-f66b554c6-66ff6 pod Dump logs for kube-rbac-proxy from watcher-operator-controller-manager-598db9dcc9-jbsh8 pod Dump logs for manager from watcher-operator-controller-manager-598db9dcc9-jbsh8 pod Dump services: barbican-operator-controller-manager-metrics-service Dump services: cinder-operator-controller-manager-metrics-service Dump services: controller-operator-metrics-service-operator Dump services: designate-operator-controller-manager-metrics-service Dump services: glance-operator-controller-manager-metrics-service Dump services: heat-operator-controller-manager-metrics-service Dump services: horizon-operator-controller-manager-metrics-service Dump services: infra-operator-controller-manager-metrics-service Dump services: infra-operator-webhook-service Dump services: ironic-operator-controller-manager-metrics-service Dump services: keystone-operator-controller-manager-metrics-service Dump services: manila-operator-controller-manager-metrics-service Dump services: mariadb-operator-controller-manager-metrics-service Dump services: neutron-operator-controller-manager-metrics-service Dump services: nova-operator-controller-manager-metrics-service Dump services: octavia-operator-controller-manager-metrics-service Dump services: openstack-baremetal-operator-controller-manager-metrics-service Dump services: openstack-baremetal-operator-webhook-service Dump services: openstack-operator-controller-manager-metrics-service Dump services: openstack-operator-index Dump services: openstack-operator-webhook-service Dump services: ovn-operator-controller-manager-metrics-service Dump services: placement-operator-controller-manager-metrics-service Dump services: rabbitmq-cluster-operator-controller-manager-metrics-service Dump services: swift-operator-controller-manager-metrics-service Dump services: telemetry-operator-controller-manager-metrics-service Dump services: test-operator-controller-manager-metrics-service Dump services: watcher-operator-controller-manager-metrics-service Dump replicaset: barbican-operator-controller-manager-6ff8b75857 Dump replicaset: cinder-operator-controller-manager-644bddb6d8 Dump replicaset: designate-operator-controller-manager-84f4f7b77b Dump replicaset: glance-operator-controller-manager-84958c4d49 Dump replicaset: heat-operator-controller-manager-5d889d78cf Dump replicaset: horizon-operator-controller-manager-9f4696d94 Dump replicaset: infra-operator-controller-manager-7d857cc749 Dump replicaset: ironic-operator-controller-manager-7975b88857 Dump replicaset: keystone-operator-controller-manager-5bd55b4bff Dump replicaset: manila-operator-controller-manager-6d68dbc695 Dump replicaset: mariadb-operator-controller-manager-88c7 Dump replicaset: neutron-operator-controller-manager-64d7b59854 Dump replicaset: nova-operator-controller-manager-c7c776c96 Dump replicaset: octavia-operator-controller-manager-76fcc6dc7c Dump replicaset: openstack-baremetal-operator-controller-manager-6d776955 Dump replicaset: openstack-operator-controller-manager-5dc7c668c Dump replicaset: openstack-operator-controller-operator-5c6649c9b9 Dump replicaset: ovn-operator-controller-manager-9976ff44c Dump replicaset: placement-operator-controller-manager-589c58c6c Dump replicaset: rabbitmq-cluster-operator-manager-79d8469568 Dump replicaset: swift-operator-controller-manager-bc7dc7bd9 Dump replicaset: telemetry-operator-controller-manager-b8d54b5d7 Dump replicaset: test-operator-controller-manager-f66b554c6 Dump replicaset: watcher-operator-controller-manager-598db9dcc9 Dump deployments: barbican-operator-controller-manager Dump deployments: cinder-operator-controller-manager Dump deployments: designate-operator-controller-manager Dump deployments: glance-operator-controller-manager Dump deployments: heat-operator-controller-manager Dump deployments: horizon-operator-controller-manager Dump deployments: infra-operator-controller-manager Dump deployments: ironic-operator-controller-manager Dump deployments: keystone-operator-controller-manager Dump deployments: manila-operator-controller-manager Dump deployments: mariadb-operator-controller-manager Dump deployments: neutron-operator-controller-manager Dump deployments: nova-operator-controller-manager Dump deployments: octavia-operator-controller-manager Dump deployments: openstack-baremetal-operator-controller-manager Dump deployments: openstack-operator-controller-manager Dump deployments: openstack-operator-controller-operator Dump deployments: ovn-operator-controller-manager Dump deployments: placement-operator-controller-manager Dump deployments: rabbitmq-cluster-operator-manager Dump deployments: swift-operator-controller-manager Dump deployments: telemetry-operator-controller-manager Dump deployments: test-operator-controller-manager Dump deployments: watcher-operator-controller-manager Dump jobs: 32696112cd108f810e92200d76b5367dd09d983c683f8a54ad13dc2a2a0a899 Dump cert: infra-operator-serving-cert Dump cert: openstack-baremetal-operator-serving-cert Dump cert: openstack-operator-serving-cert Dump issuer: infra-operator-selfsigned-issuer Dump issuer: openstack-baremetal-operator-selfsigned-issuer Dump issuer: openstack-operator-selfsigned-issuer Dump certificaterequests: infra-operator-serving-cert-1 Dump certificaterequests: openstack-baremetal-operator-serving-cert-1 Dump certificaterequests: openstack-operator-serving-cert-1 Gathering secrets in namespace baremetal-operator-system Gathering secrets in namespace openshift-machine-api Gathering csv Gathering subscription Gathering installplan No resources found in openshift-machine-api namespace. Dump logs for control-plane-machine-set-operator from control-plane-machine-set-operator-78cbb6b69f-ndhrr pod Dump logs for kube-rbac-proxy from machine-api-operator-5694c8668f-8p4lg pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for machine-api-operator from machine-api-operator-5694c8668f-8p4lg pod Dump services: cluster-autoscaler-operator Dump services: control-plane-machine-set-operator Dump services: machine-api-controllers Dump services: machine-api-operator Dump services: machine-api-operator-machine-webhook Dump services: machine-api-operator-webhook Dump replicaset: control-plane-machine-set-operator-78cbb6b69f Dump replicaset: machine-api-operator-5694c8668f Dump deployments: control-plane-machine-set-operator Dump deployments: machine-api-operator Gathering secrets in namespace cert-manager Gathering csv Gathering subscription Gathering installplan No resources found in cert-manager namespace. Dump logs for cert-manager-controller from cert-manager-5b446d88c5-2wj9r pod Dump logs for cert-manager-cainjector from cert-manager-cainjector-7f985d654d-qj8wf pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for cert-manager-webhook from cert-manager-webhook-5655c58dd6-77l4k pod Dump services: cert-manager Dump services: cert-manager-webhook Dump replicaset: cert-manager-5b446d88c5 Dump replicaset: cert-manager-cainjector-7f985d654d Dump replicaset: cert-manager-webhook-5655c58dd6 Dump deployments: cert-manager Dump deployments: cert-manager-cainjector Dump deployments: cert-manager-webhook Gathering secrets in namespace openshift-nmstate Gathering csv Gathering subscription Gathering installplan No resources found in openshift-nmstate namespace. Dump logs for nmstate-console-plugin from nmstate-console-plugin-864bb6dfb5-g9n7q pod Dump logs for nmstate-handler from nmstate-handler-x2tc2 pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for kube-rbac-proxy from nmstate-metrics-58fcddf996-4nbj6 pod Dump logs for nmstate-metrics from nmstate-metrics-58fcddf996-4nbj6 pod Dump logs for nmstate-operator from nmstate-operator-5d6f6cfd66-d2f5j pod Dump logs for nmstate-webhook from nmstate-webhook-6d689559c5-bxqvs pod Dump services: nmstate-console-plugin Dump services: nmstate-monitor Dump services: nmstate-webhook Dump daemonset: nmstate-handler Dump replicaset: nmstate-console-plugin-864bb6dfb5 Dump replicaset: nmstate-metrics-58fcddf996 Dump replicaset: nmstate-operator-5d6f6cfd66 Dump replicaset: nmstate-webhook-6d689559c5 Dump deployments: nmstate-console-plugin Dump deployments: nmstate-metrics Dump deployments: nmstate-operator Dump deployments: nmstate-webhook Gathering secrets in namespace openshift-operators-redhat Gathering secrets in namespace openshift-logging Gathering secrets in namespace metallb-system Gathering csv Gathering subscription Gathering installplan Dump logs for controller from controller-5d688f5ffc-kr68l pod No resources found in metallb-system namespace. Dump logs for kube-rbac-proxy from controller-5d688f5ffc-kr68l pod Dump logs for cp-frr-files from frr-k8s-hwndx pod Dump init container logs from frr-k8s-hwndx pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for cp-reloader from frr-k8s-hwndx pod Dump init container logs from frr-k8s-hwndx pod Dump logs for cp-metrics from frr-k8s-hwndx pod Dump init container logs from frr-k8s-hwndx pod Dump logs for controller from frr-k8s-hwndx pod Dump logs for frr from frr-k8s-hwndx pod Dump logs for frr-metrics from frr-k8s-hwndx pod Dump logs for kube-rbac-proxy from frr-k8s-hwndx pod Dump logs for kube-rbac-proxy-frr from frr-k8s-hwndx pod Dump logs for reloader from frr-k8s-hwndx pod Dump logs for frr-k8s-webhook-server from frr-k8s-webhook-server-5478bdb765-667c9 pod Dump logs for manager from metallb-operator-controller-manager-6cb9fbcf6-fj2qz pod Dump logs for webhook-server from metallb-operator-webhook-server-6f4f79dfb8-6s6zz pod Dump logs for kube-rbac-proxy from speaker-dsmgz pod Dump logs for speaker from speaker-dsmgz pod Dump services: controller-monitor-service Dump services: frr-k8s-monitor-service Dump services: frr-k8s-webhook-service Dump services: metallb-operator-controller-manager-service Dump services: metallb-operator-webhook-server-service Dump services: metallb-operator-webhook-service Dump services: speaker-monitor-service Dump services: webhook-service Dump daemonset: frr-k8s Dump daemonset: speaker Dump replicaset: controller-5d688f5ffc Dump replicaset: frr-k8s-webhook-server-5478bdb765 Dump replicaset: metallb-operator-controller-manager-6cb9fbcf6 Dump replicaset: metallb-operator-webhook-server-6f4f79dfb8 Dump deployments: controller Dump deployments: frr-k8s-webhook-server Dump deployments: metallb-operator-controller-manager Dump deployments: metallb-operator-webhook-server Gathering secrets in namespace openshift-marketplace Gathering csv Gathering subscription Gathering installplan No resources found in openshift-marketplace namespace. Dump logs for util from 9a6e092ce660f08e14c0b0ceab3711fa43f2b70244f9df8a7a069040bcjkg2z pod Dump init container logs from 9a6e092ce660f08e14c0b0ceab3711fa43f2b70244f9df8a7a069040bcjkg2z pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for pull from 9a6e092ce660f08e14c0b0ceab3711fa43f2b70244f9df8a7a069040bcjkg2z pod Dump init container logs from 9a6e092ce660f08e14c0b0ceab3711fa43f2b70244f9df8a7a069040bcjkg2z pod Dump logs for extract from 9a6e092ce660f08e14c0b0ceab3711fa43f2b70244f9df8a7a069040bcjkg2z pod Dump logs for util from a6d815214afcb93f379916e45350d3de39072121f31a1d7eaaf6e22c2dlgh5l pod Dump init container logs from a6d815214afcb93f379916e45350d3de39072121f31a1d7eaaf6e22c2dlgh5l pod Dump logs for pull from a6d815214afcb93f379916e45350d3de39072121f31a1d7eaaf6e22c2dlgh5l pod Dump init container logs from a6d815214afcb93f379916e45350d3de39072121f31a1d7eaaf6e22c2dlgh5l pod Dump logs for extract from a6d815214afcb93f379916e45350d3de39072121f31a1d7eaaf6e22c2dlgh5l pod Dump logs for extract-utilities from certified-operators-8gwld pod Dump init container logs from certified-operators-8gwld pod Dump logs for extract-content from certified-operators-8gwld pod Dump init container logs from certified-operators-8gwld pod Dump logs for registry-server from certified-operators-8gwld pod Dump logs for extract-utilities from community-operators-8pxk9 pod Dump init container logs from community-operators-8pxk9 pod Dump logs for extract-content from community-operators-8pxk9 pod Dump init container logs from community-operators-8pxk9 pod Dump logs for registry-server from community-operators-8pxk9 pod Dump logs for util from f29efc416ca216184f30dbb4b19e0f463bdcecc8ef634322abbad88d96tgt78 pod Dump init container logs from f29efc416ca216184f30dbb4b19e0f463bdcecc8ef634322abbad88d96tgt78 pod Dump logs for pull from f29efc416ca216184f30dbb4b19e0f463bdcecc8ef634322abbad88d96tgt78 pod Dump init container logs from f29efc416ca216184f30dbb4b19e0f463bdcecc8ef634322abbad88d96tgt78 pod Dump logs for extract from f29efc416ca216184f30dbb4b19e0f463bdcecc8ef634322abbad88d96tgt78 pod Dump logs for marketplace-operator from marketplace-operator-79b997595-7qh78 pod Dump logs for extract-utilities from redhat-marketplace-8k7hb pod Dump init container logs from redhat-marketplace-8k7hb pod Dump logs for extract-content from redhat-marketplace-8k7hb pod Dump init container logs from redhat-marketplace-8k7hb pod Dump logs for registry-server from redhat-marketplace-8k7hb pod Dump logs for extract-utilities from redhat-operators-zgrpv pod Dump init container logs from redhat-operators-zgrpv pod Dump logs for extract-content from redhat-operators-zgrpv pod Dump init container logs from redhat-operators-zgrpv pod Dump logs for registry-server from redhat-operators-zgrpv pod Dump services: certified-operators Dump services: community-operators Dump services: marketplace-operator-metrics Dump services: redhat-marketplace Dump services: redhat-operators Dump replicaset: marketplace-operator-79b997595 Dump deployments: marketplace-operator Dump jobs: 9a6e092ce660f08e14c0b0ceab3711fa43f2b70244f9df8a7a069040bce86ee Dump jobs: a6d815214afcb93f379916e45350d3de39072121f31a1d7eaaf6e22c2d8e94a Dump jobs: f29efc416ca216184f30dbb4b19e0f463bdcecc8ef634322abbad88d964a8fa Gathering secrets in namespace openshift-operators Gathering csv Gathering subscription Gathering installplan No resources found in openshift-operators namespace. Dump logs for prometheus-operator from obo-prometheus-operator-7c8cf85677-4rpbl pod Dump logs for prometheus-operator-admission-webhook from obo-prometheus-operator-admission-webhook-f9c5648db-9pbgh pod Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ Dump logs for prometheus-operator-admission-webhook from obo-prometheus-operator-admission-webhook-f9c5648db-bvsr4 pod Dump logs for operator from observability-operator-cc5f78dfc-nvvj2 pod Dump logs for perses-operator from perses-operator-54bc95c9fb-r4wwh pod Dump services: obo-prometheus-operator Dump services: obo-prometheus-operator-admission-webhook Dump services: obo-prometheus-operator-admission-webhook-service Dump services: observability-operator Dump replicaset: obo-prometheus-operator-7c8cf85677 Dump replicaset: obo-prometheus-operator-admission-webhook-f9c5648db Dump replicaset: observability-operator-cc5f78dfc Dump replicaset: perses-operator-54bc95c9fb Dump deployments: obo-prometheus-operator Dump deployments: obo-prometheus-operator-admission-webhook Dump deployments: observability-operator Dump deployments: perses-operator Gather ctlplane service info: openstack Gather ctlplane service info: ovn Gather ctlplane service info: rabbitmq Copying OVN NB database from ovsdbserver-nb-0 Defaulted container "ovsdbserver-nb" out of: ovsdbserver-nb, openstack-network-exporter Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) tar: Removing leading `/' from member names Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Copying OVN SB database from ovsdbserver-sb-0 Defaulted container "ovsdbserver-sb" out of: ovsdbserver-sb, openstack-network-exporter tar: Removing leading `/' from member names Gather ctlplane service info: keystone Gather ctlplane service info: glance Gather ctlplane service info: cinder Gather ctlplane service info: nova Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Gather ctlplane service info: placement Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Gather ctlplane service info: neutron Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init) Gather ctlplane service info: swift Gather ctlplane service info: barbican Gather ctlplane service info: ceilometer Extensions list not supported by Identity API --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/cliff/app.py", line 410, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python3.9/site-packages/osc_lib/command/command.py", line 39, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.9/site-packages/cliff/display.py", line 117, in run column_names, data = self.take_action(parsed_args) File "/usr/lib/python3.9/site-packages/observabilityclient/v1/cli.py", line 27, in take_action metrics = client.query.list(disable_rbac=parsed_args.disable_rbac) File "/usr/lib/python3.9/site-packages/observabilityclient/v1/python_api.py", line 27, in list metric_names = self.prom.label_values("__name__") File "/usr/lib/python3.9/site-packages/observabilityclient/prometheus_client.py", line 143, in label_values decoded = self._get(f"label/{label}/values") File "/usr/lib/python3.9/site-packages/observabilityclient/prometheus_client.py", line 79, in _get raise PrometheusAPIClientError(resp) observabilityclient.prometheus_client.PrometheusAPIClientError: During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib64/python3.9/logging/__init__.py", line 1083, in emit msg = self.format(record) File "/usr/lib64/python3.9/logging/__init__.py", line 927, in format return fmt.format(record) File "/usr/lib64/python3.9/logging/__init__.py", line 663, in format record.message = record.getMessage() File "/usr/lib64/python3.9/logging/__init__.py", line 365, in getMessage msg = str(self.msg) File "/usr/lib/python3.9/site-packages/observabilityclient/prometheus_client.py", line 30, in __str__ decoded = self.resp.json() File "/usr/lib/python3.9/site-packages/requests/models.py", line 891, in json return complexjson.loads( File "/usr/lib64/python3.9/site-packages/simplejson/__init__.py", line 525, in loads return _default_decoder.decode(s) File "/usr/lib64/python3.9/site-packages/simplejson/decoder.py", line 370, in decode obj, end = self.raw_decode(s) File "/usr/lib64/python3.9/site-packages/simplejson/decoder.py", line 400, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Call stack: File "/usr/bin/openstack", line 10, in sys.exit(main()) File "/usr/lib/python3.9/site-packages/openstackclient/shell.py", line 145, in main return OpenStackShell().run(argv) File "/usr/lib/python3.9/site-packages/osc_lib/shell.py", line 134, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python3.9/site-packages/cliff/app.py", line 278, in run result = self.run_subcommand(remainder) File "/usr/lib/python3.9/site-packages/osc_lib/shell.py", line 175, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/lib/python3.9/site-packages/cliff/app.py", line 422, in run_subcommand self.LOG.error(err) Unable to print the message and arguments - possible formatting error. Use the traceback above to help find the error. command terminated with exit code 1 tar: Removing leading `/' from member names The /must-gather/must-gather.tar.xz now can be attached to the support case.