LAST SEEN TYPE REASON OBJECT MESSAGE 115m Normal RequirementsUnknown clusterserviceversion/cluster-logging.v6.2.7 requirements not yet checked 115m Normal RequirementsNotMet clusterserviceversion/cluster-logging.v6.2.7 one or more requirements couldn't be found 115m Normal SuccessfulCreate replicaset/cluster-logging-operator-79cf69ddc8 Created pod: cluster-logging-operator-79cf69ddc8-mllkq 115m Normal Scheduled pod/cluster-logging-operator-79cf69ddc8-mllkq Successfully assigned openshift-logging/cluster-logging-operator-79cf69ddc8-mllkq to crc 115m Normal AllRequirementsMet clusterserviceversion/cluster-logging.v6.2.7 all requirements found, attempting install 115m Normal ScalingReplicaSet deployment/cluster-logging-operator Scaled up replica set cluster-logging-operator-79cf69ddc8 to 1 115m Normal InstallSucceeded clusterserviceversion/cluster-logging.v6.2.7 waiting for install components to report healthy 115m Normal InstallWaiting clusterserviceversion/cluster-logging.v6.2.7 installing: waiting for deployment cluster-logging-operator to become ready: deployment "cluster-logging-operator" not available: Deployment does not have minimum availability. 115m Normal AddedInterface pod/cluster-logging-operator-79cf69ddc8-mllkq Add eth0 [10.217.0.48/23] from ovn-kubernetes 114m Warning Failed pod/cluster-logging-operator-79cf69ddc8-mllkq Error: ErrImagePull 114m Warning Failed pod/cluster-logging-operator-79cf69ddc8-mllkq Failed to pull image "registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:80cac88d8ff5b40036e5983f5dacfc08702afe9c7a66b48d1c88bcb149c285b3": rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled 114m Warning Failed pod/cluster-logging-operator-79cf69ddc8-mllkq Error: ImagePullBackOff 114m Normal BackOff pod/cluster-logging-operator-79cf69ddc8-mllkq Back-off pulling image "registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:80cac88d8ff5b40036e5983f5dacfc08702afe9c7a66b48d1c88bcb149c285b3" 114m Normal Pulling pod/cluster-logging-operator-79cf69ddc8-mllkq Pulling image "registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:80cac88d8ff5b40036e5983f5dacfc08702afe9c7a66b48d1c88bcb149c285b3" 114m Normal Started pod/cluster-logging-operator-79cf69ddc8-mllkq Started container cluster-logging-operator 114m Normal Created pod/cluster-logging-operator-79cf69ddc8-mllkq Created container cluster-logging-operator 114m Normal Pulled pod/cluster-logging-operator-79cf69ddc8-mllkq Successfully pulled image "registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:80cac88d8ff5b40036e5983f5dacfc08702afe9c7a66b48d1c88bcb149c285b3" in 1.317s (1.317s including waiting). Image size: 343173849 bytes. 114m Normal InstallSucceeded clusterserviceversion/cluster-logging.v6.2.7 install strategy completed with no errors 114m Normal Provisioning persistentvolumeclaim/storage-logging-loki-compactor-0 External provisioner is provisioning volume for claim "openshift-logging/storage-logging-loki-compactor-0" 114m Normal SuccessfulCreate replicaset/logging-loki-query-frontend-69d9546745 Created pod: logging-loki-query-frontend-69d9546745-ds2pd 114m Normal Provisioning persistentvolumeclaim/wal-logging-loki-ingester-0 External provisioner is provisioning volume for claim "openshift-logging/wal-logging-loki-ingester-0" 114m Normal WaitForFirstConsumer persistentvolumeclaim/wal-logging-loki-ingester-0 waiting for first consumer to be created before binding 114m Normal ProvisioningSucceeded persistentvolumeclaim/storage-logging-loki-ingester-0 Successfully provisioned volume pvc-7543c543-6d8e-426c-b60c-873ef9c222b4 114m Normal Provisioning persistentvolumeclaim/storage-logging-loki-ingester-0 External provisioner is provisioning volume for claim "openshift-logging/storage-logging-loki-ingester-0" 114m Normal ExternalProvisioning persistentvolumeclaim/storage-logging-loki-ingester-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 114m Normal WaitForFirstConsumer persistentvolumeclaim/storage-logging-loki-ingester-0 waiting for first consumer to be created before binding 114m Normal ProvisioningSucceeded persistentvolumeclaim/storage-logging-loki-index-gateway-0 Successfully provisioned volume pvc-5ba318b7-8ed5-448b-aa19-2f3b9e593fb6 114m Normal Provisioning persistentvolumeclaim/storage-logging-loki-index-gateway-0 External provisioner is provisioning volume for claim "openshift-logging/storage-logging-loki-index-gateway-0" 114m Normal ExternalProvisioning persistentvolumeclaim/storage-logging-loki-index-gateway-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 114m Normal SuccessfulCreate statefulset/logging-loki-index-gateway create Pod logging-loki-index-gateway-0 in StatefulSet logging-loki-index-gateway successful 114m Normal SuccessfulCreate statefulset/logging-loki-index-gateway create Claim storage-logging-loki-index-gateway-0 Pod logging-loki-index-gateway-0 in StatefulSet logging-loki-index-gateway success 114m Normal SuccessfulCreate statefulset/logging-loki-ingester create Claim storage-logging-loki-ingester-0 Pod logging-loki-ingester-0 in StatefulSet logging-loki-ingester success 114m Normal NoPods poddisruptionbudget/logging-loki-ingester No matching pods found 114m Normal WaitForPodScheduled persistentvolumeclaim/storage-logging-loki-index-gateway-0 waiting for pod logging-loki-index-gateway-0 to be scheduled 114m Normal ProvisioningSucceeded persistentvolumeclaim/storage-logging-loki-compactor-0 Successfully provisioned volume pvc-e0645b0f-75d1-4a0f-b83e-1d09a112fc35 114m Normal SuccessfulCreate statefulset/logging-loki-compactor create Claim storage-logging-loki-compactor-0 Pod logging-loki-compactor-0 in StatefulSet logging-loki-compactor success 114m Normal SuccessfulCreate statefulset/logging-loki-compactor create Pod logging-loki-compactor-0 in StatefulSet logging-loki-compactor successful 114m Normal Scheduled pod/logging-loki-distributor-5f678c8dd6-wxx4s Successfully assigned openshift-logging/logging-loki-distributor-5f678c8dd6-wxx4s to crc 114m Normal SuccessfulCreate statefulset/logging-loki-ingester create Claim wal-logging-loki-ingester-0 Pod logging-loki-ingester-0 in StatefulSet logging-loki-ingester success 114m Normal SuccessfulCreate statefulset/logging-loki-ingester create Pod logging-loki-ingester-0 in StatefulSet logging-loki-ingester successful 114m Normal ScalingReplicaSet deployment/logging-loki-gateway Scaled up replica set logging-loki-gateway-785c7cc549 to 2 114m Normal ExternalProvisioning persistentvolumeclaim/storage-logging-loki-compactor-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 114m Normal SuccessfulCreate replicaset/logging-loki-gateway-785c7cc549 Created pod: logging-loki-gateway-785c7cc549-fc8ss 114m Normal WaitForPodScheduled persistentvolumeclaim/storage-logging-loki-compactor-0 waiting for pod logging-loki-compactor-0 to be scheduled 114m Normal NoPods poddisruptionbudget/logging-loki-query-frontend No matching pods found 114m Normal SuccessfulCreate replicaset/logging-loki-distributor-5f678c8dd6 Created pod: logging-loki-distributor-5f678c8dd6-wxx4s 114m Normal ScalingReplicaSet deployment/logging-loki-distributor Scaled up replica set logging-loki-distributor-5f678c8dd6 to 1 114m Normal Scheduled pod/logging-loki-gateway-785c7cc549-fc8ss Successfully assigned openshift-logging/logging-loki-gateway-785c7cc549-fc8ss to crc 114m Normal SuccessfulCreate replicaset/logging-loki-gateway-785c7cc549 Created pod: logging-loki-gateway-785c7cc549-thnm6 114m Normal Scheduled pod/logging-loki-querier-76788598db-lxv2v Successfully assigned openshift-logging/logging-loki-querier-76788598db-lxv2v to crc 114m Normal ProvisioningSucceeded persistentvolumeclaim/wal-logging-loki-ingester-0 Successfully provisioned volume pvc-3a4e7a1b-e72a-45a8-ae09-4d6263e9b29a 114m Normal ScalingReplicaSet deployment/logging-loki-query-frontend Scaled up replica set logging-loki-query-frontend-69d9546745 to 1 114m Normal ExternalProvisioning persistentvolumeclaim/wal-logging-loki-ingester-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 114m Normal SuccessfulCreate replicaset/logging-loki-querier-76788598db Created pod: logging-loki-querier-76788598db-lxv2v 114m Normal ScalingReplicaSet deployment/logging-loki-querier Scaled up replica set logging-loki-querier-76788598db to 1 114m Normal NoPods poddisruptionbudget/logging-loki-querier No matching pods found 114m Normal Scheduled pod/logging-loki-query-frontend-69d9546745-ds2pd Successfully assigned openshift-logging/logging-loki-query-frontend-69d9546745-ds2pd to crc 114m Normal Scheduled pod/logging-loki-gateway-785c7cc549-thnm6 Successfully assigned openshift-logging/logging-loki-gateway-785c7cc549-thnm6 to crc 114m Normal Scheduled pod/logging-loki-ingester-0 Successfully assigned openshift-logging/logging-loki-ingester-0 to crc 114m Normal Pulling pod/logging-loki-distributor-5f678c8dd6-wxx4s Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 114m Normal Scheduled pod/logging-loki-index-gateway-0 Successfully assigned openshift-logging/logging-loki-index-gateway-0 to crc 114m Normal Scheduled pod/logging-loki-compactor-0 Successfully assigned openshift-logging/logging-loki-compactor-0 to crc 114m Normal AddedInterface pod/logging-loki-distributor-5f678c8dd6-wxx4s Add eth0 [10.217.0.51/23] from ovn-kubernetes 114m Normal Pulling pod/logging-loki-query-frontend-69d9546745-ds2pd Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 114m Normal AddedInterface pod/logging-loki-query-frontend-69d9546745-ds2pd Add eth0 [10.217.0.53/23] from ovn-kubernetes 114m Normal Pulling pod/logging-loki-gateway-785c7cc549-thnm6 Pulling image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" 114m Normal Pulling pod/logging-loki-querier-76788598db-lxv2v Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 114m Normal AddedInterface pod/logging-loki-querier-76788598db-lxv2v Add eth0 [10.217.0.52/23] from ovn-kubernetes 114m Normal Pulling pod/logging-loki-gateway-785c7cc549-fc8ss Pulling image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" 114m Normal AddedInterface pod/logging-loki-gateway-785c7cc549-fc8ss Add eth0 [10.217.0.55/23] from ovn-kubernetes 114m Normal AddedInterface pod/logging-loki-gateway-785c7cc549-thnm6 Add eth0 [10.217.0.54/23] from ovn-kubernetes 114m Normal AddedInterface pod/logging-loki-compactor-0 Add eth0 [10.217.0.58/23] from ovn-kubernetes 114m Normal Pulling pod/logging-loki-compactor-0 Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 114m Normal AddedInterface pod/logging-loki-index-gateway-0 Add eth0 [10.217.0.61/23] from ovn-kubernetes 114m Normal Pulling pod/logging-loki-index-gateway-0 Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 114m Normal Pulling pod/logging-loki-ingester-0 Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 114m Normal AddedInterface pod/logging-loki-ingester-0 Add eth0 [10.217.0.57/23] from ovn-kubernetes 114m Normal Pulled pod/logging-loki-gateway-785c7cc549-fc8ss Successfully pulled image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" in 7.422s (7.422s including waiting). Image size: 180263561 bytes. 114m Normal Pulled pod/logging-loki-compactor-0 Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 6.849s (6.849s including waiting). Image size: 225282830 bytes. 114m Normal Pulled pod/logging-loki-index-gateway-0 Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 6.823s (6.823s including waiting). Image size: 225282830 bytes. 114m Normal Pulled pod/logging-loki-distributor-5f678c8dd6-wxx4s Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 8.115s (8.115s including waiting). Image size: 225282830 bytes. 114m Normal Pulled pod/logging-loki-ingester-0 Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 6.698s (6.698s including waiting). Image size: 225282830 bytes. 114m Normal Pulled pod/logging-loki-querier-76788598db-lxv2v Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 7.063s (7.063s including waiting). Image size: 225282830 bytes. 114m Normal Pulled pod/logging-loki-query-frontend-69d9546745-ds2pd Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 7.46s (7.46s including waiting). Image size: 225282830 bytes. 114m Normal Pulled pod/logging-loki-gateway-785c7cc549-thnm6 Successfully pulled image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" in 7.191s (7.191s including waiting). Image size: 180263561 bytes. 114m Normal Created pod/logging-loki-querier-76788598db-lxv2v Created container loki-querier 114m Normal Created pod/logging-loki-compactor-0 Created container loki-compactor 114m Normal Started pod/logging-loki-ingester-0 Started container loki-ingester 114m Normal Created pod/logging-loki-ingester-0 Created container loki-ingester 114m Normal Started pod/logging-loki-compactor-0 Started container loki-compactor 114m Normal Created pod/logging-loki-distributor-5f678c8dd6-wxx4s Created container loki-distributor 114m Normal Started pod/logging-loki-distributor-5f678c8dd6-wxx4s Started container loki-distributor 114m Normal Created pod/logging-loki-gateway-785c7cc549-fc8ss Created container gateway 114m Normal Started pod/logging-loki-index-gateway-0 Started container loki-index-gateway 114m Normal Created pod/logging-loki-index-gateway-0 Created container loki-index-gateway 114m Normal Started pod/logging-loki-gateway-785c7cc549-fc8ss Started container gateway 114m Normal Started pod/logging-loki-query-frontend-69d9546745-ds2pd Started container loki-query-frontend 114m Normal Created pod/logging-loki-query-frontend-69d9546745-ds2pd Created container loki-query-frontend 114m Normal Pulling pod/logging-loki-gateway-785c7cc549-fc8ss Pulling image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" 114m Normal Created pod/logging-loki-gateway-785c7cc549-thnm6 Created container gateway 114m Normal Started pod/logging-loki-querier-76788598db-lxv2v Started container loki-querier 114m Normal Pulling pod/logging-loki-gateway-785c7cc549-thnm6 Pulling image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" 114m Normal Started pod/logging-loki-gateway-785c7cc549-thnm6 Started container gateway 114m Normal Started pod/logging-loki-gateway-785c7cc549-fc8ss Started container opa 114m Normal Created pod/logging-loki-gateway-785c7cc549-fc8ss Created container opa 114m Normal Pulled pod/logging-loki-gateway-785c7cc549-fc8ss Successfully pulled image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" in 3.215s (3.215s including waiting). Image size: 160768505 bytes. 113m Normal Created pod/logging-loki-gateway-785c7cc549-thnm6 Created container opa 113m Normal Started pod/logging-loki-gateway-785c7cc549-thnm6 Started container opa 113m Normal Pulled pod/logging-loki-gateway-785c7cc549-thnm6 Successfully pulled image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" in 9.959s (9.959s including waiting). Image size: 160768505 bytes. 113m Warning ProbeError pod/logging-loki-ingester-0 Readiness probe error: HTTP probe failed with statuscode: 503... 113m Warning ProbeError pod/logging-loki-ingester-0 Readiness probe error: HTTP probe failed with statuscode: 503... 113m Warning Unhealthy pod/logging-loki-ingester-0 Readiness probe failed: HTTP probe failed with statuscode: 503 112m Normal Scheduled pod/collector-pqg96 Successfully assigned openshift-logging/collector-pqg96 to crc 112m Normal SuccessfulDelete daemonset/collector Deleted pod: collector-pqg96 112m Normal SuccessfulCreate daemonset/collector Created pod: collector-pqg96 112m Warning FailedMount pod/collector-pqg96 MountVolume.SetUp failed for volume "metrics" : secret "collector-metrics" not found 112m Normal Scheduled pod/collector-czl2w Successfully assigned openshift-logging/collector-czl2w to crc 112m Normal Pulling pod/collector-czl2w Pulling image "registry.redhat.io/openshift-logging/vector-rhel9@sha256:fa2cfa2ed336ce105c8dea5bfe0825407e37ef296193ae162f515213fe43c8d5" 112m Normal AddedInterface pod/collector-czl2w Add eth0 [10.217.0.83/23] from ovn-kubernetes 112m Normal SuccessfulCreate daemonset/collector Created pod: collector-czl2w 112m Normal Created pod/collector-czl2w Created container collector 112m Normal Pulled pod/collector-czl2w Successfully pulled image "registry.redhat.io/openshift-logging/vector-rhel9@sha256:fa2cfa2ed336ce105c8dea5bfe0825407e37ef296193ae162f515213fe43c8d5" in 9.884s (9.884s including waiting). Image size: 344557702 bytes. 112m Normal Started pod/collector-czl2w Started container collector 106m Normal IPAllocated service/openstack-logging Assigned IP ["172.17.0.80"] 101m Warning Unhealthy pod/logging-loki-index-gateway-0 Readiness probe failed: Get "https://10.217.0.61:3101/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 101m Warning ProbeError pod/logging-loki-index-gateway-0 Readiness probe error: Get "https://10.217.0.61:3101/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 56m Warning ProbeError pod/logging-loki-distributor-5f678c8dd6-wxx4s Readiness probe error: Get "https://10.217.0.51:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/logging-loki-distributor-5f678c8dd6-wxx4s Readiness probe failed: Get "https://10.217.0.51:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/logging-loki-gateway-785c7cc549-thnm6 Readiness probe failed: Get "https://10.217.0.54:8081/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/logging-loki-gateway-785c7cc549-thnm6 Readiness probe error: Get "https://10.217.0.54:8081/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 56m Normal nodeAssigned service/openstack-logging announcing from node "crc" with protocol "layer2" 56m Warning deprecatedAnnotation service/openstack-logging Service uses deprecated annotation metallb.universe.tf/loadBalancerIPs 56m Warning deprecatedAnnotation service/openstack-logging Service uses deprecated annotation metallb.universe.tf/allow-shared-ip 56m Warning deprecatedAnnotation service/openstack-logging Service uses deprecated annotation metallb.universe.tf/address-pool