LAST SEEN TYPE REASON OBJECT MESSAGE 71m Normal RequirementsUnknown clusterserviceversion/cluster-logging.v6.2.7 requirements not yet checked 71m Normal RequirementsNotMet clusterserviceversion/cluster-logging.v6.2.7 one or more requirements couldn't be found 71m Normal AllRequirementsMet clusterserviceversion/cluster-logging.v6.2.7 all requirements found, attempting install 71m Normal SuccessfulCreate replicaset/cluster-logging-operator-79cf69ddc8 Created pod: cluster-logging-operator-79cf69ddc8-n4gqd 71m Normal Scheduled pod/cluster-logging-operator-79cf69ddc8-n4gqd Successfully assigned openshift-logging/cluster-logging-operator-79cf69ddc8-n4gqd to crc 71m Normal ScalingReplicaSet deployment/cluster-logging-operator Scaled up replica set cluster-logging-operator-79cf69ddc8 to 1 71m Normal AddedInterface pod/cluster-logging-operator-79cf69ddc8-n4gqd Add eth0 [10.217.0.51/23] from ovn-kubernetes 71m Normal InstallSucceeded clusterserviceversion/cluster-logging.v6.2.7 waiting for install components to report healthy 71m Normal InstallWaiting clusterserviceversion/cluster-logging.v6.2.7 installing: waiting for deployment cluster-logging-operator to become ready: deployment "cluster-logging-operator" not available: Deployment does not have minimum availability. 71m Normal Pulling pod/cluster-logging-operator-79cf69ddc8-n4gqd Pulling image "registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:80cac88d8ff5b40036e5983f5dacfc08702afe9c7a66b48d1c88bcb149c285b3" 70m Normal Created pod/cluster-logging-operator-79cf69ddc8-n4gqd Created container cluster-logging-operator 70m Normal Started pod/cluster-logging-operator-79cf69ddc8-n4gqd Started container cluster-logging-operator 70m Normal Pulled pod/cluster-logging-operator-79cf69ddc8-n4gqd Successfully pulled image "registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:80cac88d8ff5b40036e5983f5dacfc08702afe9c7a66b48d1c88bcb149c285b3" in 7.972s (7.972s including waiting). Image size: 343173849 bytes. 70m Normal InstallSucceeded clusterserviceversion/cluster-logging.v6.2.7 install strategy completed with no errors 70m Normal SuccessfulCreate statefulset/logging-loki-compactor create Pod logging-loki-compactor-0 in StatefulSet logging-loki-compactor successful 70m Normal NoPods poddisruptionbudget/logging-loki-querier No matching pods found 70m Normal Provisioning persistentvolumeclaim/wal-logging-loki-ingester-0 External provisioner is provisioning volume for claim "openshift-logging/wal-logging-loki-ingester-0" 70m Normal WaitForFirstConsumer persistentvolumeclaim/wal-logging-loki-ingester-0 waiting for first consumer to be created before binding 70m Normal ProvisioningSucceeded persistentvolumeclaim/storage-logging-loki-ingester-0 Successfully provisioned volume pvc-23b80e46-11f2-4981-be58-ee3fb1d879db 70m Normal Provisioning persistentvolumeclaim/storage-logging-loki-ingester-0 External provisioner is provisioning volume for claim "openshift-logging/storage-logging-loki-ingester-0" 70m Normal ExternalProvisioning persistentvolumeclaim/storage-logging-loki-ingester-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 70m Normal WaitForFirstConsumer persistentvolumeclaim/storage-logging-loki-ingester-0 waiting for first consumer to be created before binding 70m Normal ProvisioningSucceeded persistentvolumeclaim/storage-logging-loki-index-gateway-0 Successfully provisioned volume pvc-963b82d8-6b99-4118-8e96-79d67cd54b1f 70m Normal ExternalProvisioning persistentvolumeclaim/storage-logging-loki-index-gateway-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 70m Normal Provisioning persistentvolumeclaim/storage-logging-loki-index-gateway-0 External provisioner is provisioning volume for claim "openshift-logging/storage-logging-loki-index-gateway-0" 70m Normal WaitForFirstConsumer persistentvolumeclaim/storage-logging-loki-index-gateway-0 waiting for first consumer to be created before binding 70m Warning FailedMount pod/logging-loki-gateway-8597d8df56-shvtm MountVolume.SetUp failed for volume "tls-secret" : secret "logging-loki-gateway-http" not found 70m Normal SuccessfulCreate replicaset/logging-loki-gateway-8597d8df56 Created pod: logging-loki-gateway-8597d8df56-shvtm 70m Normal SuccessfulCreate replicaset/logging-loki-gateway-8597d8df56 Created pod: logging-loki-gateway-8597d8df56-dkqb6 70m Normal ProvisioningSucceeded persistentvolumeclaim/storage-logging-loki-compactor-0 Successfully provisioned volume pvc-4c971192-3e56-4b4a-9539-ac14f1293968 70m Normal Provisioning persistentvolumeclaim/storage-logging-loki-compactor-0 External provisioner is provisioning volume for claim "openshift-logging/storage-logging-loki-compactor-0" 70m Normal ExternalProvisioning persistentvolumeclaim/storage-logging-loki-compactor-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 70m Normal ScalingReplicaSet deployment/logging-loki-query-frontend Scaled up replica set logging-loki-query-frontend-69d9546745 to 1 70m Normal SuccessfulCreate replicaset/logging-loki-query-frontend-69d9546745 Created pod: logging-loki-query-frontend-69d9546745-bs4dm 70m Normal ScalingReplicaSet deployment/logging-loki-gateway Scaled up replica set logging-loki-gateway-8597d8df56 to 2 70m Normal SuccessfulCreate statefulset/logging-loki-index-gateway create Claim storage-logging-loki-index-gateway-0 Pod logging-loki-index-gateway-0 in StatefulSet logging-loki-index-gateway success 70m Normal SuccessfulCreate statefulset/logging-loki-compactor create Claim storage-logging-loki-compactor-0 Pod logging-loki-compactor-0 in StatefulSet logging-loki-compactor success 70m Normal Scheduled pod/logging-loki-gateway-8597d8df56-shvtm Successfully assigned openshift-logging/logging-loki-gateway-8597d8df56-shvtm to crc 70m Normal Scheduled pod/logging-loki-distributor-5f678c8dd6-zr5dw Successfully assigned openshift-logging/logging-loki-distributor-5f678c8dd6-zr5dw to crc 70m Normal SuccessfulCreate statefulset/logging-loki-index-gateway create Pod logging-loki-index-gateway-0 in StatefulSet logging-loki-index-gateway successful 70m Normal SuccessfulCreate statefulset/logging-loki-ingester create Claim storage-logging-loki-ingester-0 Pod logging-loki-ingester-0 in StatefulSet logging-loki-ingester success 70m Normal Scheduled pod/logging-loki-query-frontend-69d9546745-bs4dm Successfully assigned openshift-logging/logging-loki-query-frontend-69d9546745-bs4dm to crc 70m Normal ProvisioningSucceeded persistentvolumeclaim/wal-logging-loki-ingester-0 Successfully provisioned volume pvc-aaff93a3-7bb3-4ac2-99b5-1ccd9aa8fe4d 70m Normal ExternalProvisioning persistentvolumeclaim/wal-logging-loki-ingester-0 Waiting for a volume to be created either by the external provisioner 'kubevirt.io.hostpath-provisioner' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. 70m Normal ScalingReplicaSet deployment/logging-loki-querier Scaled up replica set logging-loki-querier-76788598db to 1 70m Normal SuccessfulCreate replicaset/logging-loki-querier-76788598db Created pod: logging-loki-querier-76788598db-q4xmw 70m Normal SuccessfulCreate statefulset/logging-loki-ingester create Claim wal-logging-loki-ingester-0 Pod logging-loki-ingester-0 in StatefulSet logging-loki-ingester success 70m Normal NoPods poddisruptionbudget/logging-loki-ingester No matching pods found 70m Normal Scheduled pod/logging-loki-querier-76788598db-q4xmw Successfully assigned openshift-logging/logging-loki-querier-76788598db-q4xmw to crc 70m Normal SuccessfulCreate statefulset/logging-loki-ingester create Pod logging-loki-ingester-0 in StatefulSet logging-loki-ingester successful 70m Normal SuccessfulCreate replicaset/logging-loki-distributor-5f678c8dd6 Created pod: logging-loki-distributor-5f678c8dd6-zr5dw 70m Normal ScalingReplicaSet deployment/logging-loki-distributor Scaled up replica set logging-loki-distributor-5f678c8dd6 to 1 70m Normal Scheduled pod/logging-loki-gateway-8597d8df56-dkqb6 Successfully assigned openshift-logging/logging-loki-gateway-8597d8df56-dkqb6 to crc 70m Warning FailedMount pod/logging-loki-gateway-8597d8df56-dkqb6 MountVolume.SetUp failed for volume "tls-secret" : secret "logging-loki-gateway-http" not found 70m Normal AddedInterface pod/logging-loki-distributor-5f678c8dd6-zr5dw Add eth0 [10.217.0.53/23] from ovn-kubernetes 70m Normal Pulling pod/logging-loki-query-frontend-69d9546745-bs4dm Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 70m Normal AddedInterface pod/logging-loki-gateway-8597d8df56-dkqb6 Add eth0 [10.217.0.58/23] from ovn-kubernetes 70m Normal Pulling pod/logging-loki-querier-76788598db-q4xmw Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 70m Normal Pulling pod/logging-loki-distributor-5f678c8dd6-zr5dw Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 70m Normal AddedInterface pod/logging-loki-query-frontend-69d9546745-bs4dm Add eth0 [10.217.0.55/23] from ovn-kubernetes 70m Normal AddedInterface pod/logging-loki-gateway-8597d8df56-shvtm Add eth0 [10.217.0.57/23] from ovn-kubernetes 70m Normal Scheduled pod/logging-loki-ingester-0 Successfully assigned openshift-logging/logging-loki-ingester-0 to crc 70m Normal AddedInterface pod/logging-loki-querier-76788598db-q4xmw Add eth0 [10.217.0.54/23] from ovn-kubernetes 70m Normal Scheduled pod/logging-loki-compactor-0 Successfully assigned openshift-logging/logging-loki-compactor-0 to crc 70m Normal Pulling pod/logging-loki-gateway-8597d8df56-shvtm Pulling image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" 70m Normal Scheduled pod/logging-loki-index-gateway-0 Successfully assigned openshift-logging/logging-loki-index-gateway-0 to crc 70m Normal Pulling pod/logging-loki-index-gateway-0 Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 70m Normal Pulling pod/logging-loki-gateway-8597d8df56-dkqb6 Pulling image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" 70m Normal Pulling pod/logging-loki-compactor-0 Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 70m Normal AddedInterface pod/logging-loki-compactor-0 Add eth0 [10.217.0.76/23] from ovn-kubernetes 70m Normal AddedInterface pod/logging-loki-index-gateway-0 Add eth0 [10.217.0.81/23] from ovn-kubernetes 70m Normal AddedInterface pod/logging-loki-ingester-0 Add eth0 [10.217.0.60/23] from ovn-kubernetes 70m Normal Pulling pod/logging-loki-ingester-0 Pulling image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" 70m Normal Created pod/logging-loki-distributor-5f678c8dd6-zr5dw Created container loki-distributor 70m Normal Started pod/logging-loki-querier-76788598db-q4xmw Started container loki-querier 70m Normal Started pod/logging-loki-query-frontend-69d9546745-bs4dm Started container loki-query-frontend 70m Normal Created pod/logging-loki-query-frontend-69d9546745-bs4dm Created container loki-query-frontend 70m Normal Pulled pod/logging-loki-gateway-8597d8df56-shvtm Successfully pulled image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" in 3.708s (3.708s including waiting). Image size: 180263561 bytes. 70m Normal Created pod/logging-loki-gateway-8597d8df56-shvtm Created container gateway 70m Normal Started pod/logging-loki-gateway-8597d8df56-shvtm Started container gateway 70m Normal Pulling pod/logging-loki-gateway-8597d8df56-shvtm Pulling image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" 70m Normal Pulled pod/logging-loki-query-frontend-69d9546745-bs4dm Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 3.927s (3.927s including waiting). Image size: 225282830 bytes. 70m Normal Pulled pod/logging-loki-distributor-5f678c8dd6-zr5dw Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 4.297s (4.297s including waiting). Image size: 225282830 bytes. 70m Normal Started pod/logging-loki-distributor-5f678c8dd6-zr5dw Started container loki-distributor 70m Normal Started pod/logging-loki-compactor-0 Started container loki-compactor 70m Normal Created pod/logging-loki-querier-76788598db-q4xmw Created container loki-querier 70m Normal Pulled pod/logging-loki-querier-76788598db-q4xmw Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 4.125s (4.125s including waiting). Image size: 225282830 bytes. 70m Normal Created pod/logging-loki-compactor-0 Created container loki-compactor 70m Normal Pulled pod/logging-loki-compactor-0 Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 3.211s (3.211s including waiting). Image size: 225282830 bytes. 70m Normal Pulled pod/logging-loki-gateway-8597d8df56-dkqb6 Successfully pulled image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel9@sha256:574afd46f23637b1685ddb00b647bb71d2e2d527611182fd2826bb818c5eb198" in 3.338s (3.338s including waiting). Image size: 180263561 bytes. 70m Normal Created pod/logging-loki-gateway-8597d8df56-dkqb6 Created container gateway 70m Normal Started pod/logging-loki-gateway-8597d8df56-dkqb6 Started container gateway 70m Normal Started pod/logging-loki-ingester-0 Started container loki-ingester 70m Normal Created pod/logging-loki-ingester-0 Created container loki-ingester 70m Normal Pulled pod/logging-loki-ingester-0 Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 3.258s (3.258s including waiting). Image size: 225282830 bytes. 70m Normal Pulling pod/logging-loki-gateway-8597d8df56-dkqb6 Pulling image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" 70m Normal Started pod/logging-loki-index-gateway-0 Started container loki-index-gateway 70m Normal Created pod/logging-loki-index-gateway-0 Created container loki-index-gateway 70m Normal Pulled pod/logging-loki-index-gateway-0 Successfully pulled image "registry.redhat.io/openshift-logging/logging-loki-rhel9@sha256:8c073f8de8288bf63489a400d7f93e96562652832965d33e0515f48d46282459" in 2.977s (2.977s including waiting). Image size: 225282830 bytes. 70m Normal Pulled pod/logging-loki-gateway-8597d8df56-dkqb6 Successfully pulled image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" in 2.254s (2.254s including waiting). Image size: 160768505 bytes. 70m Normal Pulled pod/logging-loki-gateway-8597d8df56-shvtm Successfully pulled image "registry.redhat.io/openshift-logging/opa-openshift-rhel9@sha256:182dc5ab627d64b5e565a2edb170903ac381fc534b4366fc5ffae088ea0de5e5" in 2.182s (2.182s including waiting). Image size: 160768505 bytes. 70m Normal Started pod/logging-loki-gateway-8597d8df56-shvtm Started container opa 70m Normal Started pod/logging-loki-gateway-8597d8df56-dkqb6 Started container opa 70m Normal Created pod/logging-loki-gateway-8597d8df56-shvtm Created container opa 70m Normal Created pod/logging-loki-gateway-8597d8df56-dkqb6 Created container opa 70m Warning ProbeError pod/logging-loki-ingester-0 Readiness probe error: HTTP probe failed with statuscode: 503... 69m Warning Unhealthy pod/logging-loki-ingester-0 Readiness probe failed: HTTP probe failed with statuscode: 503 69m Warning ProbeError pod/logging-loki-ingester-0 Readiness probe error: HTTP probe failed with statuscode: 503... 69m Normal Scheduled pod/collector-wwn5l Successfully assigned openshift-logging/collector-wwn5l to crc 69m Normal SuccessfulCreate daemonset/collector Created pod: collector-wwn5l 69m Normal SuccessfulDelete daemonset/collector Deleted pod: collector-wwn5l 69m Warning FailedMount pod/collector-wwn5l MountVolume.SetUp failed for volume "collector-syslog-receiver" : secret "collector-syslog-receiver" not found 69m Warning FailedMount pod/collector-wwn5l MountVolume.SetUp failed for volume "metrics" : secret "collector-metrics" not found 69m Normal Scheduled pod/collector-dg4sw Successfully assigned openshift-logging/collector-dg4sw to crc 69m Normal SuccessfulCreate daemonset/collector Created pod: collector-dg4sw 69m Normal AddedInterface pod/collector-dg4sw Add eth0 [10.217.0.86/23] from ovn-kubernetes 69m Normal Pulling pod/collector-dg4sw Pulling image "registry.redhat.io/openshift-logging/vector-rhel9@sha256:fa2cfa2ed336ce105c8dea5bfe0825407e37ef296193ae162f515213fe43c8d5" 69m Normal Pulled pod/collector-dg4sw Successfully pulled image "registry.redhat.io/openshift-logging/vector-rhel9@sha256:fa2cfa2ed336ce105c8dea5bfe0825407e37ef296193ae162f515213fe43c8d5" in 7.202s (7.202s including waiting). Image size: 344557702 bytes. 69m Normal Started pod/collector-dg4sw Started container collector 69m Normal Created pod/collector-dg4sw Created container collector 63m Normal IPAllocated service/openstack-logging Assigned IP ["172.17.0.80"] 63m Warning deprecatedAnnotation service/openstack-logging Service uses deprecated annotation metallb.universe.tf/loadBalancerIPs 63m Normal nodeAssigned service/openstack-logging announcing from node "crc" with protocol "layer2" 63m Warning deprecatedAnnotation service/openstack-logging Service uses deprecated annotation metallb.universe.tf/allow-shared-ip 63m Warning deprecatedAnnotation service/openstack-logging Service uses deprecated annotation metallb.universe.tf/address-pool 5m47s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe failed: Get "https://10.217.0.57:8083/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m47s Warning ProbeError pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe error: Get "https://10.217.0.57:8083/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m42s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe failed: Get "https://10.217.0.57:8081/ready": context deadline exceeded 5m42s Warning ProbeError pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe error: Get "https://10.217.0.58:8083/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m42s Warning ProbeError pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe error: Get "https://10.217.0.57:8081/ready": context deadline exceeded... 5m42s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe failed: Get "https://10.217.0.58:8083/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m37s Warning ProbeError pod/logging-loki-distributor-5f678c8dd6-zr5dw Liveness probe error: Get "https://10.217.0.53:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m37s Warning Unhealthy pod/logging-loki-distributor-5f678c8dd6-zr5dw Liveness probe failed: Get "https://10.217.0.53:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m37s Warning Unhealthy pod/logging-loki-query-frontend-69d9546745-bs4dm Liveness probe failed: Get "https://10.217.0.55:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m37s Warning ProbeError pod/logging-loki-query-frontend-69d9546745-bs4dm Liveness probe error: Get "https://10.217.0.55:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m37s Warning ProbeError pod/logging-loki-querier-76788598db-q4xmw Liveness probe error: Get "https://10.217.0.54:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m37s Warning Unhealthy pod/logging-loki-querier-76788598db-q4xmw Liveness probe failed: Get "https://10.217.0.54:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m36s Warning ProbeError pod/logging-loki-gateway-8597d8df56-shvtm Liveness probe error: Get "https://10.217.0.57:8083/live": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m36s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-shvtm Liveness probe failed: Get "https://10.217.0.57:8083/live": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m36s Warning Unhealthy pod/logging-loki-compactor-0 Liveness probe failed: Get "https://10.217.0.76:3101/loki/api/v1/status/buildinfo": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m36s Warning ProbeError pod/logging-loki-compactor-0 Liveness probe error: Get "https://10.217.0.76:3101/loki/api/v1/status/buildinfo": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m36s Warning ProbeError pod/logging-loki-gateway-8597d8df56-dkqb6 Liveness probe error: Get "https://10.217.0.58:8083/live": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m36s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-dkqb6 Liveness probe failed: Get "https://10.217.0.58:8083/live": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m36s Warning Unhealthy pod/logging-loki-index-gateway-0 Liveness probe failed: Get "https://10.217.0.81:3101/loki/api/v1/status/buildinfo": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m36s Warning ProbeError pod/logging-loki-index-gateway-0 Liveness probe error: Get "https://10.217.0.81:3101/loki/api/v1/status/buildinfo": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m36s Warning ProbeError pod/logging-loki-ingester-0 Liveness probe error: Get "https://10.217.0.60:3101/loki/api/v1/status/buildinfo": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m36s Warning Unhealthy pod/logging-loki-ingester-0 Liveness probe failed: Get "https://10.217.0.60:3101/loki/api/v1/status/buildinfo": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m32s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe failed: Get "https://10.217.0.57:8081/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m28s Warning ProbeError pod/logging-loki-distributor-5f678c8dd6-zr5dw Readiness probe error: Get "https://10.217.0.53:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m28s Warning Unhealthy pod/logging-loki-distributor-5f678c8dd6-zr5dw Readiness probe failed: Get "https://10.217.0.53:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe error: Get "https://10.217.0.58:8083/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m27s Warning ProbeError pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe error: Get "https://10.217.0.58:8081/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m27s Warning Unhealthy pod/logging-loki-compactor-0 Readiness probe failed: Get "https://10.217.0.76:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning Unhealthy pod/logging-loki-query-frontend-69d9546745-bs4dm Readiness probe failed: Get "https://10.217.0.55:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-query-frontend-69d9546745-bs4dm Readiness probe error: Get "https://10.217.0.55:3101/loki/api/v1/status/buildinfo": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m27s Warning Unhealthy pod/logging-loki-querier-76788598db-q4xmw Readiness probe failed: Get "https://10.217.0.54:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-querier-76788598db-q4xmw Readiness probe error: Get "https://10.217.0.54:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m27s Warning ProbeError pod/logging-loki-distributor-5f678c8dd6-zr5dw Readiness probe error: Get "https://10.217.0.53:3101/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m27s Warning Unhealthy pod/logging-loki-distributor-5f678c8dd6-zr5dw Readiness probe failed: Get "https://10.217.0.53:3101/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m27s Warning Unhealthy pod/logging-loki-ingester-0 Readiness probe failed: Get "https://10.217.0.60:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-ingester-0 Readiness probe error: Get "https://10.217.0.60:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m27s Warning ProbeError pod/logging-loki-compactor-0 Readiness probe error: Get "https://10.217.0.76:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m27s Warning Unhealthy pod/logging-loki-index-gateway-0 Readiness probe failed: Get "https://10.217.0.81:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-index-gateway-0 Readiness probe error: Get "https://10.217.0.81:3101/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m27s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe failed: Get "https://10.217.0.58:8081/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe error: Get "https://10.217.0.57:8081/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)... 5m27s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe failed: Get "https://10.217.0.58:8083/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe failed: Get "https://10.217.0.57:8083/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m27s Warning ProbeError pod/logging-loki-gateway-8597d8df56-shvtm Readiness probe error: Get "https://10.217.0.57:8083/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)... 5m22s Warning Unhealthy pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe failed: Get "https://10.217.0.58:8083/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m22s Warning ProbeError pod/logging-loki-gateway-8597d8df56-dkqb6 Readiness probe error: Get "https://10.217.0.58:8083/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...