--- I0318 09:06:04.831943 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I0318 09:06:04.833339 13 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0318 09:06:04.833382 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I0318 09:06:04.833674 13 controller.go:128] Shutting down kubernetes service endpoint reconciler I0318 09:06:04.857096 13 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I0318 09:06:04.857136 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W0318 09:06:04.859450 13 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I0318 09:06:04.873938 13 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0318 09:06:04.873970 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I0318 09:06:04.874177 13 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I0318 09:06:04.874192 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I0318 09:06:04.874319 13 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I0318 09:06:04.917382 13 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I0318 09:06:04.917422 13 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I0318 09:06:04.917484 13 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I0318 09:06:04.917537 13 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0318 09:06:04.917661 13 controller.go:157] Shutting down quota evaluator I0318 09:06:04.917678 13 controller.go:176] quota evaluator worker shutdown I0318 09:06:04.918156 13 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I0318 09:06:04.918200 13 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0318 09:06:04.918214 13 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0318 09:06:04.918233 13 controller.go:86] Shutting down OpenAPI V3 AggregationController I0318 09:06:04.918246 13 controller.go:84] Shutting down OpenAPI AggregationController I0318 09:06:04.918287 13 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0318 09:06:04.918333 13 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I0318 09:06:04.918381 13 secure_serving.go:258] Stopped listening on [::]:6443 I0318 09:06:04.918394 13 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0318 09:06:04.918408 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I0318 09:06:04.918419 13 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0318 09:06:04.918449 13 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0318 09:06:04.918576 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I0318 09:06:04.918650 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I0318 09:06:04.918666 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I0318 09:06:04.918680 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0318 09:06:04.918695 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I0318 09:06:04.918708 13 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0318 09:06:04.920188 13 controller.go:132] Ending legacy_token_tracking_controller I0318 09:06:04.920213 13 controller.go:133] Shutting down legacy_token_tracking_controller I0318 09:06:04.920236 13 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I0318 09:06:04.920267 13 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I0318 09:06:04.920273 13 controller.go:120] Shutting down OpenAPI V3 controller I0318 09:06:04.920295 13 customresource_discovery_controller.go:328] Shutting down DiscoveryController I0318 09:06:04.920311 13 controller.go:170] Shutting down OpenAPI controller I0318 09:06:04.920326 13 autoregister_controller.go:168] Shutting down autoregister controller I0318 09:06:04.920341 13 crdregistration_controller.go:146] Shutting down crd-autoregister controller I0318 09:06:04.920354 13 naming_controller.go:305] Shutting down NamingConditionController I0318 09:06:04.920369 13 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I0318 09:06:04.920386 13 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I0318 09:06:04.920400 13 establishing_controller.go:92] Shutting down EstablishingController I0318 09:06:04.920413 13 crd_finalizer.go:281] Shutting down CRDFinalizer I0318 09:06:04.920431 13 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I0318 09:06:04.920450 13 remote_available_controller.go:449] Shutting down RemoteAvailability controller I0318 09:06:04.920247 13 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I0318 09:06:04.923862 13 controller.go:176] quota evaluator worker shutdown I0318 09:06:04.923915 13 gc_controller.go:91] Shutting down apiserver lease garbage collector I0318 09:06:04.923924 13 system_namespaces_controller.go:76] Shutting down system namespaces controller I0318 09:06:04.923937 13 apf_controller.go:389] Shutting down API Priority and Fairness config worker I0318 09:06:04.923958 13 local_available_controller.go:172] Shutting down LocalAvailability controller I0318 09:06:04.923987 13 controller.go:176] quota evaluator worker shutdown I0318 09:06:04.924171 13 controller.go:176] quota evaluator worker shutdown I0318 09:06:04.924184 13 controller.go:176] quota evaluator worker shutdown E0318 09:06:04.925674 13 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=12252&timeout=8m14s&timeoutSeconds=494&watch=true" auditID="e5824cf1-877f-49df-a7b5-da3a9536569b" I0318 09:06:06.918773 13 genericapiserver.go:742] [graceful-termination] apiserver is exiting I0318 09:06:06.918837 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W0318 09:06:06.925958 13 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W0318 09:06:06.926787 13 cacher.go:171] Terminating all watchers from cacher nodes W0318 09:06:06.927686 13 cacher.go:171] Terminating all watchers from cacher podtemplates W0318 09:06:06.928432 13 cacher.go:171] Terminating all watchers from cacher secrets W0318 09:06:06.931787 13 cacher.go:171] Terminating all watchers from cacher serviceaccounts W0318 09:06:06.938955 13 cacher.go:171] Terminating all watchers from cacher configmaps W0318 09:06:06.957744 13 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W0318 09:06:06.958538 13 cacher.go:171] Terminating all watchers from cacher services W0318 09:06:06.964693 13 cacher.go:171] Terminating all watchers from cacher resourcequotas W0318 09:06:06.965542 13 cacher.go:171] Terminating all watchers from cacher namespaces W0318 09:06:06.968190 13 cacher.go:171] Terminating all watchers from cacher pods W0318 09:06:06.969513 13 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W0318 09:06:06.970862 13 cacher.go:171] Terminating all watchers from cacher persistentvolumes W0318 09:06:06.973336 13 cacher.go:171] Terminating all watchers from cacher endpoints W0318 09:06:06.974649 13 cacher.go:171] Terminating all watchers from cacher limitranges W0318 09:06:06.975279 13 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W0318 09:06:06.976473 13 cacher.go:171] Terminating all watchers from cacher jobs.batch W0318 09:06:06.977754 13 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W0318 09:06:06.978242 13 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W0318 09:06:06.979612 13 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W0318 09:06:06.980350 13 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W0318 09:06:06.982310 13 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W0318 09:06:06.982627 13 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W0318 09:06:06.983135 13 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W0318 09:06:06.984215 13 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W0318 09:06:06.984830 13 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W0318 09:06:06.986025 13 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W0318 09:06:06.986873 13 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W0318 09:06:06.988322 13 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W0318 09:06:06.989196 13 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W0318 09:06:06.990223 13 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W0318 09:06:06.991064 13 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W0318 09:06:06.992000 13 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W0318 09:06:06.992744 13 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W0318 09:06:06.993958 13 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W0318 09:06:06.994484 13 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W0318 09:06:06.995594 13 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W0318 09:06:06.995936 13 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W0318 09:06:06.996219 13 cacher.go:171] Terminating all watchers from cacher deployments.apps W0318 09:06:06.997391 13 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W0318 09:06:06.998332 13 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W0318 09:06:06.999820 13 cacher.go:171] Terminating all watchers from cacher replicasets.apps W0318 09:06:07.000205 13 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W0318 09:06:07.001250 13 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W0318 09:06:07.001993 13 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W0318 09:06:07.003253 13 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W0318 09:06:07.003745 13 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W0318 09:06:07.004692 13 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W0318 09:06:07.005903 13 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W0318 09:06:07.006389 13 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W0318 09:06:07.007836 13 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W0318 09:06:07.008715 13 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W0318 09:06:07.009529 13 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W0318 09:06:07.010084 13 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W0318 09:06:07.011432 13 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W0318 09:06:07.012424 13 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W0318 09:06:07.013008 13 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W0318 09:06:07.014391 13 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W0318 09:06:07.014926 13 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W0318 09:06:07.015493 13 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W0318 09:06:07.018078 13 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W0318 09:06:07.018625 13 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W0318 09:06:07.019250 13 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W0318 09:06:07.019869 13 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W0318 09:06:07.020946 13 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W0318 09:06:07.021362 13 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W0318 09:06:07.021941 13 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W0318 09:06:07.022835 13 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W0318 09:06:07.023776 13 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W0318 09:06:07.024558 13 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W0318 09:06:07.025799 13 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W0318 09:06:07.026189 13 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W0318 09:06:07.026822 13 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W0318 09:06:07.027471 13 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W0318 09:06:07.028371 13 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W0318 09:06:07.028804 13 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W0318 09:06:07.029154 13 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W0318 09:06:07.029907 13 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W0318 09:06:07.030753 13 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W0318 09:06:07.031386 13 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W0318 09:06:07.031822 13 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W0318 09:06:07.033888 13 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W0318 09:06:07.034479 13 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W0318 09:06:07.035048 13 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W0318 09:06:07.035745 13 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W0318 09:06:07.036064 13 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W0318 09:06:07.036528 13 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W0318 09:06:07.037109 13 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W0318 09:06:07.037530 13 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W0318 09:06:07.038021 13 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W0318 09:06:07.038830 13 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W0318 09:06:07.039928 13 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W0318 09:06:07.041867 13 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W0318 09:06:07.042525 13 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W0318 09:06:07.043067 13 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W0318 09:06:07.043778 13 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W0318 09:06:07.044256 13 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W0318 09:06:07.045045 13 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W0318 09:06:07.045511 13 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W0318 09:06:07.046090 13 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W0318 09:06:07.048770 13 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W0318 09:06:07.049224 13 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W0318 09:06:07.051199 13 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W0318 09:06:07.051879 13 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W0318 09:06:07.052208 13 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W0318 09:06:07.052432 13 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W0318 09:06:07.052737 13 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W0318 09:06:07.053520 13 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W0318 09:06:07.053988 13 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W0318 09:06:07.054787 13 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W0318 09:06:07.055409 13 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W0318 09:06:07.055763 13 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W0318 09:06:07.055994 13 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W0318 09:06:07.056543 13 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W0318 09:06:07.057363 13 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W0318 09:06:07.057864 13 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0318 09:06:07.059014 13 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0318 09:06:07.059483 13 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W0318 09:06:07.059899 13 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W0318 09:06:07.060255 13 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W0318 09:06:07.060891 13 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W0318 09:06:07.061454 13 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W0318 09:06:07.062045 13 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W0318 09:06:07.062584 13 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W0318 09:06:07.063085 13 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W0318 09:06:07.063497 13 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W0318 09:06:07.064499 13 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W0318 09:06:07.065000 13 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W0318 09:06:07.065321 13 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W0318 09:06:07.065762 13 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W0318 09:06:07.066325 13 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W0318 09:06:07.067010 13 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W0318 09:06:07.067505 13 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W0318 09:06:07.068215 13 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W0318 09:06:07.068766 13 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W0318 09:06:07.069334 13 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W0318 09:06:07.071300 13 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W0318 09:06:07.072483 13 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W0318 09:06:07.073133 13 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W0318 09:06:07.073675 13 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W0318 09:06:07.075544 13 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W0318 09:06:07.076167 13 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W0318 09:06:07.077032 13 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W0318 09:06:07.077423 13 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W0318 09:06:07.077929 13 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W0318 09:06:07.078498 13 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W0318 09:06:07.079436 13 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W0318 09:06:07.080118 13 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W0318 09:06:07.080686 13 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W0318 09:06:07.081080 13 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W0318 09:06:07.081471 13 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W0318 09:06:07.082010 13 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W0318 09:06:07.082300 13 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W0318 09:06:07.082651 13 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W0318 09:06:07.082998 13 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W0318 09:06:07.083711 13 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W0318 09:06:07.084129 13 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W0318 09:06:07.084600 13 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W0318 09:06:07.085157 13 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W0318 09:06:07.085588 13 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W0318 09:06:07.085988 13 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W0318 09:06:07.086485 13 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W0318 09:06:07.086961 13 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W0318 09:06:07.087635 13 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W0318 09:06:07.088027 13 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W0318 09:06:07.088390 13 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W0318 09:06:07.089023 13 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W0318 09:06:07.089854 13 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W0318 09:06:07.090361 13 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W0318 09:06:07.090821 13 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W0318 09:06:07.091627 13 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W0318 09:06:07.091916 13 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W0318 09:06:07.092453 13 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W0318 09:06:07.093095 13 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W0318 09:06:07.093394 13 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W0318 09:06:07.093818 13 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W0318 09:06:07.094382 13 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W0318 09:06:07.095148 13 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W0318 09:06:07.095554 13 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io W0318 09:06:07.096091 13 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W0318 09:06:07.096319 13 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W0318 09:06:07.096920 13 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W0318 09:06:07.097515 13 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io I0318 09:06:07.132522 1 main.go:235] Termination finished with exit code 0 I0318 09:06:07.132580 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" --- I0318 09:09:04.077315 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I0318 09:09:04.078117 10 controller.go:128] Shutting down kubernetes service endpoint reconciler I0318 09:09:04.078122 10 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0318 09:09:04.078342 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I0318 09:09:04.089417 10 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I0318 09:09:04.089461 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W0318 09:09:04.094829 10 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I0318 09:09:04.109938 10 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0318 09:09:04.109968 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I0318 09:09:04.110121 10 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I0318 09:09:04.110133 10 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I0318 09:09:04.110146 10 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I0318 09:09:04.110220 10 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I0318 09:09:04.110171 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I0318 09:09:04.110607 10 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I0318 09:09:04.110673 10 controller.go:157] Shutting down quota evaluator I0318 09:09:04.110698 10 controller.go:176] quota evaluator worker shutdown I0318 09:09:04.110793 10 local_available_controller.go:172] Shutting down LocalAvailability controller I0318 09:09:04.111327 10 crd_finalizer.go:281] Shutting down CRDFinalizer I0318 09:09:04.111334 10 controller.go:132] Ending legacy_token_tracking_controller I0318 09:09:04.111358 10 controller.go:133] Shutting down legacy_token_tracking_controller I0318 09:09:04.111387 10 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I0318 09:09:04.111409 10 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0318 09:09:04.111469 10 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0318 09:09:04.111490 10 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0318 09:09:04.111586 10 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0318 09:09:04.112314 10 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I0318 09:09:04.112427 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I0318 09:09:04.112461 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I0318 09:09:04.112480 10 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0318 09:09:04.112491 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0318 09:09:04.112506 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I0318 09:09:04.112521 10 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0318 09:09:04.112535 10 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0318 09:09:04.111416 10 system_namespaces_controller.go:76] Shutting down system namespaces controller I0318 09:09:04.111436 10 remote_available_controller.go:449] Shutting down RemoteAvailability controller I0318 09:09:04.111453 10 gc_controller.go:91] Shutting down apiserver lease garbage collector I0318 09:09:04.111469 10 apf_controller.go:389] Shutting down API Priority and Fairness config worker I0318 09:09:04.111518 10 controller.go:84] Shutting down OpenAPI AggregationController I0318 09:09:04.111550 10 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I0318 09:09:04.111602 10 controller.go:86] Shutting down OpenAPI V3 AggregationController I0318 09:09:04.111608 10 controller.go:176] quota evaluator worker shutdown I0318 09:09:04.111611 10 controller.go:176] quota evaluator worker shutdown I0318 09:09:04.111614 10 controller.go:176] quota evaluator worker shutdown I0318 09:09:04.111617 10 controller.go:176] quota evaluator worker shutdown I0318 09:09:04.111663 10 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I0318 09:09:04.111685 10 controller.go:120] Shutting down OpenAPI V3 controller I0318 09:09:04.111696 10 customresource_discovery_controller.go:328] Shutting down DiscoveryController I0318 09:09:04.111704 10 autoregister_controller.go:168] Shutting down autoregister controller I0318 09:09:04.111714 10 controller.go:170] Shutting down OpenAPI controller I0318 09:09:04.112482 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I0318 09:09:04.111723 10 crdregistration_controller.go:146] Shutting down crd-autoregister controller I0318 09:09:04.111731 10 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I0318 09:09:04.111739 10 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I0318 09:09:04.111748 10 establishing_controller.go:92] Shutting down EstablishingController I0318 09:09:04.111755 10 naming_controller.go:305] Shutting down NamingConditionController I0318 09:09:04.112252 10 secure_serving.go:258] Stopped listening on [::]:6443 I0318 09:09:04.113234 10 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0318 09:09:04.113265 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I0318 09:09:04.112441 10 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I0318 09:09:04.112468 10 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller E0318 09:09:04.115406 10 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=16254&timeout=5m20s&timeoutSeconds=320&watch=true" auditID="ecc87184-e9e8-4c92-bf7c-fc702ca90750" I0318 09:09:06.113045 10 genericapiserver.go:742] [graceful-termination] apiserver is exiting I0318 09:09:06.113119 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W0318 09:09:06.120854 10 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W0318 09:09:06.122182 10 cacher.go:171] Terminating all watchers from cacher nodes W0318 09:09:06.124107 10 cacher.go:171] Terminating all watchers from cacher secrets W0318 09:09:06.128732 10 cacher.go:171] Terminating all watchers from cacher resourcequotas W0318 09:09:06.129331 10 cacher.go:171] Terminating all watchers from cacher persistentvolumes W0318 09:09:06.129747 10 cacher.go:171] Terminating all watchers from cacher serviceaccounts W0318 09:09:06.131199 10 cacher.go:171] Terminating all watchers from cacher pods W0318 09:09:06.131657 10 cacher.go:171] Terminating all watchers from cacher endpoints W0318 09:09:06.132611 10 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W0318 09:09:06.133234 10 cacher.go:171] Terminating all watchers from cacher configmaps W0318 09:09:06.153019 10 cacher.go:171] Terminating all watchers from cacher podtemplates W0318 09:09:06.153911 10 cacher.go:171] Terminating all watchers from cacher namespaces W0318 09:09:06.154742 10 cacher.go:171] Terminating all watchers from cacher limitranges W0318 09:09:06.155733 10 cacher.go:171] Terminating all watchers from cacher services W0318 09:09:06.157873 10 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W0318 09:09:06.161858 10 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W0318 09:09:06.166282 10 cacher.go:171] Terminating all watchers from cacher jobs.batch W0318 09:09:06.167117 10 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W0318 09:09:06.167596 10 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W0318 09:09:06.168241 10 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W0318 09:09:06.168725 10 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W0318 09:09:06.169751 10 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W0318 09:09:06.170340 10 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W0318 09:09:06.170909 10 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W0318 09:09:06.171413 10 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W0318 09:09:06.172042 10 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W0318 09:09:06.173495 10 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W0318 09:09:06.175366 10 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W0318 09:09:06.176218 10 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W0318 09:09:06.177100 10 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W0318 09:09:06.185049 10 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W0318 09:09:06.185425 10 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W0318 09:09:06.186005 10 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W0318 09:09:06.186531 10 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W0318 09:09:06.190032 10 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W0318 09:09:06.190849 10 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W0318 09:09:06.191993 10 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W0318 09:09:06.196887 10 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W0318 09:09:06.197838 10 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W0318 09:09:06.198741 10 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W0318 09:09:06.200122 10 cacher.go:171] Terminating all watchers from cacher replicasets.apps W0318 09:09:06.201000 10 cacher.go:171] Terminating all watchers from cacher deployments.apps W0318 09:09:06.203785 10 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W0318 09:09:06.204416 10 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W0318 09:09:06.205210 10 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W0318 09:09:06.205922 10 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W0318 09:09:06.214038 10 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W0318 09:09:06.215218 10 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W0318 09:09:06.216855 10 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W0318 09:09:06.217698 10 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W0318 09:09:06.218239 10 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W0318 09:09:06.218885 10 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W0318 09:09:06.221955 10 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W0318 09:09:06.223140 10 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W0318 09:09:06.223726 10 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W0318 09:09:06.224623 10 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W0318 09:09:06.225180 10 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W0318 09:09:06.225979 10 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W0318 09:09:06.226476 10 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W0318 09:09:06.226860 10 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W0318 09:09:06.227192 10 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W0318 09:09:06.228260 10 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W0318 09:09:06.228676 10 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W0318 09:09:06.229007 10 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W0318 09:09:06.229623 10 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W0318 09:09:06.229994 10 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W0318 09:09:06.230492 10 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W0318 09:09:06.231457 10 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W0318 09:09:06.232336 10 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W0318 09:09:06.234541 10 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W0318 09:09:06.234925 10 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W0318 09:09:06.235680 10 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W0318 09:09:06.236727 10 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W0318 09:09:06.237420 10 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W0318 09:09:06.237992 10 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W0318 09:09:06.239068 10 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W0318 09:09:06.239398 10 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W0318 09:09:06.239914 10 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W0318 09:09:06.240996 10 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W0318 09:09:06.241264 10 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W0318 09:09:06.241808 10 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W0318 09:09:06.242046 10 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W0318 09:09:06.243287 10 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W0318 09:09:06.244151 10 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W0318 09:09:06.245954 10 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W0318 09:09:06.246653 10 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W0318 09:09:06.247766 10 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W0318 09:09:06.248158 10 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W0318 09:09:06.249591 10 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W0318 09:09:06.249933 10 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W0318 09:09:06.250293 10 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W0318 09:09:06.250753 10 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W0318 09:09:06.251217 10 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W0318 09:09:06.251746 10 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W0318 09:09:06.252067 10 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W0318 09:09:06.252370 10 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W0318 09:09:06.252750 10 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W0318 09:09:06.253141 10 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W0318 09:09:06.253670 10 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W0318 09:09:06.254230 10 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W0318 09:09:06.254579 10 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W0318 09:09:06.255545 10 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W0318 09:09:06.256080 10 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W0318 09:09:06.258415 10 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W0318 09:09:06.259169 10 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W0318 09:09:06.259779 10 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W0318 09:09:06.260267 10 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W0318 09:09:06.260959 10 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W0318 09:09:06.262391 10 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W0318 09:09:06.263527 10 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W0318 09:09:06.264174 10 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W0318 09:09:06.264501 10 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W0318 09:09:06.265162 10 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W0318 09:09:06.265661 10 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W0318 09:09:06.266023 10 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W0318 09:09:06.266439 10 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0318 09:09:06.266787 10 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0318 09:09:06.267118 10 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W0318 09:09:06.267618 10 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io W0318 09:09:06.268251 10 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W0318 09:09:06.268595 10 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W0318 09:09:06.269017 10 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W0318 09:09:06.269288 10 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W0318 09:09:06.269672 10 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W0318 09:09:06.270007 10 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W0318 09:09:06.270354 10 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W0318 09:09:06.270779 10 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W0318 09:09:06.271305 10 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W0318 09:09:06.271719 10 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W0318 09:09:06.272077 10 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W0318 09:09:06.272390 10 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W0318 09:09:06.272880 10 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W0318 09:09:06.273337 10 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W0318 09:09:06.273906 10 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W0318 09:09:06.274721 10 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W0318 09:09:06.275211 10 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W0318 09:09:06.275910 10 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W0318 09:09:06.276289 10 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W0318 09:09:06.276697 10 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W0318 09:09:06.279676 10 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W0318 09:09:06.280062 10 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W0318 09:09:06.280533 10 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W0318 09:09:06.281152 10 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W0318 09:09:06.281739 10 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W0318 09:09:06.282004 10 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W0318 09:09:06.283070 10 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W0318 09:09:06.283593 10 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W0318 09:09:06.283844 10 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W0318 09:09:06.284179 10 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W0318 09:09:06.284647 10 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W0318 09:09:06.285237 10 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W0318 09:09:06.286875 10 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W0318 09:09:06.287152 10 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W0318 09:09:06.287802 10 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W0318 09:09:06.288152 10 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W0318 09:09:06.288462 10 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W0318 09:09:06.288856 10 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W0318 09:09:06.289435 10 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W0318 09:09:06.289888 10 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W0318 09:09:06.290240 10 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W0318 09:09:06.290856 10 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W0318 09:09:06.291294 10 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W0318 09:09:06.291799 10 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W0318 09:09:06.292248 10 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W0318 09:09:06.292630 10 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W0318 09:09:06.292992 10 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W0318 09:09:06.293399 10 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W0318 09:09:06.293997 10 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W0318 09:09:06.294447 10 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W0318 09:09:06.295065 10 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W0318 09:09:06.295438 10 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W0318 09:09:06.295959 10 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W0318 09:09:06.296381 10 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W0318 09:09:06.296882 10 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W0318 09:09:06.297337 10 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W0318 09:09:06.297885 10 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W0318 09:09:06.298310 10 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W0318 09:09:06.298749 10 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W0318 09:09:06.299449 10 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W0318 09:09:06.300105 10 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W0318 09:09:06.300641 10 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W0318 09:09:06.301187 10 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W0318 09:09:06.301917 10 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io I0318 09:09:06.333332 1 main.go:235] Termination finished with exit code 0 I0318 09:09:06.333472 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" ---