--- I0312 21:10:55.760428 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I0312 21:10:55.760976 11 controller.go:128] Shutting down kubernetes service endpoint reconciler I0312 21:10:55.761386 11 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0312 21:10:55.761499 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I0312 21:10:55.781063 11 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" W0312 21:10:55.781073 11 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I0312 21:10:55.781315 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished I0312 21:10:55.795119 11 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0312 21:10:55.795169 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I0312 21:10:55.795297 11 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I0312 21:10:55.795389 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I0312 21:10:55.795332 11 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I0312 21:10:55.844601 11 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I0312 21:10:55.844661 11 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I0312 21:10:55.844666 11 controller.go:157] Shutting down quota evaluator I0312 21:10:55.844687 11 controller.go:176] quota evaluator worker shutdown I0312 21:10:55.844707 11 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I0312 21:10:55.844753 11 controller.go:176] quota evaluator worker shutdown I0312 21:10:55.844760 11 controller.go:176] quota evaluator worker shutdown I0312 21:10:55.844773 11 controller.go:176] quota evaluator worker shutdown I0312 21:10:55.844779 11 controller.go:176] quota evaluator worker shutdown I0312 21:10:55.845339 11 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I0312 21:10:55.845433 11 controller.go:120] Shutting down OpenAPI V3 controller I0312 21:10:55.845456 11 customresource_discovery_controller.go:328] Shutting down DiscoveryController I0312 21:10:55.845480 11 autoregister_controller.go:168] Shutting down autoregister controller I0312 21:10:55.845497 11 crdregistration_controller.go:146] Shutting down crd-autoregister controller I0312 21:10:55.845515 11 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I0312 21:10:55.845534 11 crd_finalizer.go:281] Shutting down CRDFinalizer I0312 21:10:55.845550 11 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I0312 21:10:55.845565 11 establishing_controller.go:92] Shutting down EstablishingController I0312 21:10:55.845596 11 controller.go:170] Shutting down OpenAPI controller I0312 21:10:55.845616 11 naming_controller.go:305] Shutting down NamingConditionController I0312 21:10:55.845646 11 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I0312 21:10:55.845669 11 system_namespaces_controller.go:76] Shutting down system namespaces controller I0312 21:10:55.845688 11 gc_controller.go:91] Shutting down apiserver lease garbage collector I0312 21:10:55.845707 11 apf_controller.go:389] Shutting down API Priority and Fairness config worker I0312 21:10:55.845766 11 remote_available_controller.go:449] Shutting down RemoteAvailability controller I0312 21:10:55.845788 11 local_available_controller.go:172] Shutting down LocalAvailability controller I0312 21:10:55.845830 11 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I0312 21:10:55.845847 11 controller.go:132] Ending legacy_token_tracking_controller I0312 21:10:55.845859 11 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I0312 21:10:55.845861 11 controller.go:133] Shutting down legacy_token_tracking_controller I0312 21:10:55.845937 11 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I0312 21:10:55.845986 11 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0312 21:10:55.846009 11 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0312 21:10:55.846022 11 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I0312 21:10:55.846226 11 secure_serving.go:258] Stopped listening on [::]:6443 I0312 21:10:55.846382 11 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0312 21:10:55.846405 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I0312 21:10:55.846608 11 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0312 21:10:55.847011 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I0312 21:10:55.847091 11 controller.go:86] Shutting down OpenAPI V3 AggregationController I0312 21:10:55.847190 11 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" E0312 21:10:55.847192 11 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=12982&timeout=5m24s&timeoutSeconds=324&watch=true" auditID="8ec268b7-b1ab-41af-91bc-4f35585937a5" I0312 21:10:55.847189 11 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0312 21:10:55.847232 11 controller.go:84] Shutting down OpenAPI AggregationController I0312 21:10:55.847250 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0312 21:10:55.847264 11 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0312 21:10:55.847279 11 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0312 21:10:55.847614 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I0312 21:10:55.850699 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I0312 21:10:55.850735 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I0312 21:10:57.846693 11 genericapiserver.go:742] [graceful-termination] apiserver is exiting I0312 21:10:57.846746 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W0312 21:10:57.850137 11 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W0312 21:10:57.851110 11 cacher.go:171] Terminating all watchers from cacher persistentvolumes W0312 21:10:57.852199 11 cacher.go:171] Terminating all watchers from cacher secrets W0312 21:10:57.854402 11 cacher.go:171] Terminating all watchers from cacher endpoints W0312 21:10:57.860504 11 cacher.go:171] Terminating all watchers from cacher resourcequotas W0312 21:10:57.862416 11 cacher.go:171] Terminating all watchers from cacher podtemplates W0312 21:10:57.863235 11 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W0312 21:10:57.864793 11 cacher.go:171] Terminating all watchers from cacher namespaces W0312 21:10:57.873210 11 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W0312 21:10:57.874163 11 cacher.go:171] Terminating all watchers from cacher limitranges W0312 21:10:57.875029 11 cacher.go:171] Terminating all watchers from cacher serviceaccounts W0312 21:10:57.877365 11 cacher.go:171] Terminating all watchers from cacher pods W0312 21:10:57.878856 11 cacher.go:171] Terminating all watchers from cacher services W0312 21:10:57.882574 11 cacher.go:171] Terminating all watchers from cacher nodes W0312 21:10:57.884427 11 cacher.go:171] Terminating all watchers from cacher configmaps W0312 21:10:57.900025 11 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W0312 21:10:57.901519 11 cacher.go:171] Terminating all watchers from cacher jobs.batch W0312 21:10:57.903184 11 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W0312 21:10:57.905447 11 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W0312 21:10:57.906511 11 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W0312 21:10:57.908472 11 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W0312 21:10:57.909577 11 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W0312 21:10:57.914863 11 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W0312 21:10:57.916318 11 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W0312 21:10:57.917371 11 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W0312 21:10:57.918190 11 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W0312 21:10:57.919273 11 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W0312 21:10:57.921205 11 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W0312 21:10:57.924030 11 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W0312 21:10:57.927430 11 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W0312 21:10:57.930033 11 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W0312 21:10:57.930534 11 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W0312 21:10:57.931781 11 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W0312 21:10:57.932524 11 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W0312 21:10:57.933643 11 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W0312 21:10:57.934422 11 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W0312 21:10:57.935570 11 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W0312 21:10:57.936086 11 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W0312 21:10:57.936780 11 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W0312 21:10:57.937776 11 cacher.go:171] Terminating all watchers from cacher replicasets.apps W0312 21:10:57.938195 11 cacher.go:171] Terminating all watchers from cacher deployments.apps W0312 21:10:57.940570 11 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W0312 21:10:57.941967 11 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W0312 21:10:57.943317 11 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W0312 21:10:57.944285 11 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W0312 21:10:57.945025 11 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W0312 21:10:57.945485 11 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W0312 21:10:57.946635 11 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W0312 21:10:57.948215 11 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W0312 21:10:57.948668 11 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W0312 21:10:57.949395 11 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W0312 21:10:57.950213 11 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W0312 21:10:57.951409 11 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W0312 21:10:57.952271 11 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W0312 21:10:57.952670 11 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W0312 21:10:57.953221 11 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W0312 21:10:57.954141 11 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W0312 21:10:57.955096 11 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W0312 21:10:57.956355 11 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W0312 21:10:57.957787 11 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W0312 21:10:57.958758 11 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W0312 21:10:57.959632 11 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W0312 21:10:57.960183 11 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W0312 21:10:57.960517 11 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W0312 21:10:57.961034 11 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W0312 21:10:57.961385 11 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W0312 21:10:57.962595 11 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W0312 21:10:57.963261 11 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W0312 21:10:57.963702 11 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W0312 21:10:57.964355 11 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W0312 21:10:57.964973 11 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W0312 21:10:57.965703 11 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W0312 21:10:57.966470 11 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W0312 21:10:57.967254 11 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W0312 21:10:57.968639 11 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W0312 21:10:57.971833 11 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W0312 21:10:57.973203 11 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W0312 21:10:57.973883 11 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W0312 21:10:57.977002 11 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W0312 21:10:57.977347 11 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W0312 21:10:57.977833 11 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W0312 21:10:57.978462 11 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W0312 21:10:57.979004 11 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W0312 21:10:57.979569 11 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W0312 21:10:57.980606 11 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W0312 21:10:57.981103 11 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W0312 21:10:57.982358 11 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W0312 21:10:57.982996 11 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W0312 21:10:57.983755 11 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W0312 21:10:57.984287 11 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W0312 21:10:57.985087 11 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W0312 21:10:57.985505 11 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W0312 21:10:57.986363 11 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W0312 21:10:57.987052 11 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W0312 21:10:57.987538 11 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W0312 21:10:57.988053 11 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W0312 21:10:57.988623 11 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W0312 21:10:57.990310 11 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W0312 21:10:57.990660 11 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W0312 21:10:57.991048 11 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W0312 21:10:57.991987 11 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W0312 21:10:57.993047 11 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W0312 21:10:57.993579 11 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W0312 21:10:57.994129 11 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W0312 21:10:57.994986 11 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W0312 21:10:57.995500 11 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W0312 21:10:57.995892 11 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W0312 21:10:57.996273 11 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W0312 21:10:57.997485 11 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W0312 21:10:57.998566 11 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W0312 21:10:57.999290 11 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W0312 21:10:57.999751 11 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W0312 21:10:58.000572 11 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W0312 21:10:58.001123 11 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W0312 21:10:58.001567 11 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W0312 21:10:58.002007 11 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W0312 21:10:58.002340 11 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W0312 21:10:58.003043 11 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W0312 21:10:58.003584 11 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W0312 21:10:58.004112 11 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W0312 21:10:58.004967 11 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W0312 21:10:58.005466 11 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W0312 21:10:58.005869 11 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W0312 21:10:58.006414 11 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W0312 21:10:58.006775 11 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W0312 21:10:58.007386 11 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W0312 21:10:58.007927 11 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W0312 21:10:58.009673 11 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W0312 21:10:58.010025 11 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W0312 21:10:58.010396 11 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W0312 21:10:58.010924 11 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W0312 21:10:58.011494 11 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W0312 21:10:58.011890 11 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W0312 21:10:58.012459 11 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W0312 21:10:58.013514 11 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W0312 21:10:58.013929 11 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W0312 21:10:58.014579 11 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0312 21:10:58.015108 11 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0312 21:10:58.015567 11 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W0312 21:10:58.016079 11 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W0312 21:10:58.016373 11 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W0312 21:10:58.016861 11 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W0312 21:10:58.017202 11 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W0312 21:10:58.018186 11 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W0312 21:10:58.018784 11 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W0312 21:10:58.019089 11 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W0312 21:10:58.020079 11 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W0312 21:10:58.020428 11 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W0312 21:10:58.020798 11 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W0312 21:10:58.021552 11 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W0312 21:10:58.021976 11 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W0312 21:10:58.022326 11 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W0312 21:10:58.022881 11 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W0312 21:10:58.023390 11 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W0312 21:10:58.023687 11 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W0312 21:10:58.024126 11 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W0312 21:10:58.024573 11 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W0312 21:10:58.025144 11 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W0312 21:10:58.025620 11 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W0312 21:10:58.026248 11 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W0312 21:10:58.026982 11 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W0312 21:10:58.027615 11 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W0312 21:10:58.028023 11 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W0312 21:10:58.028501 11 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W0312 21:10:58.029001 11 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W0312 21:10:58.030127 11 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W0312 21:10:58.031565 11 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W0312 21:10:58.031854 11 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W0312 21:10:58.032437 11 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W0312 21:10:58.033185 11 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W0312 21:10:58.033556 11 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W0312 21:10:58.034056 11 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W0312 21:10:58.034469 11 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W0312 21:10:58.034865 11 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W0312 21:10:58.036424 11 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W0312 21:10:58.036771 11 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W0312 21:10:58.037253 11 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W0312 21:10:58.037825 11 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W0312 21:10:58.038371 11 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W0312 21:10:58.038954 11 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W0312 21:10:58.039313 11 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W0312 21:10:58.040368 11 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W0312 21:10:58.040924 11 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io I0312 21:10:58.077059 1 main.go:235] Termination finished with exit code 0 I0312 21:10:58.077393 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" --- I0312 21:13:47.469073 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I0312 21:13:47.473358 11 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0312 21:13:47.473375 11 controller.go:128] Shutting down kubernetes service endpoint reconciler I0312 21:13:47.473426 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I0312 21:13:47.482978 11 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I0312 21:13:47.483033 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W0312 21:13:47.483891 11 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I0312 21:13:47.500720 11 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0312 21:13:47.500761 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I0312 21:13:47.500911 11 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I0312 21:13:47.500938 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I0312 21:13:47.500974 11 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I0312 21:13:47.541575 11 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I0312 21:13:47.541621 11 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I0312 21:13:47.541693 11 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I0312 21:13:47.541745 11 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0312 21:13:47.541763 11 controller.go:157] Shutting down quota evaluator I0312 21:13:47.541775 11 controller.go:176] quota evaluator worker shutdown I0312 21:13:47.542317 11 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0312 21:13:47.542345 11 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I0312 21:13:47.542366 11 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0312 21:13:47.542380 11 controller.go:84] Shutting down OpenAPI AggregationController I0312 21:13:47.542393 11 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0312 21:13:47.542407 11 controller.go:86] Shutting down OpenAPI V3 AggregationController I0312 21:13:47.542421 11 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I0312 21:13:47.542433 11 controller.go:176] quota evaluator worker shutdown I0312 21:13:47.542441 11 controller.go:176] quota evaluator worker shutdown I0312 21:13:47.542448 11 controller.go:176] quota evaluator worker shutdown I0312 21:13:47.542458 11 controller.go:176] quota evaluator worker shutdown I0312 21:13:47.542945 11 secure_serving.go:258] Stopped listening on [::]:6443 I0312 21:13:47.542971 11 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0312 21:13:47.542987 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I0312 21:13:47.543234 11 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0312 21:13:47.543576 11 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I0312 21:13:47.543635 11 controller.go:120] Shutting down OpenAPI V3 controller I0312 21:13:47.543658 11 customresource_discovery_controller.go:328] Shutting down DiscoveryController I0312 21:13:47.543683 11 autoregister_controller.go:168] Shutting down autoregister controller I0312 21:13:47.543702 11 crdregistration_controller.go:146] Shutting down crd-autoregister controller I0312 21:13:47.543722 11 establishing_controller.go:92] Shutting down EstablishingController I0312 21:13:47.543741 11 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I0312 21:13:47.543761 11 naming_controller.go:305] Shutting down NamingConditionController I0312 21:13:47.543781 11 controller.go:170] Shutting down OpenAPI controller I0312 21:13:47.543800 11 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I0312 21:13:47.543836 11 crd_finalizer.go:281] Shutting down CRDFinalizer I0312 21:13:47.543870 11 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I0312 21:13:47.543901 11 controller.go:132] Ending legacy_token_tracking_controller I0312 21:13:47.543908 11 controller.go:133] Shutting down legacy_token_tracking_controller I0312 21:13:47.543930 11 local_available_controller.go:172] Shutting down LocalAvailability controller I0312 21:13:47.543947 11 gc_controller.go:91] Shutting down apiserver lease garbage collector I0312 21:13:47.543956 11 remote_available_controller.go:449] Shutting down RemoteAvailability controller I0312 21:13:47.543969 11 system_namespaces_controller.go:76] Shutting down system namespaces controller I0312 21:13:47.543978 11 apf_controller.go:389] Shutting down API Priority and Fairness config worker E0312 21:13:47.544884 11 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=16062&timeout=6m43s&timeoutSeconds=403&watch=true" auditID="391ea2b3-e033-4b6d-b39e-3517b8790ce8" I0312 21:13:47.545022 11 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I0312 21:13:47.545087 11 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I0312 21:13:47.545666 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I0312 21:13:47.545836 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0312 21:13:47.545861 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I0312 21:13:47.545882 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I0312 21:13:47.545907 11 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0312 21:13:47.545923 11 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I0312 21:13:47.545944 11 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0312 21:13:49.548907 11 genericapiserver.go:742] [graceful-termination] apiserver is exiting I0312 21:13:49.548966 11 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W0312 21:13:49.560252 11 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W0312 21:13:49.562415 11 cacher.go:171] Terminating all watchers from cacher endpoints W0312 21:13:49.564360 11 cacher.go:171] Terminating all watchers from cacher persistentvolumes W0312 21:13:49.577702 11 cacher.go:171] Terminating all watchers from cacher secrets W0312 21:13:49.582535 11 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W0312 21:13:49.586344 11 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W0312 21:13:49.586689 11 cacher.go:171] Terminating all watchers from cacher nodes W0312 21:13:49.587211 11 cacher.go:171] Terminating all watchers from cacher resourcequotas W0312 21:13:49.587629 11 cacher.go:171] Terminating all watchers from cacher podtemplates W0312 21:13:49.587975 11 cacher.go:171] Terminating all watchers from cacher limitranges W0312 21:13:49.588343 11 cacher.go:171] Terminating all watchers from cacher serviceaccounts W0312 21:13:49.589637 11 cacher.go:171] Terminating all watchers from cacher configmaps W0312 21:13:49.596072 11 cacher.go:171] Terminating all watchers from cacher services W0312 21:13:49.614110 11 cacher.go:171] Terminating all watchers from cacher namespaces W0312 21:13:49.616440 11 cacher.go:171] Terminating all watchers from cacher pods W0312 21:13:49.628883 11 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W0312 21:13:49.629909 11 cacher.go:171] Terminating all watchers from cacher jobs.batch W0312 21:13:49.631453 11 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W0312 21:13:49.632155 11 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W0312 21:13:49.633111 11 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W0312 21:13:49.634014 11 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W0312 21:13:49.634715 11 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W0312 21:13:49.635134 11 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W0312 21:13:49.635929 11 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W0312 21:13:49.636470 11 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W0312 21:13:49.637085 11 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W0312 21:13:49.637676 11 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W0312 21:13:49.638616 11 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W0312 21:13:49.639132 11 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W0312 21:13:49.639896 11 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W0312 21:13:49.643816 11 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W0312 21:13:49.644337 11 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W0312 21:13:49.645148 11 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W0312 21:13:49.645729 11 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W0312 21:13:49.646576 11 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W0312 21:13:49.647740 11 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W0312 21:13:49.648952 11 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W0312 21:13:49.649415 11 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W0312 21:13:49.649700 11 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W0312 21:13:49.650105 11 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W0312 21:13:49.650983 11 cacher.go:171] Terminating all watchers from cacher deployments.apps W0312 21:13:49.655109 11 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W0312 21:13:49.655460 11 cacher.go:171] Terminating all watchers from cacher replicasets.apps W0312 21:13:49.656267 11 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W0312 21:13:49.656917 11 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W0312 21:13:49.657363 11 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W0312 21:13:49.657786 11 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W0312 21:13:49.663099 11 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W0312 21:13:49.664200 11 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W0312 21:13:49.664868 11 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W0312 21:13:49.665433 11 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W0312 21:13:49.667119 11 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W0312 21:13:49.667586 11 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W0312 21:13:49.668644 11 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W0312 21:13:49.670226 11 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W0312 21:13:49.670711 11 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W0312 21:13:49.671617 11 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W0312 21:13:49.672297 11 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W0312 21:13:49.672703 11 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W0312 21:13:49.673136 11 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W0312 21:13:49.673700 11 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W0312 21:13:49.674199 11 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W0312 21:13:49.675207 11 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W0312 21:13:49.675987 11 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W0312 21:13:49.676384 11 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W0312 21:13:49.676679 11 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W0312 21:13:49.678523 11 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W0312 21:13:49.679323 11 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W0312 21:13:49.679645 11 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W0312 21:13:49.680127 11 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W0312 21:13:49.680352 11 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W0312 21:13:49.680714 11 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W0312 21:13:49.681858 11 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W0312 21:13:49.682282 11 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W0312 21:13:49.682632 11 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W0312 21:13:49.683101 11 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W0312 21:13:49.683892 11 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W0312 21:13:49.684340 11 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W0312 21:13:49.685059 11 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W0312 21:13:49.685477 11 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W0312 21:13:49.686204 11 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io W0312 21:13:49.687000 11 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W0312 21:13:49.687581 11 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W0312 21:13:49.688365 11 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W0312 21:13:49.689045 11 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W0312 21:13:49.689460 11 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W0312 21:13:49.690145 11 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W0312 21:13:49.690464 11 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W0312 21:13:49.690897 11 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W0312 21:13:49.691628 11 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W0312 21:13:49.691976 11 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0312 21:13:49.692453 11 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0312 21:13:49.693009 11 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W0312 21:13:49.693279 11 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W0312 21:13:49.693710 11 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W0312 21:13:49.694193 11 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W0312 21:13:49.694997 11 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W0312 21:13:49.695424 11 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W0312 21:13:49.695746 11 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W0312 21:13:49.696427 11 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W0312 21:13:49.696660 11 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W0312 21:13:49.697412 11 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W0312 21:13:49.697799 11 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W0312 21:13:49.698096 11 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W0312 21:13:49.698454 11 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W0312 21:13:49.698936 11 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W0312 21:13:49.699445 11 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W0312 21:13:49.700469 11 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W0312 21:13:49.700976 11 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W0312 21:13:49.701786 11 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W0312 21:13:49.702225 11 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W0312 21:13:49.703190 11 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W0312 21:13:49.703712 11 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W0312 21:13:49.704267 11 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W0312 21:13:49.704589 11 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W0312 21:13:49.705186 11 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W0312 21:13:49.705461 11 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W0312 21:13:49.705859 11 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W0312 21:13:49.706491 11 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W0312 21:13:49.707191 11 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W0312 21:13:49.708830 11 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W0312 21:13:49.709559 11 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W0312 21:13:49.710141 11 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W0312 21:13:49.710654 11 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W0312 21:13:49.711265 11 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W0312 21:13:49.711832 11 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W0312 21:13:49.712191 11 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W0312 21:13:49.712648 11 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W0312 21:13:49.713275 11 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W0312 21:13:49.713706 11 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W0312 21:13:49.714251 11 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W0312 21:13:49.715598 11 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W0312 21:13:49.716135 11 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W0312 21:13:49.716474 11 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W0312 21:13:49.717171 11 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W0312 21:13:49.717886 11 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W0312 21:13:49.718668 11 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W0312 21:13:49.719413 11 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W0312 21:13:49.720179 11 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W0312 21:13:49.720956 11 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W0312 21:13:49.721254 11 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W0312 21:13:49.721779 11 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W0312 21:13:49.722545 11 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W0312 21:13:49.723004 11 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W0312 21:13:49.723609 11 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W0312 21:13:49.724299 11 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W0312 21:13:49.724594 11 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W0312 21:13:49.725182 11 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W0312 21:13:49.725771 11 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W0312 21:13:49.726309 11 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W0312 21:13:49.726705 11 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W0312 21:13:49.727358 11 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W0312 21:13:49.727997 11 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W0312 21:13:49.728463 11 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W0312 21:13:49.728834 11 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W0312 21:13:49.729119 11 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W0312 21:13:49.729558 11 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W0312 21:13:49.730084 11 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W0312 21:13:49.730544 11 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W0312 21:13:49.730982 11 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W0312 21:13:49.731346 11 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W0312 21:13:49.731753 11 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W0312 21:13:49.732215 11 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W0312 21:13:49.732936 11 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W0312 21:13:49.733392 11 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W0312 21:13:49.734378 11 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W0312 21:13:49.734667 11 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W0312 21:13:49.735141 11 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W0312 21:13:49.736106 11 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W0312 21:13:49.741767 11 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W0312 21:13:49.742436 11 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W0312 21:13:49.743085 11 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W0312 21:13:49.744471 11 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W0312 21:13:49.745405 11 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W0312 21:13:49.745941 11 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W0312 21:13:49.746465 11 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W0312 21:13:49.747195 11 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W0312 21:13:49.748524 11 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W0312 21:13:49.749822 11 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W0312 21:13:49.750325 11 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W0312 21:13:49.750619 11 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W0312 21:13:49.751275 11 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W0312 21:13:49.753280 11 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io I0312 21:13:49.790960 1 main.go:235] Termination finished with exit code 0 I0312 21:13:49.791049 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" ---