--- I1204 12:03:49.411063 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I1204 12:03:49.413944 10 controller.go:128] Shutting down kubernetes service endpoint reconciler I1204 12:03:49.413947 10 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I1204 12:03:49.414098 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I1204 12:03:49.645885 10 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I1204 12:03:49.645929 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W1204 12:03:49.651730 10 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I1204 12:03:49.808271 10 store.go:1641] "Monitoring resource count at path" resource="dataimages.metal3.io" path="//metal3.io/dataimages" I1204 12:03:49.852890 10 healthz.go:278] shutdown check failed: readyz [-]shutdown failed: process is shutting down I1204 12:03:49.873382 10 cacher.go:463] cacher (dataimages.metal3.io): initialized I1204 12:03:49.873424 10 reflector.go:368] Caches populated for metal3.io/v1alpha1, Kind=DataImage from storage/cacher.go:/metal3.io/dataimages I1204 12:03:49.874552 10 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I1204 12:03:49.874588 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I1204 12:03:49.874922 10 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I1204 12:03:49.874946 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I1204 12:03:49.875151 10 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I1204 12:03:50.002050 10 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I1204 12:03:50.002107 10 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I1204 12:03:50.002134 10 controller.go:157] Shutting down quota evaluator I1204 12:03:50.002165 10 controller.go:176] quota evaluator worker shutdown I1204 12:03:50.002174 10 controller.go:84] Shutting down OpenAPI AggregationController I1204 12:03:50.002215 10 remote_available_controller.go:449] Shutting down RemoteAvailability controller I1204 12:03:50.002274 10 controller.go:132] Ending legacy_token_tracking_controller I1204 12:03:50.002302 10 controller.go:133] Shutting down legacy_token_tracking_controller I1204 12:03:50.002310 10 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I1204 12:03:50.002330 10 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1204 12:03:50.002352 10 controller.go:176] quota evaluator worker shutdown I1204 12:03:50.002360 10 controller.go:86] Shutting down OpenAPI V3 AggregationController I1204 12:03:50.002404 10 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I1204 12:03:50.002280 10 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1204 12:03:50.002433 10 controller.go:176] quota evaluator worker shutdown I1204 12:03:50.002199 10 local_available_controller.go:172] Shutting down LocalAvailability controller I1204 12:03:50.002449 10 controller.go:176] quota evaluator worker shutdown I1204 12:03:50.002156 10 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I1204 12:03:50.002383 10 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1204 12:03:50.002534 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I1204 12:03:50.002465 10 secure_serving.go:258] Stopped listening on [::]:6443 I1204 12:03:50.002568 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I1204 12:03:50.002590 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1204 12:03:50.002619 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I1204 12:03:50.002630 10 naming_controller.go:305] Shutting down NamingConditionController I1204 12:03:50.002641 10 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1204 12:03:50.002661 10 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1204 12:03:50.002678 10 customresource_discovery_controller.go:328] Shutting down DiscoveryController I1204 12:03:50.002681 10 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I1204 12:03:50.002701 10 autoregister_controller.go:168] Shutting down autoregister controller I1204 12:03:50.002727 10 crdregistration_controller.go:146] Shutting down crd-autoregister controller I1204 12:03:50.002514 10 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I1204 12:03:50.002676 10 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I1204 12:03:50.002850 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I1204 12:03:50.002500 10 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I1204 12:03:50.002383 10 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I1204 12:03:50.002701 10 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I1204 12:03:50.002683 10 controller.go:120] Shutting down OpenAPI V3 controller I1204 12:03:50.003585 10 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I1204 12:03:50.002443 10 controller.go:176] quota evaluator worker shutdown I1204 12:03:50.003663 10 crd_finalizer.go:281] Shutting down CRDFinalizer I1204 12:03:50.003651 10 controller.go:170] Shutting down OpenAPI controller I1204 12:03:50.003690 10 establishing_controller.go:92] Shutting down EstablishingController I1204 12:03:50.003710 10 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I1204 12:03:50.003734 10 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I1204 12:03:50.003739 10 system_namespaces_controller.go:76] Shutting down system namespaces controller I1204 12:03:50.003765 10 apf_controller.go:389] Shutting down API Priority and Fairness config worker I1204 12:03:50.003785 10 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I1204 12:03:50.003864 10 gc_controller.go:91] Shutting down apiserver lease garbage collector E1204 12:03:50.005928 10 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=11811&timeout=9m1s&timeoutSeconds=541&watch=true" auditID="7a2f760e-01cf-4c84-8fbd-14a56c0c5fa7" I1204 12:03:52.002686 10 genericapiserver.go:742] [graceful-termination] apiserver is exiting I1204 12:03:52.002755 10 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W1204 12:03:52.063208 10 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W1204 12:03:52.064148 10 cacher.go:171] Terminating all watchers from cacher nodes W1204 12:03:52.067040 10 cacher.go:171] Terminating all watchers from cacher pods W1204 12:03:52.070159 10 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W1204 12:03:52.070902 10 cacher.go:171] Terminating all watchers from cacher limitranges W1204 12:03:52.071399 10 cacher.go:171] Terminating all watchers from cacher persistentvolumes W1204 12:03:52.072015 10 cacher.go:171] Terminating all watchers from cacher namespaces W1204 12:03:52.076453 10 cacher.go:171] Terminating all watchers from cacher configmaps W1204 12:03:52.088001 10 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W1204 12:03:52.089148 10 cacher.go:171] Terminating all watchers from cacher resourcequotas W1204 12:03:52.089946 10 cacher.go:171] Terminating all watchers from cacher services W1204 12:03:52.095694 10 cacher.go:171] Terminating all watchers from cacher serviceaccounts W1204 12:03:52.096720 10 cacher.go:171] Terminating all watchers from cacher podtemplates W1204 12:03:52.097413 10 cacher.go:171] Terminating all watchers from cacher endpoints W1204 12:03:52.098885 10 cacher.go:171] Terminating all watchers from cacher secrets W1204 12:03:52.103773 10 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W1204 12:03:52.104869 10 cacher.go:171] Terminating all watchers from cacher jobs.batch W1204 12:03:52.105664 10 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W1204 12:03:52.106234 10 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W1204 12:03:52.108956 10 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W1204 12:03:52.110031 10 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W1204 12:03:52.110726 10 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W1204 12:03:52.111209 10 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W1204 12:03:52.112530 10 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W1204 12:03:52.113442 10 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W1204 12:03:52.114408 10 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W1204 12:03:52.116500 10 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W1204 12:03:52.120752 10 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W1204 12:03:52.122471 10 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W1204 12:03:52.123985 10 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W1204 12:03:52.125631 10 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W1204 12:03:52.126239 10 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W1204 12:03:52.126785 10 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W1204 12:03:52.127380 10 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W1204 12:03:52.127906 10 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W1204 12:03:52.129428 10 cacher.go:171] Terminating all watchers from cacher deployments.apps W1204 12:03:52.130556 10 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W1204 12:03:52.131287 10 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W1204 12:03:52.133243 10 cacher.go:171] Terminating all watchers from cacher replicasets.apps W1204 12:03:52.133707 10 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W1204 12:03:52.134284 10 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W1204 12:03:52.134878 10 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W1204 12:03:52.136692 10 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W1204 12:03:52.137442 10 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W1204 12:03:52.139046 10 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W1204 12:03:52.140071 10 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W1204 12:03:52.140684 10 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W1204 12:03:52.141105 10 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W1204 12:03:52.141865 10 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W1204 12:03:52.142735 10 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W1204 12:03:52.143613 10 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W1204 12:03:52.144229 10 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W1204 12:03:52.144596 10 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W1204 12:03:52.147097 10 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W1204 12:03:52.147551 10 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W1204 12:03:52.148093 10 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W1204 12:03:52.148691 10 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W1204 12:03:52.149235 10 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W1204 12:03:52.149756 10 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W1204 12:03:52.150346 10 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W1204 12:03:52.150971 10 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W1204 12:03:52.151589 10 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W1204 12:03:52.152292 10 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W1204 12:03:52.153267 10 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W1204 12:03:52.153775 10 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W1204 12:03:52.154407 10 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W1204 12:03:52.154965 10 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W1204 12:03:52.156092 10 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W1204 12:03:52.157360 10 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W1204 12:03:52.157951 10 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W1204 12:03:52.158398 10 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W1204 12:03:52.158947 10 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W1204 12:03:52.159765 10 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W1204 12:03:52.160231 10 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W1204 12:03:52.161502 10 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W1204 12:03:52.161916 10 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W1204 12:03:52.162844 10 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W1204 12:03:52.163707 10 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W1204 12:03:52.164306 10 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W1204 12:03:52.165128 10 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W1204 12:03:52.165654 10 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W1204 12:03:52.166385 10 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W1204 12:03:52.167741 10 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W1204 12:03:52.168188 10 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W1204 12:03:52.168573 10 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W1204 12:03:52.169066 10 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W1204 12:03:52.169880 10 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W1204 12:03:52.170229 10 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W1204 12:03:52.170718 10 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W1204 12:03:52.171418 10 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W1204 12:03:52.174453 10 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W1204 12:03:52.174898 10 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W1204 12:03:52.175633 10 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W1204 12:03:52.176051 10 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W1204 12:03:52.177221 10 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W1204 12:03:52.177691 10 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W1204 12:03:52.178721 10 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W1204 12:03:52.179268 10 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W1204 12:03:52.179905 10 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W1204 12:03:52.180527 10 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W1204 12:03:52.181119 10 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W1204 12:03:52.182121 10 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W1204 12:03:52.182581 10 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W1204 12:03:52.186383 10 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W1204 12:03:52.187076 10 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W1204 12:03:52.187479 10 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W1204 12:03:52.187835 10 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W1204 12:03:52.188326 10 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W1204 12:03:52.190595 10 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W1204 12:03:52.191071 10 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W1204 12:03:52.191470 10 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W1204 12:03:52.192583 10 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W1204 12:03:52.193233 10 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W1204 12:03:52.193742 10 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W1204 12:03:52.194508 10 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W1204 12:03:52.195492 10 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W1204 12:03:52.195997 10 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W1204 12:03:52.196556 10 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W1204 12:03:52.197164 10 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W1204 12:03:52.197690 10 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W1204 12:03:52.198161 10 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W1204 12:03:52.199052 10 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W1204 12:03:52.199575 10 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W1204 12:03:52.200155 10 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W1204 12:03:52.200756 10 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W1204 12:03:52.201223 10 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W1204 12:03:52.202078 10 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W1204 12:03:52.203063 10 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W1204 12:03:52.203570 10 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W1204 12:03:52.205019 10 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W1204 12:03:52.205350 10 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W1204 12:03:52.206050 10 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W1204 12:03:52.206743 10 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W1204 12:03:52.207222 10 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W1204 12:03:52.208172 10 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W1204 12:03:52.210792 10 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W1204 12:03:52.211244 10 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W1204 12:03:52.211922 10 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W1204 12:03:52.212579 10 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W1204 12:03:52.213306 10 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W1204 12:03:52.214304 10 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W1204 12:03:52.215090 10 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W1204 12:03:52.215553 10 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W1204 12:03:52.216003 10 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W1204 12:03:52.216724 10 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W1204 12:03:52.217858 10 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W1204 12:03:52.218603 10 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W1204 12:03:52.220078 10 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com I1204 12:03:52.245137 1 main.go:235] Termination finished with exit code 0 I1204 12:03:52.245403 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" --- I1204 12:07:26.129781 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I1204 12:07:26.130285 13 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I1204 12:07:26.130320 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I1204 12:07:26.130564 13 controller.go:128] Shutting down kubernetes service endpoint reconciler I1204 12:07:26.140168 13 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I1204 12:07:26.140213 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W1204 12:07:26.148960 13 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I1204 12:07:26.195514 13 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I1204 12:07:26.195556 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I1204 12:07:26.195705 13 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I1204 12:07:26.195746 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I1204 12:07:26.195712 13 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I1204 12:07:26.202406 13 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I1204 12:07:26.202483 13 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I1204 12:07:26.202593 13 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I1204 12:07:26.202649 13 controller.go:157] Shutting down quota evaluator I1204 12:07:26.202664 13 controller.go:176] quota evaluator worker shutdown I1204 12:07:26.203742 13 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I1204 12:07:26.203781 13 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I1204 12:07:26.204248 13 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I1204 12:07:26.204270 13 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I1204 12:07:26.204266 13 system_namespaces_controller.go:76] Shutting down system namespaces controller I1204 12:07:26.204293 13 apf_controller.go:389] Shutting down API Priority and Fairness config worker I1204 12:07:26.204308 13 controller.go:120] Shutting down OpenAPI V3 controller I1204 12:07:26.204317 13 remote_available_controller.go:449] Shutting down RemoteAvailability controller I1204 12:07:26.204328 13 customresource_discovery_controller.go:328] Shutting down DiscoveryController I1204 12:07:26.204336 13 local_available_controller.go:172] Shutting down LocalAvailability controller I1204 12:07:26.204351 13 controller.go:170] Shutting down OpenAPI controller I1204 12:07:26.204354 13 controller.go:132] Ending legacy_token_tracking_controller I1204 12:07:26.204360 13 controller.go:133] Shutting down legacy_token_tracking_controller I1204 12:07:26.204367 13 autoregister_controller.go:168] Shutting down autoregister controller I1204 12:07:26.204376 13 gc_controller.go:91] Shutting down apiserver lease garbage collector I1204 12:07:26.204384 13 crdregistration_controller.go:146] Shutting down crd-autoregister controller I1204 12:07:26.204401 13 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I1204 12:07:26.204416 13 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I1204 12:07:26.204431 13 establishing_controller.go:92] Shutting down EstablishingController I1204 12:07:26.204446 13 naming_controller.go:305] Shutting down NamingConditionController I1204 12:07:26.204461 13 crd_finalizer.go:281] Shutting down CRDFinalizer I1204 12:07:26.205279 13 controller.go:86] Shutting down OpenAPI V3 AggregationController I1204 12:07:26.205345 13 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I1204 12:07:26.205486 13 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1204 12:07:26.205506 13 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1204 12:07:26.205595 13 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1204 12:07:26.205632 13 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I1204 12:07:26.205718 13 controller.go:84] Shutting down OpenAPI AggregationController I1204 12:07:26.205753 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1204 12:07:26.205778 13 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I1204 12:07:26.205843 13 secure_serving.go:258] Stopped listening on [::]:6443 I1204 12:07:26.205882 13 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I1204 12:07:26.205905 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening E1204 12:07:26.206086 13 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=14631&timeout=9m10s&timeoutSeconds=550&watch=true" auditID="f6733d9c-b177-4714-a6e5-62ed2ed5bcb9" I1204 12:07:26.206218 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I1204 12:07:26.206245 13 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1204 12:07:26.206276 13 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1204 12:07:26.206485 13 controller.go:176] quota evaluator worker shutdown I1204 12:07:26.206505 13 controller.go:176] quota evaluator worker shutdown I1204 12:07:26.206519 13 controller.go:176] quota evaluator worker shutdown I1204 12:07:26.206528 13 controller.go:176] quota evaluator worker shutdown I1204 12:07:26.206656 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I1204 12:07:26.206677 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I1204 12:07:26.206656 13 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I1204 12:07:26.207838 13 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I1204 12:07:28.206208 13 genericapiserver.go:742] [graceful-termination] apiserver is exiting I1204 12:07:28.206268 13 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W1204 12:07:28.212378 13 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W1204 12:07:28.213298 13 cacher.go:171] Terminating all watchers from cacher limitranges W1204 12:07:28.214132 13 cacher.go:171] Terminating all watchers from cacher secrets W1204 12:07:28.218535 13 cacher.go:171] Terminating all watchers from cacher serviceaccounts W1204 12:07:28.220790 13 cacher.go:171] Terminating all watchers from cacher resourcequotas W1204 12:07:28.221366 13 cacher.go:171] Terminating all watchers from cacher namespaces W1204 12:07:28.224215 13 cacher.go:171] Terminating all watchers from cacher configmaps W1204 12:07:28.225863 13 cacher.go:171] Terminating all watchers from cacher podtemplates W1204 12:07:28.227104 13 cacher.go:171] Terminating all watchers from cacher nodes W1204 12:07:28.229701 13 cacher.go:171] Terminating all watchers from cacher pods W1204 12:07:28.250184 13 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W1204 12:07:28.252563 13 cacher.go:171] Terminating all watchers from cacher services W1204 12:07:28.255645 13 cacher.go:171] Terminating all watchers from cacher endpoints W1204 12:07:28.259018 13 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W1204 12:07:28.259632 13 cacher.go:171] Terminating all watchers from cacher persistentvolumes W1204 12:07:28.261091 13 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W1204 12:07:28.262151 13 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W1204 12:07:28.269498 13 cacher.go:171] Terminating all watchers from cacher jobs.batch W1204 12:07:28.270342 13 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W1204 12:07:28.271489 13 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W1204 12:07:28.273109 13 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W1204 12:07:28.273884 13 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W1204 12:07:28.274403 13 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W1204 12:07:28.275117 13 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W1204 12:07:28.276082 13 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W1204 12:07:28.276561 13 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W1204 12:07:28.277502 13 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W1204 12:07:28.279681 13 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W1204 12:07:28.281609 13 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W1204 12:07:28.284744 13 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W1204 12:07:28.286165 13 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W1204 12:07:28.286653 13 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W1204 12:07:28.287001 13 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W1204 12:07:28.287700 13 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W1204 12:07:28.288258 13 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W1204 12:07:28.288842 13 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W1204 12:07:28.290110 13 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W1204 12:07:28.290639 13 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W1204 12:07:28.291465 13 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W1204 12:07:28.291952 13 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W1204 12:07:28.293382 13 cacher.go:171] Terminating all watchers from cacher replicasets.apps W1204 12:07:28.294176 13 cacher.go:171] Terminating all watchers from cacher deployments.apps W1204 12:07:28.295792 13 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W1204 12:07:28.296765 13 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W1204 12:07:28.297601 13 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W1204 12:07:28.298678 13 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W1204 12:07:28.299488 13 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W1204 12:07:28.301143 13 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W1204 12:07:28.302267 13 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W1204 12:07:28.302656 13 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W1204 12:07:28.303368 13 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W1204 12:07:28.303768 13 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W1204 12:07:28.304444 13 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W1204 12:07:28.305162 13 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W1204 12:07:28.306284 13 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W1204 12:07:28.306919 13 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W1204 12:07:28.307575 13 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W1204 12:07:28.308121 13 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W1204 12:07:28.308947 13 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W1204 12:07:28.309589 13 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W1204 12:07:28.310148 13 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W1204 12:07:28.310846 13 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io W1204 12:07:28.312058 13 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W1204 12:07:28.312663 13 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W1204 12:07:28.314103 13 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W1204 12:07:28.314627 13 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W1204 12:07:28.317649 13 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W1204 12:07:28.318165 13 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W1204 12:07:28.318858 13 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W1204 12:07:28.319800 13 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W1204 12:07:28.320548 13 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W1204 12:07:28.321327 13 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W1204 12:07:28.321940 13 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W1204 12:07:28.322956 13 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W1204 12:07:28.323789 13 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W1204 12:07:28.325121 13 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W1204 12:07:28.325897 13 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W1204 12:07:28.329593 13 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W1204 12:07:28.330377 13 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W1204 12:07:28.331178 13 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W1204 12:07:28.331899 13 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W1204 12:07:28.332511 13 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W1204 12:07:28.333417 13 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W1204 12:07:28.334069 13 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W1204 12:07:28.334928 13 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W1204 12:07:28.335540 13 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W1204 12:07:28.336011 13 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W1204 12:07:28.336729 13 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W1204 12:07:28.337261 13 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W1204 12:07:28.338276 13 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W1204 12:07:28.338901 13 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W1204 12:07:28.339499 13 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W1204 12:07:28.340233 13 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W1204 12:07:28.340643 13 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W1204 12:07:28.341061 13 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W1204 12:07:28.342404 13 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W1204 12:07:28.342952 13 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W1204 12:07:28.343344 13 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W1204 12:07:28.343829 13 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W1204 12:07:28.344151 13 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W1204 12:07:28.344461 13 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W1204 12:07:28.345003 13 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W1204 12:07:28.346259 13 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W1204 12:07:28.346706 13 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W1204 12:07:28.347579 13 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W1204 12:07:28.348060 13 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W1204 12:07:28.348925 13 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W1204 12:07:28.350013 13 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W1204 12:07:28.350365 13 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W1204 12:07:28.351145 13 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W1204 12:07:28.351626 13 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W1204 12:07:28.352162 13 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W1204 12:07:28.353336 13 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W1204 12:07:28.353649 13 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W1204 12:07:28.354247 13 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W1204 12:07:28.354584 13 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W1204 12:07:28.355205 13 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W1204 12:07:28.355732 13 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W1204 12:07:28.356023 13 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W1204 12:07:28.356655 13 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W1204 12:07:28.357079 13 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W1204 12:07:28.358291 13 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W1204 12:07:28.358821 13 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W1204 12:07:28.359183 13 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W1204 12:07:28.359756 13 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W1204 12:07:28.360785 13 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W1204 12:07:28.362463 13 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W1204 12:07:28.363110 13 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W1204 12:07:28.364151 13 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W1204 12:07:28.365468 13 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W1204 12:07:28.366025 13 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W1204 12:07:28.366402 13 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W1204 12:07:28.366979 13 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W1204 12:07:28.367558 13 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W1204 12:07:28.368092 13 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W1204 12:07:28.369173 13 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W1204 12:07:28.369520 13 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W1204 12:07:28.370158 13 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W1204 12:07:28.370640 13 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W1204 12:07:28.371313 13 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W1204 12:07:28.372263 13 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W1204 12:07:28.372672 13 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W1204 12:07:28.373075 13 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W1204 12:07:28.373858 13 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W1204 12:07:28.374341 13 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W1204 12:07:28.375230 13 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W1204 12:07:28.376258 13 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W1204 12:07:28.376733 13 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W1204 12:07:28.377441 13 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W1204 12:07:28.377968 13 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W1204 12:07:28.378344 13 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W1204 12:07:28.379913 13 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W1204 12:07:28.380482 13 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W1204 12:07:28.381075 13 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W1204 12:07:28.381441 13 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W1204 12:07:28.382353 13 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W1204 12:07:28.382688 13 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W1204 12:07:28.383041 13 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W1204 12:07:28.383589 13 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W1204 12:07:28.384175 13 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W1204 12:07:28.384745 13 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W1204 12:07:28.385380 13 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W1204 12:07:28.386172 13 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W1204 12:07:28.388472 13 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W1204 12:07:28.389286 13 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W1204 12:07:28.390339 13 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W1204 12:07:28.390791 13 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W1204 12:07:28.391631 13 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W1204 12:07:28.392103 13 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W1204 12:07:28.392979 13 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W1204 12:07:28.393359 13 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W1204 12:07:28.393786 13 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W1204 12:07:28.394529 13 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W1204 12:07:28.395182 13 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W1204 12:07:28.395833 13 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W1204 12:07:28.396587 13 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W1204 12:07:28.397079 13 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W1204 12:07:28.397744 13 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W1204 12:07:28.398219 13 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W1204 12:07:28.398631 13 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W1204 12:07:28.398951 13 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W1204 12:07:28.399572 13 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W1204 12:07:28.400217 13 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io I1204 12:07:28.435510 1 main.go:235] Termination finished with exit code 0 I1204 12:07:28.435573 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" ---