--- I1203 20:11:44.408053 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I1203 20:11:44.408326 14 controller.go:128] Shutting down kubernetes service endpoint reconciler I1203 20:11:44.408333 14 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I1203 20:11:44.408541 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I1203 20:11:44.418972 14 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I1203 20:11:44.419038 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W1203 20:11:44.422701 14 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I1203 20:11:44.435143 14 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I1203 20:11:44.435178 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I1203 20:11:44.435387 14 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I1203 20:11:44.435402 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I1203 20:11:44.435527 14 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I1203 20:11:44.462352 14 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I1203 20:11:44.462395 14 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I1203 20:11:44.462401 14 controller.go:157] Shutting down quota evaluator I1203 20:11:44.462414 14 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I1203 20:11:44.462424 14 controller.go:176] quota evaluator worker shutdown I1203 20:11:44.462451 14 controller.go:176] quota evaluator worker shutdown I1203 20:11:44.462451 14 controller.go:176] quota evaluator worker shutdown I1203 20:11:44.462457 14 controller.go:176] quota evaluator worker shutdown I1203 20:11:44.462463 14 controller.go:176] quota evaluator worker shutdown I1203 20:11:44.462473 14 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I1203 20:11:44.462541 14 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1203 20:11:44.462586 14 controller.go:84] Shutting down OpenAPI AggregationController I1203 20:11:44.462612 14 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I1203 20:11:44.462618 14 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1203 20:11:44.462638 14 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I1203 20:11:44.462642 14 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I1203 20:11:44.462655 14 controller.go:86] Shutting down OpenAPI V3 AggregationController I1203 20:11:44.462665 14 controller.go:120] Shutting down OpenAPI V3 controller I1203 20:11:44.462680 14 customresource_discovery_controller.go:328] Shutting down DiscoveryController I1203 20:11:44.462694 14 controller.go:170] Shutting down OpenAPI controller I1203 20:11:44.462706 14 autoregister_controller.go:168] Shutting down autoregister controller I1203 20:11:44.462718 14 naming_controller.go:305] Shutting down NamingConditionController I1203 20:11:44.462729 14 crdregistration_controller.go:146] Shutting down crd-autoregister controller I1203 20:11:44.462739 14 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I1203 20:11:44.462749 14 establishing_controller.go:92] Shutting down EstablishingController I1203 20:11:44.462759 14 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I1203 20:11:44.462768 14 crd_finalizer.go:281] Shutting down CRDFinalizer I1203 20:11:44.462794 14 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I1203 20:11:44.462815 14 gc_controller.go:91] Shutting down apiserver lease garbage collector I1203 20:11:44.462833 14 local_available_controller.go:172] Shutting down LocalAvailability controller I1203 20:11:44.462850 14 system_namespaces_controller.go:76] Shutting down system namespaces controller I1203 20:11:44.462858 14 controller.go:132] Ending legacy_token_tracking_controller I1203 20:11:44.462863 14 controller.go:133] Shutting down legacy_token_tracking_controller I1203 20:11:44.462866 14 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I1203 20:11:44.462876 14 apf_controller.go:389] Shutting down API Priority and Fairness config worker I1203 20:11:44.462883 14 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I1203 20:11:44.462893 14 remote_available_controller.go:449] Shutting down RemoteAvailability controller I1203 20:11:44.462922 14 secure_serving.go:258] Stopped listening on [::]:6443 I1203 20:11:44.462938 14 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I1203 20:11:44.462955 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I1203 20:11:44.462973 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1203 20:11:44.463021 14 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1203 20:11:44.463044 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I1203 20:11:44.463061 14 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1203 20:11:44.462582 14 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1203 20:11:44.463607 14 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I1203 20:11:44.464044 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I1203 20:11:44.464151 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I1203 20:11:44.464619 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" E1203 20:11:44.464966 14 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=11156&timeout=9m52s&timeoutSeconds=592&watch=true" auditID="5c258af3-45ed-4c55-af0b-ea40afc78cda" I1203 20:11:46.463520 14 genericapiserver.go:742] [graceful-termination] apiserver is exiting I1203 20:11:46.463579 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W1203 20:11:46.471670 14 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W1203 20:11:46.472709 14 cacher.go:171] Terminating all watchers from cacher podtemplates W1203 20:11:46.473606 14 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W1203 20:11:46.475602 14 cacher.go:171] Terminating all watchers from cacher configmaps W1203 20:11:46.481972 14 cacher.go:171] Terminating all watchers from cacher endpoints W1203 20:11:46.483075 14 cacher.go:171] Terminating all watchers from cacher services W1203 20:11:46.485266 14 cacher.go:171] Terminating all watchers from cacher namespaces W1203 20:11:46.486127 14 cacher.go:171] Terminating all watchers from cacher secrets W1203 20:11:46.490877 14 cacher.go:171] Terminating all watchers from cacher serviceaccounts W1203 20:11:46.491858 14 cacher.go:171] Terminating all watchers from cacher pods W1203 20:11:46.493513 14 cacher.go:171] Terminating all watchers from cacher limitranges W1203 20:11:46.494425 14 cacher.go:171] Terminating all watchers from cacher nodes W1203 20:11:46.496642 14 cacher.go:171] Terminating all watchers from cacher persistentvolumes W1203 20:11:46.497415 14 cacher.go:171] Terminating all watchers from cacher resourcequotas W1203 20:11:46.498147 14 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W1203 20:11:46.499234 14 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W1203 20:11:46.505689 14 cacher.go:171] Terminating all watchers from cacher jobs.batch W1203 20:11:46.506594 14 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W1203 20:11:46.507756 14 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W1203 20:11:46.509886 14 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W1203 20:11:46.510647 14 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W1203 20:11:46.512438 14 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W1203 20:11:46.513376 14 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W1203 20:11:46.514187 14 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W1203 20:11:46.515024 14 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W1203 20:11:46.515563 14 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W1203 20:11:46.516088 14 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W1203 20:11:46.516840 14 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W1203 20:11:46.517545 14 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W1203 20:11:46.518533 14 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W1203 20:11:46.519458 14 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W1203 20:11:46.522827 14 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W1203 20:11:46.523843 14 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W1203 20:11:46.524445 14 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W1203 20:11:46.524972 14 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W1203 20:11:46.526129 14 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W1203 20:11:46.528039 14 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W1203 20:11:46.528613 14 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W1203 20:11:46.529055 14 cacher.go:171] Terminating all watchers from cacher deployments.apps W1203 20:11:46.530902 14 cacher.go:171] Terminating all watchers from cacher replicasets.apps W1203 20:11:46.531544 14 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W1203 20:11:46.532105 14 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W1203 20:11:46.532831 14 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W1203 20:11:46.534068 14 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W1203 20:11:46.535010 14 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W1203 20:11:46.535722 14 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W1203 20:11:46.536453 14 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W1203 20:11:46.537611 14 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W1203 20:11:46.538291 14 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W1203 20:11:46.538590 14 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W1203 20:11:46.539078 14 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W1203 20:11:46.539597 14 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W1203 20:11:46.539992 14 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W1203 20:11:46.540613 14 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W1203 20:11:46.541064 14 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W1203 20:11:46.541909 14 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W1203 20:11:46.542374 14 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W1203 20:11:46.542850 14 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W1203 20:11:46.543458 14 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W1203 20:11:46.543708 14 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W1203 20:11:46.544446 14 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W1203 20:11:46.544822 14 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W1203 20:11:46.545091 14 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W1203 20:11:46.545939 14 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W1203 20:11:46.546560 14 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W1203 20:11:46.547096 14 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W1203 20:11:46.547468 14 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W1203 20:11:46.548418 14 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W1203 20:11:46.549244 14 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W1203 20:11:46.549891 14 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W1203 20:11:46.550722 14 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W1203 20:11:46.551359 14 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W1203 20:11:46.551928 14 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W1203 20:11:46.552278 14 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W1203 20:11:46.553069 14 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W1203 20:11:46.553828 14 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W1203 20:11:46.554444 14 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W1203 20:11:46.555338 14 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W1203 20:11:46.556044 14 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W1203 20:11:46.556443 14 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W1203 20:11:46.557048 14 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W1203 20:11:46.557501 14 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W1203 20:11:46.558003 14 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W1203 20:11:46.558535 14 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W1203 20:11:46.559029 14 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W1203 20:11:46.559492 14 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W1203 20:11:46.559833 14 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W1203 20:11:46.560155 14 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W1203 20:11:46.560491 14 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W1203 20:11:46.560911 14 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W1203 20:11:46.561266 14 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W1203 20:11:46.561696 14 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W1203 20:11:46.562255 14 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W1203 20:11:46.562657 14 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W1203 20:11:46.563071 14 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W1203 20:11:46.563627 14 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W1203 20:11:46.564052 14 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W1203 20:11:46.564542 14 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W1203 20:11:46.565603 14 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W1203 20:11:46.566509 14 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W1203 20:11:46.567257 14 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W1203 20:11:46.568201 14 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W1203 20:11:46.568705 14 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W1203 20:11:46.569213 14 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W1203 20:11:46.569685 14 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W1203 20:11:46.570968 14 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W1203 20:11:46.571309 14 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W1203 20:11:46.571755 14 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W1203 20:11:46.572049 14 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W1203 20:11:46.572534 14 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W1203 20:11:46.573060 14 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W1203 20:11:46.573401 14 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W1203 20:11:46.573649 14 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W1203 20:11:46.574201 14 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W1203 20:11:46.574608 14 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W1203 20:11:46.574897 14 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W1203 20:11:46.575277 14 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W1203 20:11:46.575615 14 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W1203 20:11:46.576083 14 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W1203 20:11:46.576525 14 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W1203 20:11:46.577202 14 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W1203 20:11:46.577583 14 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W1203 20:11:46.577881 14 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W1203 20:11:46.578464 14 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W1203 20:11:46.578877 14 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W1203 20:11:46.579364 14 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W1203 20:11:46.580056 14 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W1203 20:11:46.580771 14 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W1203 20:11:46.581411 14 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W1203 20:11:46.581941 14 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W1203 20:11:46.582366 14 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W1203 20:11:46.583003 14 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W1203 20:11:46.583312 14 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W1203 20:11:46.583944 14 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W1203 20:11:46.585058 14 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W1203 20:11:46.585384 14 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W1203 20:11:46.585611 14 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W1203 20:11:46.586338 14 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W1203 20:11:46.587346 14 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W1203 20:11:46.587929 14 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W1203 20:11:46.588972 14 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W1203 20:11:46.589430 14 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W1203 20:11:46.590358 14 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W1203 20:11:46.590755 14 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W1203 20:11:46.591136 14 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W1203 20:11:46.591475 14 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W1203 20:11:46.592264 14 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W1203 20:11:46.592999 14 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W1203 20:11:46.593634 14 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W1203 20:11:46.594197 14 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W1203 20:11:46.594946 14 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W1203 20:11:46.595804 14 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W1203 20:11:46.596361 14 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W1203 20:11:46.596901 14 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W1203 20:11:46.597325 14 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W1203 20:11:46.597863 14 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W1203 20:11:46.598325 14 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W1203 20:11:46.599064 14 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W1203 20:11:46.599551 14 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W1203 20:11:46.599920 14 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W1203 20:11:46.600308 14 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W1203 20:11:46.600747 14 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W1203 20:11:46.601422 14 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W1203 20:11:46.601842 14 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W1203 20:11:46.602376 14 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W1203 20:11:46.603204 14 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W1203 20:11:46.603611 14 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W1203 20:11:46.604031 14 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W1203 20:11:46.604456 14 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W1203 20:11:46.604908 14 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W1203 20:11:46.605376 14 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W1203 20:11:46.605975 14 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W1203 20:11:46.606322 14 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W1203 20:11:46.607219 14 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W1203 20:11:46.607706 14 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W1203 20:11:46.608650 14 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W1203 20:11:46.609074 14 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W1203 20:11:46.609428 14 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W1203 20:11:46.609952 14 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W1203 20:11:46.610475 14 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W1203 20:11:46.611063 14 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io I1203 20:11:46.632848 1 main.go:235] Termination finished with exit code 0 I1203 20:11:46.633202 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" --- I1203 20:15:52.410701 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I1203 20:15:52.411836 14 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I1203 20:15:52.411871 14 controller.go:128] Shutting down kubernetes service endpoint reconciler I1203 20:15:52.411879 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I1203 20:15:52.425258 14 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I1203 20:15:52.425314 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W1203 20:15:52.429522 14 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I1203 20:15:52.444996 14 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I1203 20:15:52.445038 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I1203 20:15:52.445336 14 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I1203 20:15:52.445386 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I1203 20:15:52.445502 14 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I1203 20:15:52.494933 14 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I1203 20:15:52.495089 14 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I1203 20:15:52.495101 14 controller.go:157] Shutting down quota evaluator I1203 20:15:52.495183 14 controller.go:176] quota evaluator worker shutdown I1203 20:15:52.495219 14 controller.go:176] quota evaluator worker shutdown I1203 20:15:52.495192 14 controller.go:86] Shutting down OpenAPI V3 AggregationController I1203 20:15:52.495262 14 controller.go:176] quota evaluator worker shutdown I1203 20:15:52.495307 14 controller.go:176] quota evaluator worker shutdown I1203 20:15:52.495144 14 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I1203 20:15:52.495237 14 controller.go:176] quota evaluator worker shutdown I1203 20:15:52.495482 14 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I1203 20:15:52.495522 14 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I1203 20:15:52.495583 14 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1203 20:15:52.495624 14 gc_controller.go:91] Shutting down apiserver lease garbage collector I1203 20:15:52.495625 14 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I1203 20:15:52.495654 14 controller.go:120] Shutting down OpenAPI V3 controller I1203 20:15:52.495671 14 apf_controller.go:389] Shutting down API Priority and Fairness config worker I1203 20:15:52.495671 14 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I1203 20:15:52.495680 14 customresource_discovery_controller.go:328] Shutting down DiscoveryController I1203 20:15:52.495790 14 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I1203 20:15:52.495803 14 controller.go:84] Shutting down OpenAPI AggregationController I1203 20:15:52.495680 14 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1203 20:15:52.495827 14 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I1203 20:15:52.495840 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I1203 20:15:52.495856 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1203 20:15:52.495862 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I1203 20:15:52.495879 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I1203 20:15:52.495882 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I1203 20:15:52.495899 14 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I1203 20:15:52.495690 14 autoregister_controller.go:168] Shutting down autoregister controller I1203 20:15:52.495916 14 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I1203 20:15:52.495660 14 controller.go:132] Ending legacy_token_tracking_controller I1203 20:15:52.495945 14 controller.go:133] Shutting down legacy_token_tracking_controller I1203 20:15:52.495696 14 system_namespaces_controller.go:76] Shutting down system namespaces controller I1203 20:15:52.495700 14 crd_finalizer.go:281] Shutting down CRDFinalizer I1203 20:15:52.495701 14 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I1203 20:15:52.495708 14 remote_available_controller.go:449] Shutting down RemoteAvailability controller I1203 20:15:52.495711 14 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I1203 20:15:52.496077 14 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I1203 20:15:52.495713 14 crdregistration_controller.go:146] Shutting down crd-autoregister controller I1203 20:15:52.495652 14 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I1203 20:15:52.495718 14 local_available_controller.go:172] Shutting down LocalAvailability controller I1203 20:15:52.495722 14 establishing_controller.go:92] Shutting down EstablishingController I1203 20:15:52.495739 14 controller.go:170] Shutting down OpenAPI controller I1203 20:15:52.495747 14 naming_controller.go:305] Shutting down NamingConditionController I1203 20:15:52.495757 14 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I1203 20:15:52.496132 14 secure_serving.go:258] Stopped listening on [::]:6443 I1203 20:15:52.496352 14 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I1203 20:15:52.496372 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening E1203 20:15:52.498052 14 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=12544&timeout=7m12s&timeoutSeconds=432&watch=true" auditID="33ab8e6a-300d-421d-95c6-2a3e86214896" I1203 20:15:54.496868 14 genericapiserver.go:742] [graceful-termination] apiserver is exiting I1203 20:15:54.496925 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W1203 20:15:54.501823 14 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W1203 20:15:54.502580 14 cacher.go:171] Terminating all watchers from cacher services W1203 20:15:54.505666 14 cacher.go:171] Terminating all watchers from cacher configmaps W1203 20:15:54.519210 14 cacher.go:171] Terminating all watchers from cacher limitranges W1203 20:15:54.520291 14 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W1203 20:15:54.521773 14 cacher.go:171] Terminating all watchers from cacher resourcequotas W1203 20:15:54.522537 14 cacher.go:171] Terminating all watchers from cacher secrets W1203 20:15:54.529386 14 cacher.go:171] Terminating all watchers from cacher nodes W1203 20:15:54.532159 14 cacher.go:171] Terminating all watchers from cacher persistentvolumes W1203 20:15:54.533052 14 cacher.go:171] Terminating all watchers from cacher namespaces W1203 20:15:54.534127 14 cacher.go:171] Terminating all watchers from cacher serviceaccounts W1203 20:15:54.564079 14 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W1203 20:15:54.565307 14 cacher.go:171] Terminating all watchers from cacher endpoints W1203 20:15:54.576953 14 cacher.go:171] Terminating all watchers from cacher pods W1203 20:15:54.583436 14 cacher.go:171] Terminating all watchers from cacher podtemplates W1203 20:15:54.584339 14 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W1203 20:15:54.585836 14 cacher.go:171] Terminating all watchers from cacher jobs.batch W1203 20:15:54.586502 14 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W1203 20:15:54.593705 14 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W1203 20:15:54.594957 14 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W1203 20:15:54.595629 14 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W1203 20:15:54.596960 14 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W1203 20:15:54.597883 14 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W1203 20:15:54.599397 14 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W1203 20:15:54.600245 14 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W1203 20:15:54.601480 14 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W1203 20:15:54.602604 14 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W1203 20:15:54.605730 14 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W1203 20:15:54.607528 14 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W1203 20:15:54.609083 14 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W1203 20:15:54.609796 14 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W1203 20:15:54.621431 14 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W1203 20:15:54.622683 14 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W1203 20:15:54.627278 14 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W1203 20:15:54.628167 14 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W1203 20:15:54.628712 14 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W1203 20:15:54.630165 14 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W1203 20:15:54.630726 14 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W1203 20:15:54.631187 14 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W1203 20:15:54.631600 14 cacher.go:171] Terminating all watchers from cacher deployments.apps W1203 20:15:54.635219 14 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W1203 20:15:54.636089 14 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W1203 20:15:54.641473 14 cacher.go:171] Terminating all watchers from cacher replicasets.apps W1203 20:15:54.641904 14 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W1203 20:15:54.642696 14 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W1203 20:15:54.650876 14 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W1203 20:15:54.651793 14 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W1203 20:15:54.653571 14 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W1203 20:15:54.655126 14 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W1203 20:15:54.656082 14 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W1203 20:15:54.657173 14 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W1203 20:15:54.657667 14 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W1203 20:15:54.658527 14 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W1203 20:15:54.664522 14 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W1203 20:15:54.667524 14 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W1203 20:15:54.668177 14 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W1203 20:15:54.670751 14 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W1203 20:15:54.677116 14 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W1203 20:15:54.677683 14 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W1203 20:15:54.678822 14 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W1203 20:15:54.680024 14 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W1203 20:15:54.680700 14 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W1203 20:15:54.681871 14 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W1203 20:15:54.682507 14 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W1203 20:15:54.682870 14 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W1203 20:15:54.683393 14 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W1203 20:15:54.684044 14 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W1203 20:15:54.684809 14 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W1203 20:15:54.686571 14 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W1203 20:15:54.688536 14 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W1203 20:15:54.691065 14 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W1203 20:15:54.691472 14 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W1203 20:15:54.693995 14 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W1203 20:15:54.694353 14 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W1203 20:15:54.694645 14 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W1203 20:15:54.695041 14 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W1203 20:15:54.695885 14 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W1203 20:15:54.696446 14 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W1203 20:15:54.697749 14 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W1203 20:15:54.698343 14 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W1203 20:15:54.698829 14 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W1203 20:15:54.699148 14 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W1203 20:15:54.699487 14 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W1203 20:15:54.699766 14 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W1203 20:15:54.700224 14 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W1203 20:15:54.701142 14 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W1203 20:15:54.701446 14 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W1203 20:15:54.701910 14 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W1203 20:15:54.702176 14 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W1203 20:15:54.702500 14 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W1203 20:15:54.702901 14 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W1203 20:15:54.703626 14 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W1203 20:15:54.704020 14 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W1203 20:15:54.704557 14 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W1203 20:15:54.704935 14 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W1203 20:15:54.705438 14 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W1203 20:15:54.706193 14 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W1203 20:15:54.707089 14 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W1203 20:15:54.707643 14 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W1203 20:15:54.708140 14 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W1203 20:15:54.708452 14 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W1203 20:15:54.708979 14 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W1203 20:15:54.709376 14 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W1203 20:15:54.709739 14 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W1203 20:15:54.710027 14 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W1203 20:15:54.710411 14 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W1203 20:15:54.711302 14 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W1203 20:15:54.711923 14 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W1203 20:15:54.712435 14 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W1203 20:15:54.713901 14 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W1203 20:15:54.714416 14 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W1203 20:15:54.714857 14 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W1203 20:15:54.715059 14 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W1203 20:15:54.715698 14 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W1203 20:15:54.716294 14 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W1203 20:15:54.719455 14 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W1203 20:15:54.720000 14 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W1203 20:15:54.721568 14 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W1203 20:15:54.722319 14 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W1203 20:15:54.723512 14 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W1203 20:15:54.724269 14 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W1203 20:15:54.724731 14 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W1203 20:15:54.725208 14 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W1203 20:15:54.725951 14 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W1203 20:15:54.730393 14 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W1203 20:15:54.731793 14 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W1203 20:15:54.732287 14 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W1203 20:15:54.732938 14 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W1203 20:15:54.735764 14 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W1203 20:15:54.736906 14 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W1203 20:15:54.737864 14 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W1203 20:15:54.739066 14 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W1203 20:15:54.739427 14 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W1203 20:15:54.739874 14 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W1203 20:15:54.740724 14 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W1203 20:15:54.741763 14 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W1203 20:15:54.742498 14 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W1203 20:15:54.742938 14 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W1203 20:15:54.743323 14 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W1203 20:15:54.744092 14 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W1203 20:15:54.745331 14 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W1203 20:15:54.745697 14 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W1203 20:15:54.746213 14 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W1203 20:15:54.746803 14 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W1203 20:15:54.747387 14 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W1203 20:15:54.747994 14 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W1203 20:15:54.748743 14 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W1203 20:15:54.749343 14 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W1203 20:15:54.749849 14 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W1203 20:15:54.750513 14 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W1203 20:15:54.750993 14 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W1203 20:15:54.751906 14 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W1203 20:15:54.752400 14 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W1203 20:15:54.753676 14 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W1203 20:15:54.754071 14 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W1203 20:15:54.754657 14 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W1203 20:15:54.755236 14 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W1203 20:15:54.755671 14 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W1203 20:15:54.756071 14 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W1203 20:15:54.756688 14 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W1203 20:15:54.757150 14 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W1203 20:15:54.758002 14 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W1203 20:15:54.758500 14 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W1203 20:15:54.758856 14 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W1203 20:15:54.759229 14 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W1203 20:15:54.759507 14 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W1203 20:15:54.759751 14 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W1203 20:15:54.760298 14 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W1203 20:15:54.760664 14 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W1203 20:15:54.761170 14 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W1203 20:15:54.761616 14 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W1203 20:15:54.762812 14 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W1203 20:15:54.763348 14 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io W1203 20:15:54.764176 14 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W1203 20:15:54.764693 14 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W1203 20:15:54.765254 14 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W1203 20:15:54.766166 14 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W1203 20:15:54.766583 14 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W1203 20:15:54.767072 14 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W1203 20:15:54.767374 14 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W1203 20:15:54.768037 14 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W1203 20:15:54.768763 14 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W1203 20:15:54.769485 14 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io I1203 20:15:54.800410 1 main.go:235] Termination finished with exit code 0 I1203 20:15:54.800493 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" ---