--- I0308 03:22:21.465883 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I0308 03:22:21.467083 12 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0308 03:22:21.467110 12 controller.go:128] Shutting down kubernetes service endpoint reconciler I0308 03:22:21.467137 12 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving W0308 03:22:21.478904 12 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I0308 03:22:21.482949 12 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I0308 03:22:21.483087 12 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished I0308 03:22:21.502167 12 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0308 03:22:21.502199 12 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I0308 03:22:21.502256 12 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I0308 03:22:21.502283 12 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I0308 03:22:21.502417 12 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I0308 03:22:21.531624 12 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I0308 03:22:21.531707 12 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I0308 03:22:21.531797 12 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I0308 03:22:21.531875 12 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I0308 03:22:21.531935 12 controller.go:157] Shutting down quota evaluator I0308 03:22:21.531954 12 controller.go:176] quota evaluator worker shutdown I0308 03:22:21.532909 12 controller.go:84] Shutting down OpenAPI AggregationController I0308 03:22:21.532945 12 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0308 03:22:21.532960 12 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0308 03:22:21.532972 12 controller.go:86] Shutting down OpenAPI V3 AggregationController I0308 03:22:21.533133 12 controller.go:176] quota evaluator worker shutdown I0308 03:22:21.533159 12 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0308 03:22:21.533175 12 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0308 03:22:21.533191 12 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I0308 03:22:21.533254 12 secure_serving.go:258] Stopped listening on [::]:6443 I0308 03:22:21.533275 12 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0308 03:22:21.533275 12 controller.go:176] quota evaluator worker shutdown I0308 03:22:21.533282 12 controller.go:176] quota evaluator worker shutdown I0308 03:22:21.533290 12 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I0308 03:22:21.533555 12 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0308 03:22:21.534060 12 controller.go:176] quota evaluator worker shutdown I0308 03:22:21.534105 12 controller.go:170] Shutting down OpenAPI controller E0308 03:22:21.534467 12 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=9539&timeout=9m36s&timeoutSeconds=576&watch=true" auditID="230cb5fc-b968-4ad4-984f-23a83bea6eb8" I0308 03:22:21.534627 12 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I0308 03:22:21.534671 12 controller.go:120] Shutting down OpenAPI V3 controller I0308 03:22:21.534695 12 customresource_discovery_controller.go:328] Shutting down DiscoveryController I0308 03:22:21.534715 12 autoregister_controller.go:168] Shutting down autoregister controller I0308 03:22:21.534739 12 crdregistration_controller.go:146] Shutting down crd-autoregister controller I0308 03:22:21.534899 12 remote_available_controller.go:449] Shutting down RemoteAvailability controller I0308 03:22:21.534915 12 establishing_controller.go:92] Shutting down EstablishingController I0308 03:22:21.534935 12 gc_controller.go:91] Shutting down apiserver lease garbage collector I0308 03:22:21.534940 12 crd_finalizer.go:281] Shutting down CRDFinalizer I0308 03:22:21.534960 12 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I0308 03:22:21.534967 12 controller.go:132] Ending legacy_token_tracking_controller I0308 03:22:21.534996 12 controller.go:133] Shutting down legacy_token_tracking_controller I0308 03:22:21.535022 12 local_available_controller.go:172] Shutting down LocalAvailability controller I0308 03:22:21.534875 12 naming_controller.go:305] Shutting down NamingConditionController I0308 03:22:21.535055 12 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I0308 03:22:21.535060 12 apf_controller.go:389] Shutting down API Priority and Fairness config worker I0308 03:22:21.535265 12 system_namespaces_controller.go:76] Shutting down system namespaces controller I0308 03:22:21.534998 12 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I0308 03:22:21.534983 12 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I0308 03:22:21.535305 12 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I0308 03:22:21.535355 12 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I0308 03:22:21.535430 12 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I0308 03:22:21.536956 12 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I0308 03:22:21.537039 12 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I0308 03:22:21.537056 12 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0308 03:22:21.537070 12 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0308 03:22:21.537092 12 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0308 03:22:23.534257 12 genericapiserver.go:742] [graceful-termination] apiserver is exiting I0308 03:22:23.534309 12 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W0308 03:22:23.537409 12 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W0308 03:22:23.538439 12 cacher.go:171] Terminating all watchers from cacher services W0308 03:22:23.541639 12 cacher.go:171] Terminating all watchers from cacher endpoints W0308 03:22:23.542772 12 cacher.go:171] Terminating all watchers from cacher limitranges W0308 03:22:23.543666 12 cacher.go:171] Terminating all watchers from cacher persistentvolumes W0308 03:22:23.545778 12 cacher.go:171] Terminating all watchers from cacher secrets W0308 03:22:23.549474 12 cacher.go:171] Terminating all watchers from cacher pods W0308 03:22:23.551785 12 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W0308 03:22:23.554022 12 cacher.go:171] Terminating all watchers from cacher serviceaccounts W0308 03:22:23.556811 12 cacher.go:171] Terminating all watchers from cacher resourcequotas W0308 03:22:23.557476 12 cacher.go:171] Terminating all watchers from cacher configmaps W0308 03:22:23.563303 12 cacher.go:171] Terminating all watchers from cacher nodes W0308 03:22:23.564389 12 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W0308 03:22:23.565452 12 cacher.go:171] Terminating all watchers from cacher namespaces W0308 03:22:23.568857 12 cacher.go:171] Terminating all watchers from cacher jobs.batch W0308 03:22:23.570873 12 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W0308 03:22:23.572059 12 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W0308 03:22:23.574068 12 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W0308 03:22:23.574993 12 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W0308 03:22:23.576689 12 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W0308 03:22:23.577439 12 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W0308 03:22:23.578408 12 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W0308 03:22:23.579453 12 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W0308 03:22:23.581469 12 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W0308 03:22:23.582855 12 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W0308 03:22:23.584125 12 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W0308 03:22:23.585809 12 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W0308 03:22:23.586688 12 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W0308 03:22:23.587831 12 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W0308 03:22:23.588445 12 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W0308 03:22:23.588961 12 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W0308 03:22:23.591164 12 cacher.go:171] Terminating all watchers from cacher replicasets.apps W0308 03:22:23.592659 12 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W0308 03:22:23.594196 12 cacher.go:171] Terminating all watchers from cacher deployments.apps W0308 03:22:23.598595 12 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W0308 03:22:23.599236 12 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W0308 03:22:23.605196 12 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W0308 03:22:23.605664 12 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W0308 03:22:23.606228 12 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W0308 03:22:23.608084 12 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W0308 03:22:23.608664 12 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W0308 03:22:23.609091 12 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W0308 03:22:23.609782 12 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W0308 03:22:23.611877 12 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W0308 03:22:23.612419 12 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W0308 03:22:23.613009 12 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W0308 03:22:23.613462 12 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W0308 03:22:23.614024 12 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W0308 03:22:23.614655 12 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W0308 03:22:23.615164 12 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W0308 03:22:23.615729 12 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W0308 03:22:23.617277 12 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W0308 03:22:23.617841 12 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W0308 03:22:23.618384 12 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W0308 03:22:23.619534 12 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com W0308 03:22:23.620253 12 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W0308 03:22:23.620732 12 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W0308 03:22:23.622374 12 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W0308 03:22:23.623303 12 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W0308 03:22:23.623649 12 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W0308 03:22:23.624275 12 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W0308 03:22:23.624849 12 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W0308 03:22:23.626108 12 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W0308 03:22:23.627509 12 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W0308 03:22:23.628327 12 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W0308 03:22:23.628815 12 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W0308 03:22:23.629523 12 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W0308 03:22:23.630136 12 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W0308 03:22:23.631260 12 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W0308 03:22:23.632023 12 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W0308 03:22:23.632662 12 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W0308 03:22:23.633098 12 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W0308 03:22:23.633713 12 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W0308 03:22:23.634352 12 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W0308 03:22:23.635382 12 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W0308 03:22:23.636505 12 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W0308 03:22:23.637380 12 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W0308 03:22:23.638082 12 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W0308 03:22:23.638459 12 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W0308 03:22:23.638874 12 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W0308 03:22:23.639211 12 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W0308 03:22:23.639697 12 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W0308 03:22:23.640336 12 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W0308 03:22:23.641017 12 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W0308 03:22:23.641763 12 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W0308 03:22:23.642175 12 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W0308 03:22:23.642535 12 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W0308 03:22:23.643009 12 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W0308 03:22:23.643823 12 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W0308 03:22:23.644374 12 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W0308 03:22:23.645010 12 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W0308 03:22:23.645468 12 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W0308 03:22:23.646709 12 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W0308 03:22:23.647102 12 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W0308 03:22:23.647603 12 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W0308 03:22:23.648632 12 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W0308 03:22:23.648992 12 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W0308 03:22:23.649574 12 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W0308 03:22:23.650137 12 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W0308 03:22:23.650649 12 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W0308 03:22:23.651004 12 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W0308 03:22:23.651601 12 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W0308 03:22:23.652322 12 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W0308 03:22:23.652922 12 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W0308 03:22:23.653462 12 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W0308 03:22:23.653941 12 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W0308 03:22:23.654593 12 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W0308 03:22:23.655216 12 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W0308 03:22:23.655735 12 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W0308 03:22:23.656185 12 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W0308 03:22:23.656783 12 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W0308 03:22:23.657267 12 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W0308 03:22:23.657899 12 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W0308 03:22:23.658352 12 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W0308 03:22:23.659142 12 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W0308 03:22:23.659569 12 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W0308 03:22:23.660186 12 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W0308 03:22:23.660810 12 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W0308 03:22:23.661591 12 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W0308 03:22:23.664032 12 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W0308 03:22:23.664444 12 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W0308 03:22:23.665284 12 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W0308 03:22:23.665893 12 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com I0308 03:22:23.693525 1 main.go:235] Termination finished with exit code 0 I0308 03:22:23.693908 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" --- I0308 03:27:03.955524 1 main.go:213] Received signal terminated. Forwarding to sub-process "hyperkube". I0308 03:27:03.955903 14 genericapiserver.go:554] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0308 03:27:03.955955 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ShutdownInitiated' Received signal to terminate, becoming unready, but keeping serving I0308 03:27:03.955998 14 controller.go:128] Shutting down kubernetes service endpoint reconciler I0308 03:27:03.967040 14 genericapiserver.go:544] "[graceful-termination] shutdown event" name="AfterShutdownDelayDuration" I0308 03:27:03.967104 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'AfterShutdownDelayDuration' The minimal shutdown duration of 0s finished W0308 03:27:03.973578 14 lease.go:265] Resetting endpoints for master service "kubernetes" to [] I0308 03:27:03.992358 14 genericapiserver.go:721] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0308 03:27:03.992408 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationPreShutdownHooksFinished' All pre-shutdown hooks have been finished I0308 03:27:03.992690 14 genericapiserver.go:633] "[graceful-termination] shutdown event" name="NotAcceptingNewRequest" I0308 03:27:03.992742 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'InFlightRequestsDrained' All non long-running request(s) in-flight have drained I0308 03:27:03.992844 14 genericapiserver.go:679] "[graceful-termination] not going to wait for active watch request(s) to drain" I0308 03:27:04.053095 14 genericapiserver.go:670] [graceful-termination] in-flight non long-running request(s) have drained I0308 03:27:04.053129 14 controller.go:157] Shutting down quota evaluator I0308 03:27:04.053142 14 controller.go:176] quota evaluator worker shutdown I0308 03:27:04.053156 14 genericapiserver.go:735] "[graceful-termination] audit backend shutdown completed" I0308 03:27:04.053178 14 controller.go:176] quota evaluator worker shutdown I0308 03:27:04.053129 14 genericapiserver.go:711] "[graceful-termination] shutdown event" name="InFlightRequestsDrained" I0308 03:27:04.053187 14 controller.go:176] quota evaluator worker shutdown I0308 03:27:04.053419 14 controller.go:176] quota evaluator worker shutdown I0308 03:27:04.053460 14 controller.go:176] quota evaluator worker shutdown I0308 03:27:04.053618 14 dynamic_serving_content.go:149] "Shutting down controller" name="aggregator-proxy-cert::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.crt::/etc/kubernetes/static-pod-certs/secrets/aggregator-client/tls.key" I0308 03:27:04.053865 14 gc_controller.go:91] Shutting down apiserver lease garbage collector I0308 03:27:04.053958 14 apiaccess_count_controller.go:95] Shutting down APIRequestCount controller. I0308 03:27:04.053992 14 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0308 03:27:04.054025 14 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0308 03:27:04.054053 14 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0308 03:27:04.054077 14 controller.go:86] Shutting down OpenAPI V3 AggregationController I0308 03:27:04.054102 14 controller.go:84] Shutting down OpenAPI AggregationController I0308 03:27:04.054816 14 storage_flowcontrol.go:186] APF bootstrap ensurer is exiting I0308 03:27:04.054910 14 apiapproval_controller.go:201] Shutting down KubernetesAPIApprovalPolicyConformantConditionController I0308 03:27:04.054929 14 cluster_authentication_trust_controller.go:466] Shutting down cluster_authentication_trust_controller controller I0308 03:27:04.054918 14 dynamic_cafile_content.go:175] "Shutting down controller" name="request-header::/etc/kubernetes/static-pod-certs/configmaps/aggregator-client-ca/ca-bundle.crt" I0308 03:27:04.054960 14 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0308 03:27:04.055014 14 secure_serving.go:258] Stopped listening on [::]:6443 E0308 03:27:04.055015 14 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="GET" URI="/apis/user.openshift.io/v1/groups?allowWatchBookmarks=true&resourceVersion=11659&timeout=7m16s&timeoutSeconds=436&watch=true" auditID="9515c711-4596-4995-8354-557dcc39a84c" I0308 03:27:04.055034 14 genericapiserver.go:624] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0308 03:27:04.055041 14 clusterquotamapping.go:142] Shutting down ClusterQuotaMappingController controller I0308 03:27:04.055046 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'HTTPServerStoppedListening' HTTP Server has stopped listening I0308 03:27:04.055070 14 controller.go:120] Shutting down OpenAPI V3 controller I0308 03:27:04.055084 14 customresource_discovery_controller.go:328] Shutting down DiscoveryController I0308 03:27:04.055123 14 crd_finalizer.go:281] Shutting down CRDFinalizer I0308 03:27:04.055147 14 nonstructuralschema_controller.go:207] Shutting down NonStructuralSchemaConditionController I0308 03:27:04.055157 14 establishing_controller.go:92] Shutting down EstablishingController I0308 03:27:04.055174 14 apiservice_controller.go:134] Shutting down APIServiceRegistrationController I0308 03:27:04.055181 14 autoregister_controller.go:168] Shutting down autoregister controller I0308 03:27:04.055196 14 crdregistration_controller.go:146] Shutting down crd-autoregister controller I0308 03:27:04.055201 14 apf_controller.go:389] Shutting down API Priority and Fairness config worker I0308 03:27:04.055208 14 controller.go:170] Shutting down OpenAPI controller I0308 03:27:04.055216 14 controller.go:132] Ending legacy_token_tracking_controller I0308 03:27:04.055221 14 controller.go:133] Shutting down legacy_token_tracking_controller I0308 03:27:04.055234 14 local_available_controller.go:172] Shutting down LocalAvailability controller I0308 03:27:04.055245 14 remote_available_controller.go:449] Shutting down RemoteAvailability controller I0308 03:27:04.055258 14 system_namespaces_controller.go:76] Shutting down system namespaces controller I0308 03:27:04.055175 14 naming_controller.go:305] Shutting down NamingConditionController I0308 03:27:04.055269 14 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt" I0308 03:27:04.055112 14 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0308 03:27:04.056457 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/internal-loadbalancer-serving-certkey/tls.key" I0308 03:27:04.056712 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.crt::/etc/kubernetes/static-pod-resources/secrets/localhost-recovery-serving-certkey/tls.key" I0308 03:27:04.056579 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/external-loadbalancer-serving-certkey/tls.key" I0308 03:27:04.056589 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key" I0308 03:27:04.056595 14 dynamic_serving_content.go:149] "Shutting down controller" name="sni-serving-cert::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.crt::/etc/kubernetes/static-pod-certs/secrets/localhost-serving-cert-certkey/tls.key" I0308 03:27:06.055632 14 genericapiserver.go:742] [graceful-termination] apiserver is exiting I0308 03:27:06.055702 14 patch_genericapiserver.go:97] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-kube-apiserver", Name:"kube-apiserver-master-0", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TerminationGracefulTerminationFinished' All pending requests processed W0308 03:27:06.059046 14 cacher.go:171] Terminating all watchers from cacher apiservices.apiregistration.k8s.io W0308 03:27:06.060529 14 cacher.go:171] Terminating all watchers from cacher persistentvolumeclaims W0308 03:27:06.061609 14 cacher.go:171] Terminating all watchers from cacher pods W0308 03:27:06.066192 14 cacher.go:171] Terminating all watchers from cacher replicationcontrollers W0308 03:27:06.067387 14 cacher.go:171] Terminating all watchers from cacher limitranges W0308 03:27:06.073015 14 cacher.go:171] Terminating all watchers from cacher serviceaccounts W0308 03:27:06.077695 14 cacher.go:171] Terminating all watchers from cacher podtemplates W0308 03:27:06.079785 14 cacher.go:171] Terminating all watchers from cacher endpoints W0308 03:27:06.083047 14 cacher.go:171] Terminating all watchers from cacher services W0308 03:27:06.094013 14 cacher.go:171] Terminating all watchers from cacher persistentvolumes W0308 03:27:06.095167 14 cacher.go:171] Terminating all watchers from cacher namespaces W0308 03:27:06.105306 14 cacher.go:171] Terminating all watchers from cacher secrets W0308 03:27:06.113064 14 cacher.go:171] Terminating all watchers from cacher nodes W0308 03:27:06.119262 14 cacher.go:171] Terminating all watchers from cacher configmaps W0308 03:27:06.134234 14 cacher.go:171] Terminating all watchers from cacher resourcequotas W0308 03:27:06.138584 14 cacher.go:171] Terminating all watchers from cacher horizontalpodautoscalers.autoscaling W0308 03:27:06.141266 14 cacher.go:171] Terminating all watchers from cacher jobs.batch W0308 03:27:06.143659 14 cacher.go:171] Terminating all watchers from cacher cronjobs.batch W0308 03:27:06.146089 14 cacher.go:171] Terminating all watchers from cacher certificatesigningrequests.certificates.k8s.io W0308 03:27:06.147558 14 cacher.go:171] Terminating all watchers from cacher leases.coordination.k8s.io W0308 03:27:06.148085 14 cacher.go:171] Terminating all watchers from cacher endpointslices.discovery.k8s.io W0308 03:27:06.148635 14 cacher.go:171] Terminating all watchers from cacher networkpolicies.networking.k8s.io W0308 03:27:06.150254 14 cacher.go:171] Terminating all watchers from cacher ingresses.networking.k8s.io W0308 03:27:06.150956 14 cacher.go:171] Terminating all watchers from cacher ingressclasses.networking.k8s.io W0308 03:27:06.151856 14 cacher.go:171] Terminating all watchers from cacher runtimeclasses.node.k8s.io W0308 03:27:06.152397 14 cacher.go:171] Terminating all watchers from cacher poddisruptionbudgets.policy W0308 03:27:06.153687 14 cacher.go:171] Terminating all watchers from cacher clusterroles.rbac.authorization.k8s.io W0308 03:27:06.154596 14 cacher.go:171] Terminating all watchers from cacher clusterrolebindings.rbac.authorization.k8s.io W0308 03:27:06.156295 14 cacher.go:171] Terminating all watchers from cacher roles.rbac.authorization.k8s.io W0308 03:27:06.158952 14 cacher.go:171] Terminating all watchers from cacher rolebindings.rbac.authorization.k8s.io W0308 03:27:06.160307 14 cacher.go:171] Terminating all watchers from cacher priorityclasses.scheduling.k8s.io W0308 03:27:06.160739 14 cacher.go:171] Terminating all watchers from cacher csinodes.storage.k8s.io W0308 03:27:06.161094 14 cacher.go:171] Terminating all watchers from cacher csidrivers.storage.k8s.io W0308 03:27:06.162015 14 cacher.go:171] Terminating all watchers from cacher csistoragecapacities.storage.k8s.io W0308 03:27:06.170680 14 cacher.go:171] Terminating all watchers from cacher storageclasses.storage.k8s.io W0308 03:27:06.171677 14 cacher.go:171] Terminating all watchers from cacher volumeattachments.storage.k8s.io W0308 03:27:06.173017 14 cacher.go:171] Terminating all watchers from cacher prioritylevelconfigurations.flowcontrol.apiserver.k8s.io W0308 03:27:06.173390 14 cacher.go:171] Terminating all watchers from cacher flowschemas.flowcontrol.apiserver.k8s.io W0308 03:27:06.173904 14 cacher.go:171] Terminating all watchers from cacher replicasets.apps W0308 03:27:06.174844 14 cacher.go:171] Terminating all watchers from cacher controllerrevisions.apps W0308 03:27:06.175408 14 cacher.go:171] Terminating all watchers from cacher deployments.apps W0308 03:27:06.177732 14 cacher.go:171] Terminating all watchers from cacher statefulsets.apps W0308 03:27:06.178920 14 cacher.go:171] Terminating all watchers from cacher daemonsets.apps W0308 03:27:06.179935 14 cacher.go:171] Terminating all watchers from cacher validatingwebhookconfigurations.admissionregistration.k8s.io W0308 03:27:06.181504 14 cacher.go:171] Terminating all watchers from cacher mutatingwebhookconfigurations.admissionregistration.k8s.io W0308 03:27:06.182347 14 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicies.admissionregistration.k8s.io W0308 03:27:06.182832 14 cacher.go:171] Terminating all watchers from cacher validatingadmissionpolicybindings.admissionregistration.k8s.io W0308 03:27:06.184058 14 cacher.go:171] Terminating all watchers from cacher customresourcedefinitions.apiextensions.k8s.io W0308 03:27:06.184956 14 cacher.go:171] Terminating all watchers from cacher podmonitors.monitoring.coreos.com W0308 03:27:06.185647 14 cacher.go:171] Terminating all watchers from cacher provisionings.metal3.io W0308 03:27:06.186178 14 cacher.go:171] Terminating all watchers from cacher containerruntimeconfigs.machineconfiguration.openshift.io W0308 03:27:06.187010 14 cacher.go:171] Terminating all watchers from cacher network-attachment-definitions.k8s.cni.cncf.io W0308 03:27:06.188113 14 cacher.go:171] Terminating all watchers from cacher servicecas.operator.openshift.io W0308 03:27:06.188543 14 cacher.go:171] Terminating all watchers from cacher clusteruserdefinednetworks.k8s.ovn.org W0308 03:27:06.189021 14 cacher.go:171] Terminating all watchers from cacher clusterversions.config.openshift.io W0308 03:27:06.189843 14 cacher.go:171] Terminating all watchers from cacher metal3remediationtemplates.infrastructure.cluster.x-k8s.io W0308 03:27:06.190227 14 cacher.go:171] Terminating all watchers from cacher operatorhubs.config.openshift.io W0308 03:27:06.190743 14 cacher.go:171] Terminating all watchers from cacher oauths.config.openshift.io W0308 03:27:06.191383 14 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0308 03:27:06.191832 14 cacher.go:171] Terminating all watchers from cacher alertmanagerconfigs.monitoring.coreos.com W0308 03:27:06.192229 14 cacher.go:171] Terminating all watchers from cacher userdefinednetworks.k8s.ovn.org W0308 03:27:06.193057 14 cacher.go:171] Terminating all watchers from cacher kubeletconfigs.machineconfiguration.openshift.io W0308 03:27:06.193644 14 cacher.go:171] Terminating all watchers from cacher consolenotifications.console.openshift.io W0308 03:27:06.194059 14 cacher.go:171] Terminating all watchers from cacher machineconfigs.machineconfiguration.openshift.io W0308 03:27:06.194704 14 cacher.go:171] Terminating all watchers from cacher ingresscontrollers.operator.openshift.io W0308 03:27:06.195439 14 cacher.go:171] Terminating all watchers from cacher storageversionmigrations.migration.k8s.io W0308 03:27:06.196190 14 cacher.go:171] Terminating all watchers from cacher consoles.config.openshift.io W0308 03:27:06.196743 14 cacher.go:171] Terminating all watchers from cacher adminpolicybasedexternalroutes.k8s.ovn.org W0308 03:27:06.197194 14 cacher.go:171] Terminating all watchers from cacher nodes.config.openshift.io W0308 03:27:06.197453 14 cacher.go:171] Terminating all watchers from cacher imagepruners.imageregistry.operator.openshift.io W0308 03:27:06.198663 14 cacher.go:171] Terminating all watchers from cacher clusterextensions.olm.operatorframework.io W0308 03:27:06.199344 14 cacher.go:171] Terminating all watchers from cacher servicemonitors.monitoring.coreos.com W0308 03:27:06.199999 14 cacher.go:171] Terminating all watchers from cacher subscriptions.operators.coreos.com W0308 03:27:06.200709 14 cacher.go:171] Terminating all watchers from cacher baremetalhosts.metal3.io W0308 03:27:06.201305 14 cacher.go:171] Terminating all watchers from cacher consoles.operator.openshift.io W0308 03:27:06.201750 14 cacher.go:171] Terminating all watchers from cacher cloudcredentials.operator.openshift.io W0308 03:27:06.202161 14 cacher.go:171] Terminating all watchers from cacher overlappingrangeipreservations.whereabouts.cni.cncf.io W0308 03:27:06.202821 14 cacher.go:171] Terminating all watchers from cacher kubecontrollermanagers.operator.openshift.io W0308 03:27:06.203468 14 cacher.go:171] Terminating all watchers from cacher consoleexternalloglinks.console.openshift.io W0308 03:27:06.204004 14 cacher.go:171] Terminating all watchers from cacher alertrelabelconfigs.monitoring.openshift.io W0308 03:27:06.204587 14 cacher.go:171] Terminating all watchers from cacher preprovisioningimages.metal3.io W0308 03:27:06.204977 14 cacher.go:171] Terminating all watchers from cacher profiles.tuned.openshift.io W0308 03:27:06.205989 14 cacher.go:171] Terminating all watchers from cacher consoleclidownloads.console.openshift.io W0308 03:27:06.206500 14 cacher.go:171] Terminating all watchers from cacher clustercatalogs.olm.operatorframework.io W0308 03:27:06.206900 14 cacher.go:171] Terminating all watchers from cacher adminnetworkpolicies.policy.networking.k8s.io W0308 03:27:06.207185 14 cacher.go:171] Terminating all watchers from cacher csisnapshotcontrollers.operator.openshift.io W0308 03:27:06.207548 14 cacher.go:171] Terminating all watchers from cacher operators.operators.coreos.com W0308 03:27:06.207905 14 cacher.go:171] Terminating all watchers from cacher firmwareschemas.metal3.io W0308 03:27:06.208460 14 cacher.go:171] Terminating all watchers from cacher nodeslicepools.whereabouts.cni.cncf.io W0308 03:27:06.209088 14 cacher.go:171] Terminating all watchers from cacher featuregates.config.openshift.io W0308 03:27:06.213150 14 cacher.go:171] Terminating all watchers from cacher bmceventsubscriptions.metal3.io W0308 03:27:06.213830 14 cacher.go:171] Terminating all watchers from cacher controlplanemachinesets.machine.openshift.io W0308 03:27:06.214708 14 cacher.go:171] Terminating all watchers from cacher ipaddresses.ipam.cluster.x-k8s.io W0308 03:27:06.215139 14 cacher.go:171] Terminating all watchers from cacher egressservices.k8s.ovn.org W0308 03:27:06.215810 14 cacher.go:171] Terminating all watchers from cacher proxies.config.openshift.io W0308 03:27:06.216843 14 cacher.go:171] Terminating all watchers from cacher installplans.operators.coreos.com W0308 03:27:06.217395 14 cacher.go:171] Terminating all watchers from cacher machineautoscalers.autoscaling.openshift.io W0308 03:27:06.218600 14 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io W0308 03:27:06.219262 14 cacher.go:171] Terminating all watchers from cacher kubeapiservers.operator.openshift.io W0308 03:27:06.219730 14 cacher.go:171] Terminating all watchers from cacher clusteroperators.config.openshift.io W0308 03:27:06.221295 14 cacher.go:171] Terminating all watchers from cacher authentications.operator.openshift.io W0308 03:27:06.221737 14 cacher.go:171] Terminating all watchers from cacher storagestates.migration.k8s.io W0308 03:27:06.223063 14 cacher.go:171] Terminating all watchers from cacher clustercsidrivers.operator.openshift.io W0308 03:27:06.223435 14 cacher.go:171] Terminating all watchers from cacher storages.operator.openshift.io W0308 03:27:06.223927 14 cacher.go:171] Terminating all watchers from cacher imagecontentpolicies.config.openshift.io W0308 03:27:06.224304 14 cacher.go:171] Terminating all watchers from cacher machines.machine.openshift.io W0308 03:27:06.224705 14 cacher.go:171] Terminating all watchers from cacher machineconfigpools.machineconfiguration.openshift.io W0308 03:27:06.225073 14 cacher.go:171] Terminating all watchers from cacher controllerconfigs.machineconfiguration.openshift.io W0308 03:27:06.225386 14 cacher.go:171] Terminating all watchers from cacher alertingrules.monitoring.openshift.io W0308 03:27:06.225801 14 cacher.go:171] Terminating all watchers from cacher apiservers.config.openshift.io W0308 03:27:06.226066 14 cacher.go:171] Terminating all watchers from cacher credentialsrequests.cloudcredential.openshift.io W0308 03:27:06.226589 14 cacher.go:171] Terminating all watchers from cacher ippools.whereabouts.cni.cncf.io W0308 03:27:06.226942 14 cacher.go:171] Terminating all watchers from cacher baselineadminnetworkpolicies.policy.networking.k8s.io W0308 03:27:06.227408 14 cacher.go:171] Terminating all watchers from cacher prometheusrules.monitoring.coreos.com W0308 03:27:06.228054 14 cacher.go:171] Terminating all watchers from cacher operatorpkis.network.operator.openshift.io W0308 03:27:06.228469 14 cacher.go:171] Terminating all watchers from cacher openshiftapiservers.operator.openshift.io W0308 03:27:06.230086 14 cacher.go:171] Terminating all watchers from cacher hostfirmwaresettings.metal3.io W0308 03:27:06.230525 14 cacher.go:171] Terminating all watchers from cacher olmconfigs.operators.coreos.com W0308 03:27:06.231002 14 cacher.go:171] Terminating all watchers from cacher configs.operator.openshift.io W0308 03:27:06.231576 14 cacher.go:171] Terminating all watchers from cacher configs.samples.operator.openshift.io W0308 03:27:06.232128 14 cacher.go:171] Terminating all watchers from cacher probes.monitoring.coreos.com W0308 03:27:06.232894 14 cacher.go:171] Terminating all watchers from cacher helmchartrepositories.helm.openshift.io W0308 03:27:06.233352 14 cacher.go:171] Terminating all watchers from cacher machineosconfigs.machineconfiguration.openshift.io W0308 03:27:06.233976 14 cacher.go:171] Terminating all watchers from cacher imagedigestmirrorsets.config.openshift.io W0308 03:27:06.234951 14 cacher.go:171] Terminating all watchers from cacher etcds.operator.openshift.io W0308 03:27:06.235718 14 cacher.go:171] Terminating all watchers from cacher metal3remediations.infrastructure.cluster.x-k8s.io W0308 03:27:06.236193 14 cacher.go:171] Terminating all watchers from cacher kubestorageversionmigrators.operator.openshift.io W0308 03:27:06.236756 14 cacher.go:171] Terminating all watchers from cacher consoleyamlsamples.console.openshift.io W0308 03:27:06.237128 14 cacher.go:171] Terminating all watchers from cacher dnses.operator.openshift.io W0308 03:27:06.237598 14 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io W0308 03:27:06.238145 14 cacher.go:171] Terminating all watchers from cacher imagecontentsourcepolicies.operator.openshift.io W0308 03:27:06.238872 14 cacher.go:171] Terminating all watchers from cacher consolelinks.console.openshift.io W0308 03:27:06.239343 14 cacher.go:171] Terminating all watchers from cacher machineconfigurations.operator.openshift.io W0308 03:27:06.239769 14 cacher.go:171] Terminating all watchers from cacher infrastructures.config.openshift.io W0308 03:27:06.240992 14 cacher.go:171] Terminating all watchers from cacher clusterresourcequotas.quota.openshift.io W0308 03:27:06.241408 14 cacher.go:171] Terminating all watchers from cacher clusterserviceversions.operators.coreos.com W0308 03:27:06.241791 14 cacher.go:171] Terminating all watchers from cacher networks.operator.openshift.io W0308 03:27:06.242646 14 cacher.go:171] Terminating all watchers from cacher performanceprofiles.performance.openshift.io W0308 03:27:06.243054 14 cacher.go:171] Terminating all watchers from cacher projects.config.openshift.io W0308 03:27:06.243520 14 cacher.go:171] Terminating all watchers from cacher hardwaredata.metal3.io W0308 03:27:06.243993 14 cacher.go:171] Terminating all watchers from cacher imagetagmirrorsets.config.openshift.io W0308 03:27:06.244644 14 cacher.go:171] Terminating all watchers from cacher networks.config.openshift.io W0308 03:27:06.245416 14 cacher.go:171] Terminating all watchers from cacher rolebindingrestrictions.authorization.openshift.io W0308 03:27:06.245818 14 cacher.go:171] Terminating all watchers from cacher dnses.config.openshift.io W0308 03:27:06.246203 14 cacher.go:171] Terminating all watchers from cacher schedulers.config.openshift.io W0308 03:27:06.246734 14 cacher.go:171] Terminating all watchers from cacher machineosbuilds.machineconfiguration.openshift.io W0308 03:27:06.247121 14 cacher.go:171] Terminating all watchers from cacher hostfirmwarecomponents.metal3.io W0308 03:27:06.247577 14 cacher.go:171] Terminating all watchers from cacher ingresses.config.openshift.io W0308 03:27:06.248463 14 cacher.go:171] Terminating all watchers from cacher egressrouters.network.operator.openshift.io W0308 03:27:06.249075 14 cacher.go:171] Terminating all watchers from cacher egressips.k8s.ovn.org W0308 03:27:06.250320 14 cacher.go:171] Terminating all watchers from cacher egressfirewalls.k8s.ovn.org W0308 03:27:06.251075 14 cacher.go:171] Terminating all watchers from cacher dnsrecords.ingress.operator.openshift.io W0308 03:27:06.251732 14 cacher.go:171] Terminating all watchers from cacher clusterautoscalers.autoscaling.openshift.io W0308 03:27:06.252354 14 cacher.go:171] Terminating all watchers from cacher ipaddressclaims.ipam.cluster.x-k8s.io W0308 03:27:06.252788 14 cacher.go:171] Terminating all watchers from cacher kubeschedulers.operator.openshift.io W0308 03:27:06.253223 14 cacher.go:171] Terminating all watchers from cacher builds.config.openshift.io W0308 03:27:06.253688 14 cacher.go:171] Terminating all watchers from cacher insightsoperators.operator.openshift.io W0308 03:27:06.254113 14 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io W0308 03:27:06.254757 14 cacher.go:171] Terminating all watchers from cacher configs.imageregistry.operator.openshift.io W0308 03:27:06.255226 14 cacher.go:171] Terminating all watchers from cacher consoleplugins.console.openshift.io W0308 03:27:06.255812 14 cacher.go:171] Terminating all watchers from cacher rangeallocations.security.internal.openshift.io W0308 03:27:06.256197 14 cacher.go:171] Terminating all watchers from cacher consolequickstarts.console.openshift.io W0308 03:27:06.256506 14 cacher.go:171] Terminating all watchers from cacher projecthelmchartrepositories.helm.openshift.io W0308 03:27:06.256778 14 cacher.go:171] Terminating all watchers from cacher openshiftcontrollermanagers.operator.openshift.io W0308 03:27:06.257261 14 cacher.go:171] Terminating all watchers from cacher ipamclaims.k8s.cni.cncf.io W0308 03:27:06.257757 14 cacher.go:171] Terminating all watchers from cacher consolesamples.console.openshift.io W0308 03:27:06.258172 14 cacher.go:171] Terminating all watchers from cacher apirequestcounts.apiserver.openshift.io W0308 03:27:06.258508 14 cacher.go:171] Terminating all watchers from cacher prometheuses.monitoring.coreos.com W0308 03:27:06.258806 14 cacher.go:171] Terminating all watchers from cacher tuneds.tuned.openshift.io W0308 03:27:06.259038 14 cacher.go:171] Terminating all watchers from cacher alertmanagers.monitoring.coreos.com W0308 03:27:06.259737 14 cacher.go:171] Terminating all watchers from cacher operatorgroups.operators.coreos.com W0308 03:27:06.260073 14 cacher.go:171] Terminating all watchers from cacher authentications.config.openshift.io W0308 03:27:06.261238 14 cacher.go:171] Terminating all watchers from cacher operatorconditions.operators.coreos.com W0308 03:27:06.261636 14 cacher.go:171] Terminating all watchers from cacher egressqoses.k8s.ovn.org W0308 03:27:06.261982 14 cacher.go:171] Terminating all watchers from cacher securitycontextconstraints.security.openshift.io W0308 03:27:06.262309 14 cacher.go:171] Terminating all watchers from cacher machinehealthchecks.machine.openshift.io W0308 03:27:06.262588 14 cacher.go:171] Terminating all watchers from cacher machinesets.machine.openshift.io W0308 03:27:06.263125 14 cacher.go:171] Terminating all watchers from cacher dataimages.metal3.io W0308 03:27:06.263565 14 cacher.go:171] Terminating all watchers from cacher images.config.openshift.io W0308 03:27:06.265574 14 cacher.go:171] Terminating all watchers from cacher hostupdatepolicies.metal3.io W0308 03:27:06.265926 14 cacher.go:171] Terminating all watchers from cacher olms.operator.openshift.io W0308 03:27:06.266528 14 cacher.go:171] Terminating all watchers from cacher thanosrulers.monitoring.coreos.com W0308 03:27:06.267019 14 cacher.go:171] Terminating all watchers from cacher catalogsources.operators.coreos.com I0308 03:27:06.296564 1 main.go:235] Termination finished with exit code 0 I0308 03:27:06.296823 1 main.go:188] Deleting termination lock file "/var/log/kube-apiserver/.terminating" ---