release-openshift-ocp-installer-e2e-gcp-serial-4.3 #274

• artifacts
• build log

Test Failures

openshift-tests Monitor cluster while tests execute 54m53s

go run hack/e2e.go -v -test --test_args='--ginkgo.focus=openshift\-tests\sMonitor\scluster\swhile\stests\sexecute$'

23 error level events were detected during this test run:

Nov 08 00:42:29.813 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available
Nov 08 00:42:38.376 E ns/openshift-monitoring pod/grafana-85bf74556f-xwbt6 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=grafana-proxy container exited with code 2 (Error): 
Nov 08 00:42:38.390 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/08 00:28:37 Watching directory: "/etc/alertmanager/config"\n
Nov 08 00:42:38.390 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/08 00:28:37 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:28:37 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/08 00:28:37 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/08 00:28:37 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/08 00:28:37 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/08 00:28:37 oauthproxy.go:227: OAuthProxy configured for  Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:28:37 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/08 00:28:37 http.go:96: HTTPS: listening on [::]:9095\n
Nov 08 00:42:58.334 E ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-b-l7d6t.c.openshift-gce-devel-ci.internal container=prometheus container exited with code 1 (Error): caller=main.go:657 msg="Starting TSDB ..."\nlevel=info ts=2019-11-08T00:42:53.285Z caller=web.go:450 component=web msg="Start listening for connections" address=127.0.0.1:9090\nlevel=info ts=2019-11-08T00:42:53.289Z caller=head.go:514 component=tsdb msg="replaying WAL, this may take awhile"\nlevel=info ts=2019-11-08T00:42:53.289Z caller=head.go:562 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:672 fs_type=XFS_SUPER_MAGIC\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:673 msg="TSDB started"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:743 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:526 msg="Stopping scrape discovery manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:540 msg="Stopping notify discovery manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:562 msg="Stopping scrape manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:556 msg="Scrape manager stopped"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=manager.go:814 component="rule manager" msg="Stopping rule manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=manager.go:820 component="rule manager" msg="Rule manager stopped"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:536 msg="Notify discovery manager stopped"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:522 msg="Scrape discovery manager stopped"\nlevel=info ts=2019-11-08T00:42:53.293Z caller=notifier.go:602 component=notifier msg="Stopping notification manager..."\nlevel=info ts=2019-11-08T00:42:53.293Z caller=main.go:727 msg="Notifier manager stopped"\nlevel=error ts=2019-11-08
Nov 08 00:50:41.528 E ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 00:50:42.633 E ns/openshift-monitoring pod/alertmanager-main-0 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/08 00:29:22 Watching directory: "/etc/alertmanager/config"\n
Nov 08 00:50:42.633 E ns/openshift-monitoring pod/alertmanager-main-0 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/08 00:29:23 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:29:23 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/08 00:29:23 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/08 00:29:23 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/08 00:29:23 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/08 00:29:23 oauthproxy.go:227: OAuthProxy configured for  Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:29:23 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/08 00:29:23 http.go:96: HTTPS: listening on [::]:9095\n
Nov 08 00:50:42.657 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 2 (Error): 2019/11/08 00:42:45 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:thanos-querier\n2019/11/08 00:42:45 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/08 00:42:45 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/08 00:42:45 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9090/"\n2019/11/08 00:42:45 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/08 00:42:45 oauthproxy.go:227: OAuthProxy configured for  Client ID: system:serviceaccount:openshift-monitoring:thanos-querier\n2019/11/08 00:42:45 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/08 00:42:45 main.go:154: using htpasswd file /etc/proxy/htpasswd/auth\n2019/11/08 00:42:45 http.go:96: HTTPS: listening on [::]:9091\n
Nov 08 00:50:42.687 E ns/openshift-ingress pod/router-default-56666d9db-k7b69 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=router container exited with code 2 (Error): Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:35.170Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:40.166Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:45.170Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:50.162Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:55.170Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:00.162Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:05.168Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:13.771Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:18.769Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:23.771Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:44:52.640Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n
Nov 08 00:51:00.698 E ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prometheus container exited with code 1 (Error): caller=main.go:657 msg="Starting TSDB ..."\nlevel=info ts=2019-11-08T00:50:58.590Z caller=web.go:450 component=web msg="Start listening for connections" address=127.0.0.1:9090\nlevel=info ts=2019-11-08T00:50:58.594Z caller=head.go:514 component=tsdb msg="replaying WAL, this may take awhile"\nlevel=info ts=2019-11-08T00:50:58.594Z caller=head.go:562 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:672 fs_type=XFS_SUPER_MAGIC\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:673 msg="TSDB started"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:743 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:526 msg="Stopping scrape discovery manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:540 msg="Stopping notify discovery manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:562 msg="Stopping scrape manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:536 msg="Notify discovery manager stopped"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:522 msg="Scrape discovery manager stopped"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:556 msg="Scrape manager stopped"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=manager.go:814 component="rule manager" msg="Stopping rule manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=manager.go:820 component="rule manager" msg="Rule manager stopped"\nlevel=info ts=2019-11-08T00:50:58.597Z caller=notifier.go:602 component=notifier msg="Stopping notification manager..."\nlevel=info ts=2019-11-08T00:50:58.597Z caller=main.go:727 msg="Notifier manager stopped"\nlevel=error ts=2019-11-08
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=thanos-querier container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=prom-label-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.323 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.323 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.323 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=alertmanager container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 01:05:48.404 E ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:09:50.731 E ns/openshift-sdn pod/sdn-vz9b4 node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal container=sdn container exited with code 255 (Error): se\nI1108 01:09:42.741181    5574 vnids.go:162] Dissociate netid 6677712 from namespace "e2e-pv-6055"\nI1108 01:09:43.435269    5574 vnids.go:148] Associate netid 3999694 to namespace "e2e-taint-single-pod-1557" with mcEnabled false\nI1108 01:09:47.932752    5574 roundrobin.go:270] LoadBalancerRR: Setting endpoints for openshift-authentication/oauth-openshift:https to [10.128.0.48:6443 10.129.0.39:6443 10.130.0.44:6443]\nI1108 01:09:47.932894    5574 roundrobin.go:218] Delete endpoint 10.128.0.48:6443 for service "openshift-authentication/oauth-openshift:https"\nI1108 01:09:47.932994    5574 proxy.go:334] hybrid proxy: syncProxyRules start\nI1108 01:09:47.995758    5574 roundrobin.go:270] LoadBalancerRR: Setting endpoints for openshift-authentication/oauth-openshift:https to [10.128.0.48:6443 10.130.0.44:6443]\nI1108 01:09:47.995924    5574 roundrobin.go:218] Delete endpoint 10.129.0.39:6443 for service "openshift-authentication/oauth-openshift:https"\nI1108 01:09:48.191204    5574 proxy.go:337] hybrid proxy: mainProxy.syncProxyRules complete\nI1108 01:09:48.302814    5574 proxier.go:371] userspace proxy: processing 0 service events\nI1108 01:09:48.302915    5574 proxier.go:350] userspace syncProxyRules took 111.571773ms\nI1108 01:09:48.302935    5574 proxy.go:340] hybrid proxy: unidlingProxy.syncProxyRules complete\nI1108 01:09:48.302955    5574 proxy.go:334] hybrid proxy: syncProxyRules start\nI1108 01:09:48.581459    5574 proxy.go:337] hybrid proxy: mainProxy.syncProxyRules complete\nI1108 01:09:48.679497    5574 proxier.go:371] userspace proxy: processing 0 service events\nI1108 01:09:48.679528    5574 proxier.go:350] userspace syncProxyRules took 98.037241ms\nI1108 01:09:48.679539    5574 proxy.go:340] hybrid proxy: unidlingProxy.syncProxyRules complete\nI1108 01:09:49.810096    5574 ovs.go:169] Error executing ovs-ofctl: ovs-ofctl: /var/run/openvswitch/br0.mgmt: failed to open socket (Broken pipe)\nF1108 01:09:49.810246    5574 healthcheck.go:99] SDN healthcheck detected unhealthy OVS server, restarting: plugin is not setup\n
Nov 08 01:10:45.074 E ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:17:48.220 E ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:20:12.598 E ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 

				

Timeline:

Nov 08 00:35:13.048 I ns/openshift-kube-apiserver pod/revision-pruner-6-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal node/ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal created
Nov 08 00:35:19.966 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s
Nov 08 00:35:20.713 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s
Nov 08 00:35:22.924 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s
Nov 08 00:35:25.222 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg requeue in: 20s
Nov 08 00:35:25.956 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv
Nov 08 00:35:26.535 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s
Nov 08 00:35:27.223 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg requeue in: 20s
Nov 08 00:35:27.958 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv (2 times)
Nov 08 00:35:28.519 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6
Nov 08 00:35:29.174 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg requeue in: 20s (2 times)
Nov 08 00:35:29.825 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv (3 times)
Nov 08 00:35:30.418 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6 (2 times)
Nov 08 00:35:31.060 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg
Nov 08 00:35:31.654 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6 (3 times)
Nov 08 00:35:32.341 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg (2 times)
Nov 08 00:35:32.988 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg (3 times)
Nov 08 00:35:40.664 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv (4 times)
Nov 08 00:35:43.580 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6 (4 times)
Nov 08 00:35:45.838 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg (4 times)
Nov 08 00:40:50.098 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (3 times)
Nov 08 00:40:50.108 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (2 times)
Nov 08 00:40:50.291 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-001" not found) (3 times)
Nov 08 00:40:50.693 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-002" not found) (6 times)
Nov 08 00:40:51.094 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-003" not found) (7 times)
Nov 08 00:41:42.828 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal status is now: NodeHasSufficientMemory (8 times)
Nov 08 00:41:42.847 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal condition NetworkUnavailable changed
Nov 08 00:41:42.855 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal status is now: NodeHasNoDiskPressure (8 times)
Nov 08 00:41:42.868 I ns/openshift-sdn pod/ovs-n8qdt node/ created
Nov 08 00:41:42.891 I ns/openshift-sdn daemonset/ovs Created pod: ovs-n8qdt
Nov 08 00:41:42.903 I ns/openshift-sdn pod/ovs-n8qdt Successfully assigned openshift-sdn/ovs-n8qdt to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:41:42.911 W clusteroperator/network changed Progressing to True: Deploying: DaemonSet "openshift-sdn/ovs" update is rolling out (6 out of 7 updated)
Nov 08 00:41:42.920 I ns/openshift-monitoring pod/node-exporter-hx94v node/ created
Nov 08 00:41:42.923 - 30s   W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal node is not ready
Nov 08 00:41:42.925 I ns/openshift-sdn pod/sdn-k6rl2 node/ created
Nov 08 00:41:42.944 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-hx94v
Nov 08 00:41:42.953 I ns/openshift-sdn pod/sdn-k6rl2 Successfully assigned openshift-sdn/sdn-k6rl2 to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:41:42.959 I ns/openshift-monitoring pod/node-exporter-hx94v Successfully assigned openshift-monitoring/node-exporter-hx94v to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:41:42.993 I ns/openshift-sdn daemonset/sdn Created pod: sdn-k6rl2
Nov 08 00:41:43.017 I ns/openshift-multus pod/multus-p9kk9 node/ created
Nov 08 00:41:43.030 I ns/openshift-multus daemonset/multus Created pod: multus-p9kk9
Nov 08 00:41:43.040 I ns/openshift-multus pod/multus-p9kk9 Successfully assigned openshift-multus/multus-p9kk9 to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:41:43.052 I ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr node/ created
Nov 08 00:41:43.072 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-dbhmr
Nov 08 00:41:43.095 I ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr Successfully assigned openshift-cluster-node-tuning-operator/tuned-dbhmr to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:41:43.127 W clusteroperator/node-tuning changed Progressing to True: Reconciling: Working towards "4.3.0-0.nightly-2019-11-07-235426"
Nov 08 00:41:43.260 W clusteroperator/node-tuning changed Progressing to False: AsExpected: Cluster version is "4.3.0-0.nightly-2019-11-07-235426"
Nov 08 00:41:43.314 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal event: Registered Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal in Controller
Nov 08 00:41:43.554 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6 (5 times)
Nov 08 00:41:44.159 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6 (6 times)
Nov 08 00:41:49.345 I node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal status is now: NodeHasSufficientMemory (8 times)
Nov 08 00:41:49.352 W node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal condition NetworkUnavailable changed
Nov 08 00:41:49.358 I ns/openshift-sdn pod/ovs-pvj25 node/ created
Nov 08 00:41:49.360 I node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal status is now: NodeHasNoDiskPressure (8 times)
Nov 08 00:41:49.368 I ns/openshift-sdn pod/sdn-zr25c node/ created
Nov 08 00:41:49.372 I ns/openshift-sdn daemonset/ovs Created pod: ovs-pvj25
Nov 08 00:41:49.398 I ns/openshift-sdn pod/ovs-pvj25 Successfully assigned openshift-sdn/ovs-pvj25 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:41:49.401 I ns/openshift-sdn daemonset/sdn Created pod: sdn-zr25c
Nov 08 00:41:49.422 I ns/openshift-sdn pod/sdn-zr25c Successfully assigned openshift-sdn/sdn-zr25c to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:41:49.442 I ns/openshift-multus pod/multus-lhh6s node/ created
Nov 08 00:41:49.463 I ns/openshift-multus pod/multus-lhh6s Successfully assigned openshift-multus/multus-lhh6s to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:41:49.473 I ns/openshift-multus daemonset/multus Created pod: multus-lhh6s
Nov 08 00:41:49.494 I ns/openshift-monitoring pod/node-exporter-zjk52 node/ created
Nov 08 00:41:49.519 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-zjk52
Nov 08 00:41:49.520 I ns/openshift-monitoring pod/node-exporter-zjk52 Successfully assigned openshift-monitoring/node-exporter-zjk52 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:41:49.540 I ns/openshift-cluster-node-tuning-operator pod/tuned-td2jr node/ created
Nov 08 00:41:49.555 I ns/openshift-cluster-node-tuning-operator pod/tuned-td2jr Successfully assigned openshift-cluster-node-tuning-operator/tuned-td2jr to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:41:49.561 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-td2jr
Nov 08 00:41:49.586 W clusteroperator/node-tuning changed Progressing to True: Reconciling: Working towards "4.3.0-0.nightly-2019-11-07-235426"
Nov 08 00:41:49.708 W clusteroperator/node-tuning changed Progressing to False: AsExpected: Cluster version is "4.3.0-0.nightly-2019-11-07-235426"
Nov 08 00:41:50.023 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg (5 times)
Nov 08 00:41:50.658 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg (6 times)
Nov 08 00:41:52.558 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal condition NetworkUnavailable changed
Nov 08 00:41:52.563 I ns/openshift-sdn pod/ovs-nwxn4 node/ created
Nov 08 00:41:52.566 I ns/openshift-sdn pod/sdn-kvf4x node/ created
Nov 08 00:41:52.600 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal status is now: NodeHasSufficientMemory (8 times)
Nov 08 00:41:52.604 I ns/openshift-sdn daemonset/ovs Created pod: ovs-nwxn4
Nov 08 00:41:52.604 I ns/openshift-sdn pod/sdn-kvf4x Successfully assigned openshift-sdn/sdn-kvf4x to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:41:52.606 I ns/openshift-sdn pod/ovs-nwxn4 Successfully assigned openshift-sdn/ovs-nwxn4 to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:41:52.625 I ns/openshift-sdn daemonset/sdn Created pod: sdn-kvf4x
Nov 08 00:41:52.644 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal status is now: NodeHasNoDiskPressure (8 times)
Nov 08 00:41:52.655 I ns/openshift-multus pod/multus-fvh4s node/ created
Nov 08 00:41:52.660 I ns/openshift-multus daemonset/multus Created pod: multus-fvh4s
Nov 08 00:41:52.673 I ns/openshift-multus pod/multus-fvh4s Successfully assigned openshift-multus/multus-fvh4s to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:41:52.719 I ns/openshift-monitoring pod/node-exporter-h2k78 node/ created
Nov 08 00:41:52.726 I ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w node/ created
Nov 08 00:41:52.745 I ns/openshift-monitoring pod/node-exporter-h2k78 Successfully assigned openshift-monitoring/node-exporter-h2k78 to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:41:52.745 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-h2k78
Nov 08 00:41:52.747 I ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w Successfully assigned openshift-cluster-node-tuning-operator/tuned-ql94w to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:41:52.772 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-ql94w
Nov 08 00:41:53.318 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal event: Registered Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal in Controller
Nov 08 00:41:53.329 I node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal event: Registered Node ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal in Controller
Nov 08 00:41:53.340 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv (5 times)
Nov 08 00:41:54.079 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv (6 times)
Nov 08 00:41:56.830 I ns/openshift-sdn pod/ovs-n8qdt Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce"
Nov 08 00:41:56.875 I ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3207589b466325f1ee9decd9f41416ca20597d409d6071c3614ce2f5e9626368"
Nov 08 00:41:56.880 I ns/openshift-monitoring pod/node-exporter-hx94v Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d5edb31f42f0b66d3f9848af5a65fa82c12014aecdc916b846d0b40a2d46e0a"
Nov 08 00:41:56.949 I ns/openshift-sdn pod/sdn-k6rl2 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:41:57.009 I ns/openshift-multus pod/multus-p9kk9 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:41:57.923 - 29s   W node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal node is not ready
Nov 08 00:41:57.923 - 89s   W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal node is not ready
Nov 08 00:42:00.896 I ns/openshift-monitoring pod/node-exporter-hx94v Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d5edb31f42f0b66d3f9848af5a65fa82c12014aecdc916b846d0b40a2d46e0a"
Nov 08 00:42:01.146 I ns/openshift-monitoring pod/node-exporter-hx94v Created container node-exporter
Nov 08 00:42:01.265 I ns/openshift-monitoring pod/node-exporter-hx94v Started container node-exporter
Nov 08 00:42:01.310 I ns/openshift-monitoring pod/node-exporter-hx94v Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8"
Nov 08 00:42:04.574 I ns/openshift-multus pod/multus-lhh6s Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:04.581 I ns/openshift-monitoring pod/node-exporter-zjk52 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d5edb31f42f0b66d3f9848af5a65fa82c12014aecdc916b846d0b40a2d46e0a"
Nov 08 00:42:04.673 I ns/openshift-sdn pod/ovs-pvj25 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce"
Nov 08 00:42:04.777 I ns/openshift-cluster-node-tuning-operator pod/tuned-td2jr Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3207589b466325f1ee9decd9f41416ca20597d409d6071c3614ce2f5e9626368"
Nov 08 00:42:04.794 I ns/openshift-sdn pod/sdn-zr25c Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:06.077 I ns/openshift-sdn pod/ovs-nwxn4 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce"
Nov 08 00:42:06.085 I ns/openshift-sdn pod/sdn-kvf4x Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:06.091 I ns/openshift-multus pod/multus-fvh4s Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:06.103 I ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3207589b466325f1ee9decd9f41416ca20597d409d6071c3614ce2f5e9626368"
Nov 08 00:42:06.189 I ns/openshift-monitoring pod/node-exporter-h2k78 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d5edb31f42f0b66d3f9848af5a65fa82c12014aecdc916b846d0b40a2d46e0a"
Nov 08 00:42:08.692 I ns/openshift-monitoring pod/node-exporter-zjk52 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d5edb31f42f0b66d3f9848af5a65fa82c12014aecdc916b846d0b40a2d46e0a"
Nov 08 00:42:08.925 I ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3207589b466325f1ee9decd9f41416ca20597d409d6071c3614ce2f5e9626368"
Nov 08 00:42:08.935 I ns/openshift-sdn pod/ovs-n8qdt Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce"
Nov 08 00:42:08.953 I ns/openshift-monitoring pod/node-exporter-hx94v Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8"
Nov 08 00:42:08.980 I ns/openshift-sdn pod/sdn-k6rl2 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:09.011 I ns/openshift-multus pod/multus-p9kk9 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:09.176 I ns/openshift-sdn pod/ovs-n8qdt Created container openvswitch
Nov 08 00:42:09.253 I ns/openshift-monitoring pod/node-exporter-hx94v Created container kube-rbac-proxy
Nov 08 00:42:09.261 I ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr Created container tuned
Nov 08 00:42:09.267 I ns/openshift-sdn pod/ovs-n8qdt Started container openvswitch
Nov 08 00:42:09.320 I ns/openshift-multus pod/multus-p9kk9 Created container cni-plugins
Nov 08 00:42:09.325 I ns/openshift-sdn pod/sdn-k6rl2 Created container install-cni-plugins
Nov 08 00:42:09.331 I ns/openshift-monitoring pod/node-exporter-hx94v Started container kube-rbac-proxy
Nov 08 00:42:09.525 I ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr Started container tuned
Nov 08 00:42:09.554 I ns/openshift-monitoring pod/node-exporter-zjk52 Created container node-exporter
Nov 08 00:42:09.722 I ns/openshift-sdn pod/sdn-k6rl2 Started container install-cni-plugins
Nov 08 00:42:09.764 I ns/openshift-monitoring pod/node-exporter-zjk52 Started container node-exporter
Nov 08 00:42:09.794 I ns/openshift-monitoring pod/node-exporter-zjk52 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8"
Nov 08 00:42:09.811 I ns/openshift-monitoring pod/node-exporter-h2k78 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d5edb31f42f0b66d3f9848af5a65fa82c12014aecdc916b846d0b40a2d46e0a"
Nov 08 00:42:09.923 I ns/openshift-multus pod/multus-p9kk9 Started container cni-plugins
Nov 08 00:42:10.080 I ns/openshift-monitoring pod/node-exporter-h2k78 Created container node-exporter
Nov 08 00:42:10.132 I ns/openshift-multus pod/multus-p9kk9 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d96246be6464e56863cc2b85ad14586e5034d0ee39aced7c90adab6f46e260f"
Nov 08 00:42:10.260 I ns/openshift-monitoring pod/node-exporter-h2k78 Started container node-exporter
Nov 08 00:42:10.268 I ns/openshift-monitoring pod/node-exporter-h2k78 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8"
Nov 08 00:42:10.323 I ns/openshift-sdn pod/sdn-k6rl2 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce" already present on machine
Nov 08 00:42:10.523 I ns/openshift-sdn pod/sdn-k6rl2 Created container sdn
Nov 08 00:42:10.722 I ns/openshift-sdn pod/sdn-k6rl2 Started container sdn
Nov 08 00:42:11.491 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking.
Nov 08 00:42:12.843 I ns/openshift-multus pod/multus-p9kk9 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d96246be6464e56863cc2b85ad14586e5034d0ee39aced7c90adab6f46e260f"
Nov 08 00:42:12.982 I ns/openshift-multus pod/multus-p9kk9 Created container kube-multus
Nov 08 00:42:13.011 I ns/openshift-multus pod/multus-p9kk9 Started container kube-multus
Nov 08 00:42:16.822 I ns/openshift-cluster-node-tuning-operator pod/tuned-td2jr Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3207589b466325f1ee9decd9f41416ca20597d409d6071c3614ce2f5e9626368"
Nov 08 00:42:16.868 I ns/openshift-sdn pod/ovs-pvj25 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce"
Nov 08 00:42:16.877 I ns/openshift-sdn pod/sdn-zr25c Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:16.939 I ns/openshift-multus pod/multus-lhh6s Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:17.167 I ns/openshift-sdn pod/ovs-pvj25 Created container openvswitch
Nov 08 00:42:17.231 I ns/openshift-cluster-node-tuning-operator pod/tuned-td2jr Created container tuned
Nov 08 00:42:17.257 I ns/openshift-sdn pod/ovs-pvj25 Started container openvswitch
Nov 08 00:42:17.278 I ns/openshift-sdn pod/sdn-zr25c Created container install-cni-plugins
Nov 08 00:42:17.306 I ns/openshift-cluster-node-tuning-operator pod/tuned-td2jr Started container tuned
Nov 08 00:42:17.309 I ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3207589b466325f1ee9decd9f41416ca20597d409d6071c3614ce2f5e9626368"
Nov 08 00:42:17.322 I ns/openshift-multus pod/multus-lhh6s Created container cni-plugins
Nov 08 00:42:17.332 I ns/openshift-sdn pod/ovs-nwxn4 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce"
Nov 08 00:42:17.341 I ns/openshift-sdn pod/sdn-kvf4x Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:17.347 I ns/openshift-sdn pod/sdn-zr25c Started container install-cni-plugins
Nov 08 00:42:17.381 I ns/openshift-multus pod/multus-fvh4s Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b308839b51f004db51879b9ea80789b6577e906e70f895a245f893bd39fb5a58"
Nov 08 00:42:17.390 I ns/openshift-multus pod/multus-lhh6s Started container cni-plugins
Nov 08 00:42:17.570 I ns/openshift-sdn pod/ovs-nwxn4 Created container openvswitch
Nov 08 00:42:17.582 I ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w Created container tuned
Nov 08 00:42:17.623 I ns/openshift-sdn pod/sdn-kvf4x Created container install-cni-plugins
Nov 08 00:42:17.629 I ns/openshift-sdn pod/ovs-nwxn4 Started container openvswitch
Nov 08 00:42:17.643 I ns/openshift-multus pod/multus-fvh4s Created container cni-plugins
Nov 08 00:42:17.649 I ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w Started container tuned
Nov 08 00:42:17.682 I ns/openshift-sdn pod/sdn-kvf4x Started container install-cni-plugins
Nov 08 00:42:17.710 I ns/openshift-multus pod/multus-fvh4s Started container cni-plugins
Nov 08 00:42:17.881 I ns/openshift-multus pod/multus-lhh6s Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d96246be6464e56863cc2b85ad14586e5034d0ee39aced7c90adab6f46e260f"
Nov 08 00:42:18.013 I ns/openshift-sdn pod/sdn-zr25c Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce" already present on machine
Nov 08 00:42:18.074 I ns/openshift-monitoring pod/node-exporter-zjk52 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8"
Nov 08 00:42:18.306 I ns/openshift-sdn pod/sdn-zr25c Created container sdn
Nov 08 00:42:18.327 I ns/openshift-monitoring pod/node-exporter-zjk52 Created container kube-rbac-proxy
Nov 08 00:42:18.417 I ns/openshift-monitoring pod/node-exporter-h2k78 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8"
Nov 08 00:42:18.425 I ns/openshift-sdn pod/sdn-zr25c Started container sdn
Nov 08 00:42:18.475 I ns/openshift-multus pod/multus-fvh4s Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d96246be6464e56863cc2b85ad14586e5034d0ee39aced7c90adab6f46e260f"
Nov 08 00:42:18.488 I ns/openshift-sdn pod/sdn-kvf4x Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce" already present on machine
Nov 08 00:42:18.572 I ns/openshift-monitoring pod/node-exporter-h2k78 Created container kube-rbac-proxy
Nov 08 00:42:18.621 I ns/openshift-monitoring pod/node-exporter-zjk52 Started container kube-rbac-proxy
Nov 08 00:42:18.710 I ns/openshift-monitoring pod/node-exporter-h2k78 Started container kube-rbac-proxy
Nov 08 00:42:18.909 I ns/openshift-sdn pod/sdn-kvf4x Created container sdn
Nov 08 00:42:19.111 I ns/openshift-sdn pod/sdn-kvf4x Started container sdn
Nov 08 00:42:19.687 I node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking.
Nov 08 00:42:19.776 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking.
Nov 08 00:42:21.263 I ns/openshift-multus pod/multus-fvh4s Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d96246be6464e56863cc2b85ad14586e5034d0ee39aced7c90adab6f46e260f"
Nov 08 00:42:21.398 I ns/openshift-multus pod/multus-fvh4s Created container kube-multus
Nov 08 00:42:21.424 I ns/openshift-multus pod/multus-fvh4s Started container kube-multus
Nov 08 00:42:21.824 I ns/openshift-multus pod/multus-lhh6s Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d96246be6464e56863cc2b85ad14586e5034d0ee39aced7c90adab6f46e260f"
Nov 08 00:42:22.006 I ns/openshift-multus pod/multus-lhh6s Created container kube-multus
Nov 08 00:42:22.045 I ns/openshift-multus pod/multus-lhh6s Started container kube-multus
Nov 08 00:42:22.889 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 00:42:22.925 I ns/openshift-dns pod/dns-default-wz2js node/ created
Nov 08 00:42:22.937 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt node/ created
Nov 08 00:42:22.941 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-wz2js
Nov 08 00:42:22.951 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-rx7xt
Nov 08 00:42:22.952 I ns/openshift-dns pod/dns-default-wz2js Successfully assigned openshift-dns/dns-default-wz2js to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:42:22.955 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Successfully assigned openshift-machine-config-operator/machine-config-daemon-rx7xt to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:42:22.994 I ns/openshift-image-registry pod/node-ca-nzhnw node/ created
Nov 08 00:42:23.008 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-nzhnw
Nov 08 00:42:23.011 I ns/openshift-image-registry pod/node-ca-nzhnw Successfully assigned openshift-image-registry/node-ca-nzhnw to ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal
Nov 08 00:42:23.081 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 00:42:23.470 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Updated Machine ci-op--bqfmn-w-c-jdgh6 (7 times)
Nov 08 00:42:23.724 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6"
Nov 08 00:42:29.641 W node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 00:42:29.669 I ns/openshift-dns pod/dns-default-mwcnm node/ created
Nov 08 00:42:29.674 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ created
Nov 08 00:42:29.688 I ns/openshift-dns pod/dns-default-mwcnm Successfully assigned openshift-dns/dns-default-mwcnm to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:42:29.690 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-mwcnm
Nov 08 00:42:29.704 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-64z97
Nov 08 00:42:29.706 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Successfully assigned openshift-machine-config-operator/machine-config-daemon-64z97 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:42:29.719 I ns/openshift-image-registry pod/node-ca-k7jtb node/ created
Nov 08 00:42:29.726 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-k7jtb
Nov 08 00:42:29.744 I ns/openshift-image-registry pod/node-ca-k7jtb Successfully assigned openshift-image-registry/node-ca-k7jtb to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:42:29.813 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available
Nov 08 00:42:30.323 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6"
Nov 08 00:42:30.332 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-zzcqg Updated Machine ci-op--bqfmn-w-d-zzcqg (7 times)
Nov 08 00:42:30.453 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Created container machine-config-daemon
Nov 08 00:42:30.486 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Started container machine-config-daemon
Nov 08 00:42:30.491 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945"
Nov 08 00:42:30.661 I ns/openshift-dns pod/dns-default-wz2js Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2"
Nov 08 00:42:30.780 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6"
Nov 08 00:42:30.994 I ns/openshift-image-registry pod/node-ca-nzhnw Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee"
Nov 08 00:42:32.688 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 00:42:32.721 I ns/openshift-dns pod/dns-default-s9xrl node/ created
Nov 08 00:42:32.721 I ns/openshift-machine-config-operator pod/machine-config-daemon-8fbr7 node/ created
Nov 08 00:42:32.742 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-8fbr7
Nov 08 00:42:32.743 I ns/openshift-machine-config-operator pod/machine-config-daemon-8fbr7 Successfully assigned openshift-machine-config-operator/machine-config-daemon-8fbr7 to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:42:32.743 I ns/openshift-dns pod/dns-default-s9xrl Successfully assigned openshift-dns/dns-default-s9xrl to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:42:32.759 W ns/openshift-dns endpoints/dns-default Failed to update endpoint openshift-dns/dns-default: Operation cannot be fulfilled on endpoints "dns-default": the object has been modified; please apply your changes to the latest version and try again
Nov 08 00:42:32.760 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-s9xrl
Nov 08 00:42:32.793 I ns/openshift-image-registry pod/node-ca-79xqh node/ created
Nov 08 00:42:32.802 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-79xqh
Nov 08 00:42:32.810 I ns/openshift-image-registry pod/node-ca-79xqh Successfully assigned openshift-image-registry/node-ca-79xqh to ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal
Nov 08 00:42:33.398 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Updated Machine ci-op--bqfmn-w-b-j6wwv (7 times)
Nov 08 00:42:33.496 I ns/openshift-machine-config-operator pod/machine-config-daemon-8fbr7 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6"
Nov 08 00:42:33.950 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Setting node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal, currentConfig rendered-worker-902ed86a0e00407131e632194fec4daa to Done
Nov 08 00:42:34.293 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945"
Nov 08 00:42:34.297 I ns/openshift-dns pod/dns-default-wz2js Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2"
Nov 08 00:42:34.356 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Node "ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal" drained
Nov 08 00:42:34.500 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Created container oauth-proxy
Nov 08 00:42:34.526 I ns/openshift-dns pod/dns-default-wz2js Created container dns
Nov 08 00:42:34.564 I ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt Started container oauth-proxy
Nov 08 00:42:34.586 I ns/openshift-dns pod/dns-default-wz2js Started container dns
Nov 08 00:42:34.595 I ns/openshift-dns pod/dns-default-wz2js Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29"
Nov 08 00:42:35.533 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s
Nov 08 00:42:35.613 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Node "ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal" drained
Nov 08 00:42:36.536 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s
Nov 08 00:42:36.635 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 120s
Nov 08 00:42:36.677 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:42:36.694 W ns/openshift-monitoring pod/grafana-85bf74556f-xwbt6 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:42:36.725 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 00:42:36.786 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy
Nov 08 00:42:36.847 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager
Nov 08 00:42:36.931 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader
Nov 08 00:42:36.954 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx Stopping container kube-rbac-proxy
Nov 08 00:42:36.995 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx Stopping container thanos-querier
Nov 08 00:42:37.037 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 node/ created
Nov 08 00:42:37.049 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx Stopping container prom-label-proxy
Nov 08 00:42:37.066 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Successfully assigned openshift-monitoring/grafana-85bf74556f-dzjk9 to ci-op--bqfmn-w-b-l7d6t.c.openshift-gce-devel-ci.internal
Nov 08 00:42:37.087 I ns/openshift-monitoring replicaset/grafana-85bf74556f Created pod: grafana-85bf74556f-dzjk9
Nov 08 00:42:37.107 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx Stopping container oauth-proxy
Nov 08 00:42:37.152 I ns/openshift-monitoring pod/grafana-85bf74556f-xwbt6 Stopping container grafana-proxy
Nov 08 00:42:37.185 I ns/openshift-monitoring pod/grafana-85bf74556f-xwbt6 Stopping container grafana
Nov 08 00:42:37.194 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 node/ created
Nov 08 00:42:37.202 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Successfully assigned openshift-monitoring/thanos-querier-5ddfc789fd-ldxk4 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:42:37.217 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prometheus
Nov 08 00:42:37.240 I ns/openshift-monitoring replicaset/thanos-querier-5ddfc789fd Created pod: thanos-querier-5ddfc789fd-ldxk4
Nov 08 00:42:37.240 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prom-label-proxy
Nov 08 00:42:37.896 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6"
Nov 08 00:42:38.057 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Created container machine-config-daemon
Nov 08 00:42:38.092 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Started container machine-config-daemon
Nov 08 00:42:38.098 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945"
Nov 08 00:42:38.246 I ns/openshift-image-registry pod/node-ca-k7jtb Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee"
Nov 08 00:42:38.325 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:42:38.325 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready
Nov 08 00:42:38.325 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready
Nov 08 00:42:38.325 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=thanos-querier container stopped being ready
Nov 08 00:42:38.325 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=thanos-sidecar container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=prometheus container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=prometheus-proxy container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=rules-configmap-reloader container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=prometheus-config-reloader container stopped being ready
Nov 08 00:42:38.343 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready
Nov 08 00:42:38.376 E ns/openshift-monitoring pod/grafana-85bf74556f-xwbt6 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=grafana-proxy container exited with code 2 (Error): 
Nov 08 00:42:38.390 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/08 00:28:37 Watching directory: "/etc/alertmanager/config"\n
Nov 08 00:42:38.390 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/08 00:28:37 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:28:37 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/08 00:28:37 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/08 00:28:37 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/08 00:28:37 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/08 00:28:37 oauthproxy.go:227: OAuthProxy configured for  Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:28:37 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/08 00:28:37 http.go:96: HTTPS: listening on [::]:9095\n
Nov 08 00:42:38.500 I ns/openshift-dns pod/dns-default-mwcnm Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2"
Nov 08 00:42:39.347 W ns/openshift-monitoring pod/grafana-85bf74556f-xwbt6 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:42:39.383 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-zpgdx node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:42:39.420 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:42:39.453 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:42:39.478 I ns/openshift-monitoring pod/alertmanager-main-2 node/ created
Nov 08 00:42:39.488 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-2 in StatefulSet alertmanager-main successful (2 times)
Nov 08 00:42:39.500 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully assigned openshift-monitoring/alertmanager-main-2 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:42:39.561 I ns/openshift-monitoring pod/prometheus-k8s-0 node/ created
Nov 08 00:42:39.581 I ns/openshift-monitoring statefulset/prometheus-k8s create Pod prometheus-k8s-0 in StatefulSet prometheus-k8s successful (2 times)
Nov 08 00:42:39.582 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully assigned openshift-monitoring/prometheus-k8s-0 to ci-op--bqfmn-w-b-l7d6t.c.openshift-gce-devel-ci.internal
Nov 08 00:42:39.816 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 Node "ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal" drained
Nov 08 00:42:40.734 I node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal Setting node ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal, currentConfig rendered-worker-902ed86a0e00407131e632194fec4daa to Done
Nov 08 00:42:40.901 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 requeue in: 20s
Nov 08 00:42:40.958 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Node "ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal" drained (2 times)
Nov 08 00:42:41.842 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s (2 times)
Nov 08 00:42:41.898 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Node "ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal" drained (2 times)
Nov 08 00:42:42.596 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s (2 times)
Nov 08 00:42:42.650 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 Node "ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal" drained (2 times)
Nov 08 00:42:43.373 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 requeue in: 20s (2 times)
Nov 08 00:42:43.880 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945"
Nov 08 00:42:43.887 I ns/openshift-dns pod/dns-default-mwcnm Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2"
Nov 08 00:42:44.180 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Created container oauth-proxy
Nov 08 00:42:44.239 I ns/openshift-dns pod/dns-default-mwcnm Created container dns
Nov 08 00:42:44.267 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Started container oauth-proxy
Nov 08 00:42:44.320 I ns/openshift-dns pod/dns-default-mwcnm Started container dns
Nov 08 00:42:44.326 I ns/openshift-dns pod/dns-default-mwcnm Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29"
Nov 08 00:42:44.811 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9bff84c0b9b83894b9340f11a98428b8d561dc5c0d28ab8661c6032cc4c08705" already present on machine
Nov 08 00:42:45.057 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Created container grafana
Nov 08 00:42:45.098 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Started container grafana
Nov 08 00:42:45.105 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:42:45.287 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf187f44c7273f1328d614d1dcd9f701c5f66cf8849143e34c8d1f88d5e2ce55" already present on machine
Nov 08 00:42:45.305 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Created container grafana-proxy
Nov 08 00:42:45.337 I ns/openshift-monitoring pod/grafana-85bf74556f-dzjk9 Started container grafana-proxy
Nov 08 00:42:45.477 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Created container thanos-querier
Nov 08 00:42:45.510 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Started container thanos-querier
Nov 08 00:42:45.516 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:42:45.706 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Created container oauth-proxy
Nov 08 00:42:45.749 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Started container oauth-proxy
Nov 08 00:42:45.756 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8" already present on machine
Nov 08 00:42:45.952 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Created container kube-rbac-proxy
Nov 08 00:42:45.986 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Started container kube-rbac-proxy
Nov 08 00:42:45.995 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8aefd089984c2e7d9dd7a162bd8e23bb9e492b494f6c285c2a8272f8230ff9e" already present on machine
Nov 08 00:42:46.165 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Created container prom-label-proxy
Nov 08 00:42:46.192 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Started container prom-label-proxy
Nov 08 00:42:46.434 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (3 times)
Nov 08 00:42:46.440 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (5 times)
Nov 08 00:42:46.447 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-001" not found) (3 times)
Nov 08 00:42:46.832 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-002" not found) (3 times)
Nov 08 00:42:47.233 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-003" not found) (3 times)
Nov 08 00:42:47.580 I ns/openshift-monitoring pod/alertmanager-main-2 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:78ba454f09557364dabd918265784493287ed242fb51ac2ffde695ae78823736"
Nov 08 00:42:47.685 I ns/openshift-monitoring pod/prometheus-k8s-0 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce5cf60d260dd8be75a016c30c7724b254f607573b9388da266d78fb3ac49757"
Nov 08 00:42:49.402 I ns/openshift-dns pod/dns-default-mwcnm Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29"
Nov 08 00:42:49.635 I ns/openshift-dns pod/dns-default-mwcnm Created container dns-node-resolver
Nov 08 00:42:49.706 I ns/openshift-dns pod/dns-default-mwcnm Started container dns-node-resolver
Nov 08 00:42:52.435 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:78ba454f09557364dabd918265784493287ed242fb51ac2ffde695ae78823736"
Nov 08 00:42:52.695 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager
Nov 08 00:42:52.741 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager
Nov 08 00:42:52.750 I ns/openshift-monitoring pod/alertmanager-main-2 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db10cd1f5efd9e9e6f2b0eea0a665f53399e25de374cf916ad7d8141f8ac643d"
Nov 08 00:42:53.047 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce5cf60d260dd8be75a016c30c7724b254f607573b9388da266d78fb3ac49757"
Nov 08 00:42:53.225 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus
Nov 08 00:42:53.258 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus
Nov 08 00:42:53.265 I ns/openshift-monitoring pod/prometheus-k8s-0 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9a3a81674e3aa3d54fc056ba4907a2f48b867c375d8fe86e05d9e1c99489eade"
Nov 08 00:42:55.320 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db10cd1f5efd9e9e6f2b0eea0a665f53399e25de374cf916ad7d8141f8ac643d"
Nov 08 00:42:55.610 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Node "ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal" drained (3 times)
Nov 08 00:42:55.636 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9a3a81674e3aa3d54fc056ba4907a2f48b867c375d8fe86e05d9e1c99489eade"
Nov 08 00:42:55.840 I ns/openshift-monitoring pod/alertmanager-main-2 Created container config-reloader
Nov 08 00:42:55.895 I ns/openshift-monitoring pod/alertmanager-main-2 Started container config-reloader
Nov 08 00:42:55.901 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:42:56.027 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus-config-reloader
Nov 08 00:42:56.057 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus-config-reloader
Nov 08 00:42:56.064 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db10cd1f5efd9e9e6f2b0eea0a665f53399e25de374cf916ad7d8141f8ac643d" already present on machine
Nov 08 00:42:56.140 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager-proxy
Nov 08 00:42:56.181 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager-proxy
Nov 08 00:42:56.424 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container rules-configmap-reloader
Nov 08 00:42:56.453 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s (3 times)
Nov 08 00:42:56.454 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container rules-configmap-reloader
Nov 08 00:42:56.459 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf187f44c7273f1328d614d1dcd9f701c5f66cf8849143e34c8d1f88d5e2ce55" already present on machine
Nov 08 00:42:56.600 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Node "ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal" drained (3 times)
Nov 08 00:42:56.650 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container thanos-sidecar
Nov 08 00:42:56.688 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container thanos-sidecar
Nov 08 00:42:56.694 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:42:56.899 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus-proxy
Nov 08 00:42:56.948 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus-proxy
Nov 08 00:42:56.958 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8" already present on machine
Nov 08 00:42:57.150 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container kube-rbac-proxy
Nov 08 00:42:57.197 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container kube-rbac-proxy
Nov 08 00:42:57.206 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8aefd089984c2e7d9dd7a162bd8e23bb9e492b494f6c285c2a8272f8230ff9e" already present on machine
Nov 08 00:42:57.313 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s (3 times)
Nov 08 00:42:57.429 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prom-label-proxy
Nov 08 00:42:57.459 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prom-label-proxy
Nov 08 00:42:58.320 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce5cf60d260dd8be75a016c30c7724b254f607573b9388da266d78fb3ac49757" already present on machine
Nov 08 00:42:58.334 E ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-b-l7d6t.c.openshift-gce-devel-ci.internal container=prometheus container exited with code 1 (Error): caller=main.go:657 msg="Starting TSDB ..."\nlevel=info ts=2019-11-08T00:42:53.285Z caller=web.go:450 component=web msg="Start listening for connections" address=127.0.0.1:9090\nlevel=info ts=2019-11-08T00:42:53.289Z caller=head.go:514 component=tsdb msg="replaying WAL, this may take awhile"\nlevel=info ts=2019-11-08T00:42:53.289Z caller=head.go:562 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:672 fs_type=XFS_SUPER_MAGIC\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:673 msg="TSDB started"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:743 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:526 msg="Stopping scrape discovery manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:540 msg="Stopping notify discovery manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:562 msg="Stopping scrape manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:556 msg="Scrape manager stopped"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=manager.go:814 component="rule manager" msg="Stopping rule manager..."\nlevel=info ts=2019-11-08T00:42:53.290Z caller=manager.go:820 component="rule manager" msg="Rule manager stopped"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:536 msg="Notify discovery manager stopped"\nlevel=info ts=2019-11-08T00:42:53.290Z caller=main.go:522 msg="Scrape discovery manager stopped"\nlevel=info ts=2019-11-08T00:42:53.293Z caller=notifier.go:602 component=notifier msg="Stopping notification manager..."\nlevel=info ts=2019-11-08T00:42:53.293Z caller=main.go:727 msg="Notifier manager stopped"\nlevel=error ts=2019-11-08
Nov 08 00:42:58.541 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus (2 times)
Nov 08 00:42:59.339 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--bqfmn-w-b-l7d6t.c.openshift-gce-devel-ci.internal container=prometheus container restarted
Nov 08 00:43:00.971 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 Node "ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal" drained (3 times)
Nov 08 00:43:01.693 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 requeue in: 20s (3 times)
Nov 08 00:43:13.389 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal condition DiskPressure changed
Nov 08 00:43:13.389 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal condition MemoryPressure changed
Nov 08 00:43:13.389 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal condition PIDPressure changed
Nov 08 00:43:13.389 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 00:43:13.415 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal status is now: NodeNotReady
Nov 08 00:43:13.875 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal condition DiskPressure changed
Nov 08 00:43:13.875 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal condition MemoryPressure changed
Nov 08 00:43:13.875 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal condition PIDPressure changed
Nov 08 00:43:13.875 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 00:43:13.959 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal status is now: NodeNotReady
Nov 08 00:43:14.252 W clusteroperator/dns changed Degraded to False: AsExpected: All desired DNS DaemonSets available and operand Namespace exists
Nov 08 00:43:14.258 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Node "ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal" drained (4 times)
Nov 08 00:43:14.387 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 00:43:15.015 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s (4 times)
Nov 08 00:43:15.084 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Node "ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal" drained (4 times)
Nov 08 00:43:15.929 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s (4 times)
Nov 08 00:43:16.065 I ns/openshift-image-registry pod/node-ca-k7jtb Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee"
Nov 08 00:43:16.273 I ns/openshift-image-registry pod/node-ca-k7jtb Created container node-ca
Nov 08 00:43:16.311 I ns/openshift-image-registry pod/node-ca-k7jtb Started container node-ca
Nov 08 00:43:16.516 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Node "ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal" drained (5 times)
Nov 08 00:43:17.339 W ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv requeue in: 20s (5 times)
Nov 08 00:43:17.400 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Node "ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal" drained (5 times)
Nov 08 00:43:18.149 W ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 requeue in: 20s (5 times)
Nov 08 00:43:19.191 W node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal condition DiskPressure changed
Nov 08 00:43:19.191 W node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal condition MemoryPressure changed
Nov 08 00:43:19.191 W node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal condition PIDPressure changed
Nov 08 00:43:19.191 W node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 00:43:19.198 I node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal status is now: NodeNotReady
Nov 08 00:43:19.396 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 Node "ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal" drained (4 times)
Nov 08 00:43:20.235 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 requeue in: 20s (4 times)
Nov 08 00:43:21.752 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 Node "ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal" drained (5 times)
Nov 08 00:43:22.549 W ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 requeue in: 20s (5 times)
Nov 08 00:43:23.365 W node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:23.369 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal event: DeletingNode
Nov 08 00:43:24.361 I node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal event: Removing Node ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal from Controller
Nov 08 00:43:27.923 W node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal node is not ready
Nov 08 00:43:27.923 W ns/openshift-image-registry pod/node-ca-nzhnw node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:43:27.923 W ns/openshift-dns pod/dns-default-wz2js node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:43:29.053 I node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal event: DeletingNode
Nov 08 00:43:29.059 W node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:29.361 I node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal event: Removing Node ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal from Controller
Nov 08 00:43:33.918 W ns/openshift-sdn pod/ovs-n8qdt node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:33.928 W ns/openshift-sdn pod/ovs-n8qdt node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:33.943 W ns/openshift-multus pod/multus-556cz node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:33.953 W ns/openshift-multus pod/multus-556cz node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:33.963 W ns/openshift-image-registry pod/node-ca-nzhnw node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:33.973 W ns/openshift-image-registry pod/node-ca-nzhnw node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:33.992 W ns/openshift-cluster-node-tuning-operator pod/tuned-4c4jk node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:33.999 W ns/openshift-cluster-node-tuning-operator pod/tuned-4c4jk node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.013 W ns/openshift-image-registry pod/node-ca-gtbzs node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.021 W ns/openshift-image-registry pod/node-ca-gtbzs node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.036 W ns/openshift-sdn pod/sdn-dsnvt node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.044 W ns/openshift-sdn pod/sdn-dsnvt node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.061 W ns/openshift-dns pod/dns-default-wz2js node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.073 W ns/openshift-dns pod/dns-default-wz2js node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.089 W ns/openshift-dns pod/dns-default-w929g node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.096 W ns/openshift-dns pod/dns-default-w929g node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.109 W ns/openshift-multus pod/multus-p9kk9 node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.119 W ns/openshift-multus pod/multus-p9kk9 node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.134 W ns/openshift-machine-config-operator pod/machine-config-daemon-hxfkp node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.153 W ns/openshift-machine-config-operator pod/machine-config-daemon-hxfkp node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.174 W ns/openshift-monitoring pod/node-exporter-hx94v node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.196 W ns/openshift-monitoring pod/node-exporter-hx94v node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.222 W ns/openshift-monitoring pod/node-exporter-46zhp node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.241 W ns/openshift-monitoring pod/node-exporter-46zhp node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.256 W ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.273 W ns/openshift-machine-config-operator pod/machine-config-daemon-rx7xt node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.289 W ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.295 W ns/openshift-cluster-node-tuning-operator pod/tuned-dbhmr node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.315 W ns/openshift-sdn pod/ovs-gxbf7 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.333 W ns/openshift-sdn pod/ovs-gxbf7 node/ci-op--bqfmn-w-d-ql8g7.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.343 W ns/openshift-sdn pod/sdn-k6rl2 node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:34.351 W ns/openshift-sdn pod/sdn-k6rl2 node/ci-op--bqfmn-w-c-jdgh6.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.651 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal event: DeletingNode
Nov 08 00:43:34.656 W node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:34.739 I ns/openshift-ingress service/router-default Updated load balancer with new hosts
Nov 08 00:43:37.891 I ns/openshift-machine-api machine/ci-op--bqfmn-w-b-j6wwv Deleted machine ci-op--bqfmn-w-b-j6wwv
Nov 08 00:43:38.659 I ns/openshift-machine-api machine/ci-op--bqfmn-w-c-jdgh6 Deleted machine ci-op--bqfmn-w-c-jdgh6
Nov 08 00:43:39.362 I node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal event: Removing Node ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal from Controller
Nov 08 00:43:42.923 W ns/openshift-image-registry pod/node-ca-79xqh node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:43:42.923 W ns/openshift-dns pod/dns-default-s9xrl node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:43:42.923 W ns/openshift-machine-config-operator pod/machine-config-daemon-8fbr7 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:43:43.114 I ns/openshift-machine-api machine/ci-op--bqfmn-w-d-ql8g7 Deleted machine ci-op--bqfmn-w-d-ql8g7
Nov 08 00:43:54.367 W ns/openshift-sdn pod/ovs-nwxn4 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.377 W ns/openshift-sdn pod/ovs-nwxn4 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.388 W ns/openshift-sdn pod/sdn-kvf4x node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.407 W ns/openshift-sdn pod/sdn-kvf4x node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.426 W ns/openshift-image-registry pod/node-ca-79xqh node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.438 W ns/openshift-image-registry pod/node-ca-79xqh node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.449 W ns/openshift-multus pod/multus-fvh4s node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.459 W ns/openshift-multus pod/multus-fvh4s node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.469 W ns/openshift-machine-config-operator pod/machine-config-daemon-8fbr7 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.476 W clusteroperator/network changed Progressing to False
Nov 08 00:43:54.480 W ns/openshift-machine-config-operator pod/machine-config-daemon-8fbr7 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.496 W ns/openshift-monitoring pod/node-exporter-h2k78 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.501 W ns/openshift-monitoring pod/node-exporter-h2k78 node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.518 W ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.525 W ns/openshift-cluster-node-tuning-operator pod/tuned-ql94w node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:43:54.537 W ns/openshift-dns pod/dns-default-s9xrl node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 00:43:54.544 W ns/openshift-dns pod/dns-default-s9xrl node/ci-op--bqfmn-w-b-j6wwv.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:44:40.218 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (6 times)
Nov 08 00:44:40.229 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (2 times)
Nov 08 00:44:40.235 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-001" not found) (3 times)
Nov 08 00:44:40.613 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-002" not found) (3 times)
Nov 08 00:44:41.015 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-003" not found) (3 times)
Nov 08 00:48:07.202 I persistentvolume/pvc-37335352-3af0-4704-a877-d6093fa395c1 googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--bqfmn-dynamic-pvc-37335352-3af0-4704-a877-d6093fa395c1' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--bqfmn-w-d-zzcqg', resourceInUseByAnotherResource
Nov 08 00:48:10.452 I persistentvolume/pvc-37335352-3af0-4704-a877-d6093fa395c1 googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--bqfmn-dynamic-pvc-37335352-3af0-4704-a877-d6093fa395c1' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--bqfmn-w-d-zzcqg', resourceInUseByAnotherResource (2 times)
Nov 08 00:50:40.174 I ns/openshift-image-registry pod/node-ca-4wdf5 Marking for deletion Pod openshift-image-registry/node-ca-4wdf5
Nov 08 00:50:40.193 I ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-kqwl9
Nov 08 00:50:40.197 W ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 00:50:40.198 W ns/openshift-dns pod/dns-default-4m689 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:50:40.202 W ns/openshift-image-registry pod/node-ca-4wdf5 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:50:40.203 W ns/openshift-ingress pod/router-default-56666d9db-k7b69 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:50:40.212 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 00:50:40.213 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-kqwl9
Nov 08 00:50:40.234 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:50:40.234 W ns/openshift-monitoring pod/alertmanager-main-0 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal graceful deletion within 120s
Nov 08 00:50:40.239 I ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 Stopping container oauth-proxy
Nov 08 00:50:40.261 I ns/openshift-monitoring pod/prometheus-k8s-1 Marking for deletion Pod openshift-monitoring/prometheus-k8s-1
Nov 08 00:50:40.261 I ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 Stopping container machine-config-daemon
Nov 08 00:50:40.261 I ns/openshift-dns daemonset/dns-default Deleted pod: dns-default-4m689
Nov 08 00:50:40.284 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Marking for deletion Pod openshift-monitoring/thanos-querier-5ddfc789fd-ldxk4
Nov 08 00:50:40.286 I ns/openshift-dns pod/dns-default-4m689 Stopping container dns-node-resolver
Nov 08 00:50:40.299 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-4wdf5
Nov 08 00:50:40.303 I ns/openshift-dns pod/dns-default-4m689 Marking for deletion Pod openshift-dns/dns-default-4m689
Nov 08 00:50:40.321 I ns/openshift-dns pod/dns-default-4m689 Stopping container dns
Nov 08 00:50:40.337 I ns/openshift-ingress pod/router-default-56666d9db-k7b69 Marking for deletion Pod openshift-ingress/router-default-56666d9db-k7b69
Nov 08 00:50:40.338 I ns/openshift-image-registry pod/node-ca-4wdf5 Stopping container node-ca
Nov 08 00:50:40.341 I ns/openshift-ingress pod/router-default-56666d9db-qht49 node/ created
Nov 08 00:50:40.367 I ns/openshift-ingress replicaset/router-default-56666d9db Created pod: router-default-56666d9db-qht49
Nov 08 00:50:40.369 I ns/openshift-monitoring pod/alertmanager-main-0 Marking for deletion Pod openshift-monitoring/alertmanager-main-0
Nov 08 00:50:40.373 I ns/openshift-ingress pod/router-default-56666d9db-k7b69 Stopping container router
Nov 08 00:50:40.393 I ns/openshift-monitoring pod/prometheus-k8s-1 Stopping container prometheus
Nov 08 00:50:40.413 I ns/openshift-monitoring pod/prometheus-k8s-1 Stopping container prom-label-proxy
Nov 08 00:50:40.426 I ns/openshift-monitoring pod/prometheus-k8s-1 Stopping container kube-rbac-proxy
Nov 08 00:50:40.437 I ns/openshift-monitoring pod/prometheus-k8s-1 Stopping container prometheus-proxy
Nov 08 00:50:40.441 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Successfully assigned openshift-ingress/router-default-56666d9db-qht49 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:50:40.474 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Stopping container prom-label-proxy
Nov 08 00:50:40.484 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ created
Nov 08 00:50:40.505 I ns/openshift-monitoring replicaset/thanos-querier-5ddfc789fd Created pod: thanos-querier-5ddfc789fd-cvpnb
Nov 08 00:50:40.554 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Successfully assigned openshift-monitoring/thanos-querier-5ddfc789fd-cvpnb to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:50:40.606 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Stopping container kube-rbac-proxy
Nov 08 00:50:40.666 W clusteroperator/ingress changed Available to False: IngressUnavailable: Not all ingress controllers are available.
Nov 08 00:50:40.666 W clusteroperator/ingress changed Progressing to True: Reconciling: Not all ingress controllers are available.
Nov 08 00:50:40.822 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Stopping container oauth-proxy
Nov 08 00:50:41.014 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 Stopping container thanos-querier
Nov 08 00:50:41.223 I ns/openshift-monitoring pod/alertmanager-main-0 Stopping container alertmanager-proxy
Nov 08 00:50:41.412 I ns/openshift-monitoring pod/alertmanager-main-0 Stopping container config-reloader
Nov 08 00:50:41.528 E ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 00:50:41.612 I ns/openshift-monitoring pod/alertmanager-main-0 Stopping container alertmanager
Nov 08 00:50:41.805 W ns/openshift-ingress pod/router-default-56666d9db-k7b69 Liveness probe failed: Get http://10.128.2.4:1936/healthz: dial tcp 10.128.2.4:1936: connect: connection refused
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prometheus container stopped being ready
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prometheus-config-reloader container stopped being ready
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prometheus-proxy container stopped being ready
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=rules-configmap-reloader container stopped being ready
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=thanos-sidecar container stopped being ready
Nov 08 00:50:42.604 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready
Nov 08 00:50:42.633 E ns/openshift-monitoring pod/alertmanager-main-0 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/08 00:29:22 Watching directory: "/etc/alertmanager/config"\n
Nov 08 00:50:42.633 E ns/openshift-monitoring pod/alertmanager-main-0 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/08 00:29:23 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:29:23 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/08 00:29:23 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/08 00:29:23 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/08 00:29:23 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/08 00:29:23 oauthproxy.go:227: OAuthProxy configured for  Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/08 00:29:23 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/08 00:29:23 http.go:96: HTTPS: listening on [::]:9095\n
Nov 08 00:50:42.657 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 2 (Error): 2019/11/08 00:42:45 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:thanos-querier\n2019/11/08 00:42:45 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/08 00:42:45 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/08 00:42:45 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9090/"\n2019/11/08 00:42:45 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/08 00:42:45 oauthproxy.go:227: OAuthProxy configured for  Client ID: system:serviceaccount:openshift-monitoring:thanos-querier\n2019/11/08 00:42:45 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/08 00:42:45 main.go:154: using htpasswd file /etc/proxy/htpasswd/auth\n2019/11/08 00:42:45 http.go:96: HTTPS: listening on [::]:9091\n
Nov 08 00:50:42.687 E ns/openshift-ingress pod/router-default-56666d9db-k7b69 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=router container exited with code 2 (Error): Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:35.170Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:40.166Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:45.170Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:50.162Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:42:55.170Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:00.162Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:05.168Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:13.771Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:18.769Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:43:23.771Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-08T00:44:52.640Z	INFO	router.template	template/router.go:548	router reloaded	{"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n
Nov 08 00:50:42.814 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 00:50:42.923 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:50:48.036 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7ece12ae9ebfcce0c5b8d8a2eff009a1706a070e9021e96112dbbe6711795a35"
Nov 08 00:50:48.794 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf187f44c7273f1328d614d1dcd9f701c5f66cf8849143e34c8d1f88d5e2ce55"
Nov 08 00:50:49.744 W ns/openshift-ingress pod/router-default-56666d9db-k7b69 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:50:49.780 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-ldxk4 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:50:49.809 W ns/openshift-image-registry pod/node-ca-4wdf5 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:50:49.860 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:50:49.874 I ns/openshift-image-registry pod/node-ca-tlv48 node/ created
Nov 08 00:50:49.885 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-tlv48
Nov 08 00:50:49.899 I ns/openshift-image-registry pod/node-ca-tlv48 Successfully assigned openshift-image-registry/node-ca-tlv48 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:50:49.902 W ns/openshift-dns pod/dns-default-4m689 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:50:49.922 I ns/openshift-dns pod/dns-default-dnnr6 node/ created
Nov 08 00:50:49.932 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-dnnr6
Nov 08 00:50:49.948 W ns/openshift-monitoring pod/alertmanager-main-0 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:50:49.949 I ns/openshift-dns pod/dns-default-dnnr6 Successfully assigned openshift-dns/dns-default-dnnr6 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:50:49.974 I ns/openshift-monitoring pod/prometheus-k8s-1 node/ created
Nov 08 00:50:49.988 I ns/openshift-monitoring statefulset/prometheus-k8s create Pod prometheus-k8s-1 in StatefulSet prometheus-k8s successful (2 times)
Nov 08 00:50:49.989 I ns/openshift-monitoring pod/prometheus-k8s-1 Successfully assigned openshift-monitoring/prometheus-k8s-1 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:50:50.021 I ns/openshift-monitoring pod/alertmanager-main-0 node/ created
Nov 08 00:50:50.043 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-0 in StatefulSet alertmanager-main successful (2 times)
Nov 08 00:50:50.048 I ns/openshift-monitoring pod/alertmanager-main-0 Successfully assigned openshift-monitoring/alertmanager-main-0 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:50:50.103 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (5 times)
Nov 08 00:50:50.706 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (16 times)
Nov 08 00:50:51.610 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7ece12ae9ebfcce0c5b8d8a2eff009a1706a070e9021e96112dbbe6711795a35"
Nov 08 00:50:51.824 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Created container router
Nov 08 00:50:51.871 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Started container router
Nov 08 00:50:52.603 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf187f44c7273f1328d614d1dcd9f701c5f66cf8849143e34c8d1f88d5e2ce55"
Nov 08 00:50:52.771 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Created container thanos-querier
Nov 08 00:50:52.806 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Started container thanos-querier
Nov 08 00:50:52.811 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:50:53.002 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Created container oauth-proxy
Nov 08 00:50:53.037 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Started container oauth-proxy
Nov 08 00:50:53.043 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8" already present on machine
Nov 08 00:50:53.225 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Created container kube-rbac-proxy
Nov 08 00:50:53.258 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Started container kube-rbac-proxy
Nov 08 00:50:53.264 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8aefd089984c2e7d9dd7a162bd8e23bb9e492b494f6c285c2a8272f8230ff9e"
Nov 08 00:50:55.589 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8aefd089984c2e7d9dd7a162bd8e23bb9e492b494f6c285c2a8272f8230ff9e"
Nov 08 00:50:55.758 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Created container prom-label-proxy
Nov 08 00:50:55.788 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Started container prom-label-proxy
Nov 08 00:50:57.980 I ns/openshift-dns pod/dns-default-dnnr6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 00:50:58.144 I ns/openshift-dns pod/dns-default-dnnr6 Created container dns
Nov 08 00:50:58.175 I ns/openshift-dns pod/dns-default-dnnr6 Started container dns
Nov 08 00:50:58.181 I ns/openshift-dns pod/dns-default-dnnr6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 00:50:58.319 I ns/openshift-dns pod/dns-default-dnnr6 Created container dns-node-resolver
Nov 08 00:50:58.327 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce5cf60d260dd8be75a016c30c7724b254f607573b9388da266d78fb3ac49757" already present on machine
Nov 08 00:50:58.358 I ns/openshift-dns pod/dns-default-dnnr6 Started container dns-node-resolver
Nov 08 00:50:58.395 I ns/openshift-image-registry pod/node-ca-tlv48 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 00:50:58.514 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container prometheus
Nov 08 00:50:58.543 I ns/openshift-image-registry pod/node-ca-tlv48 Created container node-ca
Nov 08 00:50:58.550 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container prometheus
Nov 08 00:50:58.558 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9a3a81674e3aa3d54fc056ba4907a2f48b867c375d8fe86e05d9e1c99489eade" already present on machine
Nov 08 00:50:58.584 I ns/openshift-image-registry pod/node-ca-tlv48 Started container node-ca
Nov 08 00:50:58.783 I ns/openshift-monitoring pod/alertmanager-main-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:78ba454f09557364dabd918265784493287ed242fb51ac2ffde695ae78823736" already present on machine
Nov 08 00:50:58.983 I ns/openshift-monitoring pod/alertmanager-main-0 Created container alertmanager
Nov 08 00:50:59.181 I ns/openshift-monitoring pod/alertmanager-main-0 Started container alertmanager
Nov 08 00:50:59.380 I ns/openshift-monitoring pod/alertmanager-main-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db10cd1f5efd9e9e6f2b0eea0a665f53399e25de374cf916ad7d8141f8ac643d" already present on machine
Nov 08 00:50:59.589 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container prometheus-config-reloader
Nov 08 00:50:59.781 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container prometheus-config-reloader
Nov 08 00:50:59.982 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db10cd1f5efd9e9e6f2b0eea0a665f53399e25de374cf916ad7d8141f8ac643d" already present on machine
Nov 08 00:51:00.182 I ns/openshift-monitoring pod/alertmanager-main-0 Created container config-reloader
Nov 08 00:51:00.380 I ns/openshift-monitoring pod/alertmanager-main-0 Started container config-reloader
Nov 08 00:51:00.586 I ns/openshift-monitoring pod/alertmanager-main-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:51:00.698 E ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prometheus container exited with code 1 (Error): caller=main.go:657 msg="Starting TSDB ..."\nlevel=info ts=2019-11-08T00:50:58.590Z caller=web.go:450 component=web msg="Start listening for connections" address=127.0.0.1:9090\nlevel=info ts=2019-11-08T00:50:58.594Z caller=head.go:514 component=tsdb msg="replaying WAL, this may take awhile"\nlevel=info ts=2019-11-08T00:50:58.594Z caller=head.go:562 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:672 fs_type=XFS_SUPER_MAGIC\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:673 msg="TSDB started"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:743 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:526 msg="Stopping scrape discovery manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:540 msg="Stopping notify discovery manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:562 msg="Stopping scrape manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:536 msg="Notify discovery manager stopped"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:522 msg="Scrape discovery manager stopped"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=main.go:556 msg="Scrape manager stopped"\nlevel=info ts=2019-11-08T00:50:58.595Z caller=manager.go:814 component="rule manager" msg="Stopping rule manager..."\nlevel=info ts=2019-11-08T00:50:58.595Z caller=manager.go:820 component="rule manager" msg="Rule manager stopped"\nlevel=info ts=2019-11-08T00:50:58.597Z caller=notifier.go:602 component=notifier msg="Stopping notification manager..."\nlevel=info ts=2019-11-08T00:50:58.597Z caller=main.go:727 msg="Notifier manager stopped"\nlevel=error ts=2019-11-08
Nov 08 00:51:00.784 I ns/openshift-monitoring pod/alertmanager-main-0 Created container alertmanager-proxy
Nov 08 00:51:00.981 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container rules-configmap-reloader
Nov 08 00:51:01.181 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container rules-configmap-reloader
Nov 08 00:51:01.382 I ns/openshift-monitoring pod/alertmanager-main-0 Started container alertmanager-proxy
Nov 08 00:51:01.580 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf187f44c7273f1328d614d1dcd9f701c5f66cf8849143e34c8d1f88d5e2ce55" already present on machine
Nov 08 00:51:01.706 W ns/openshift-monitoring pod/prometheus-k8s-1 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal container=prometheus container restarted
Nov 08 00:51:01.782 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container thanos-sidecar
Nov 08 00:51:01.980 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container thanos-sidecar
Nov 08 00:51:02.180 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:51:02.381 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container prometheus-proxy
Nov 08 00:51:02.580 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container prometheus-proxy
Nov 08 00:51:02.779 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8" already present on machine
Nov 08 00:51:02.981 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container kube-rbac-proxy
Nov 08 00:51:03.083 W clusteroperator/ingress changed Available to True: desired and current number of IngressControllers are equal
Nov 08 00:51:03.083 W clusteroperator/ingress changed Progressing to False: desired and current number of IngressControllers are equal
Nov 08 00:51:03.179 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container kube-rbac-proxy
Nov 08 00:51:03.386 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8aefd089984c2e7d9dd7a162bd8e23bb9e492b494f6c285c2a8272f8230ff9e" already present on machine
Nov 08 00:51:03.583 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container prom-label-proxy
Nov 08 00:51:03.780 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container prom-label-proxy
Nov 08 00:51:04.012 I ns/openshift-monitoring pod/prometheus-k8s-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce5cf60d260dd8be75a016c30c7724b254f607573b9388da266d78fb3ac49757" already present on machine (2 times)
Nov 08 00:51:04.189 I ns/openshift-monitoring pod/prometheus-k8s-1 Created container prometheus (2 times)
Nov 08 00:51:04.385 I ns/openshift-monitoring pod/prometheus-k8s-1 Started container prometheus (2 times)
Nov 08 00:51:09.913 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 00:51:44.740 W ns/openshift-machine-config-operator pod/machine-config-daemon-kqwl9 node/ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:51:44.758 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv node/ created
Nov 08 00:51:44.767 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-mvfsv
Nov 08 00:51:44.778 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Successfully assigned openshift-machine-config-operator/machine-config-daemon-mvfsv to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:51:45.432 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 00:51:45.567 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Created container machine-config-daemon
Nov 08 00:51:45.602 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Started container machine-config-daemon
Nov 08 00:51:45.611 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:51:45.806 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Created container oauth-proxy
Nov 08 00:51:45.839 I ns/openshift-machine-config-operator pod/machine-config-daemon-mvfsv Started container oauth-proxy
Nov 08 00:52:46.439 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (12 times)
Nov 08 00:52:47.033 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (14 times)
Nov 08 00:53:37.243 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Marking for deletion Pod openshift-ingress/router-default-56666d9db-qht49
Nov 08 00:53:37.249 W ns/openshift-dns pod/dns-default-mwcnm node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:53:37.253 W ns/openshift-ingress pod/router-default-56666d9db-qht49 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:53:37.254 I ns/openshift-dns pod/dns-default-mwcnm Marking for deletion Pod openshift-dns/dns-default-mwcnm
Nov 08 00:53:37.275 W ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 00:53:37.275 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:53:37.278 W ns/openshift-image-registry pod/node-ca-k7jtb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:53:37.281 I ns/openshift-dns daemonset/dns-default Deleted pod: dns-default-mwcnm
Nov 08 00:53:37.283 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Marking for deletion Pod openshift-monitoring/thanos-querier-5ddfc789fd-cvpnb
Nov 08 00:53:37.284 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 120s
Nov 08 00:53:37.285 I ns/openshift-ingress pod/router-default-56666d9db-qht49 Stopping container router
Nov 08 00:53:37.287 I ns/openshift-dns pod/dns-default-mwcnm Stopping container dns-node-resolver
Nov 08 00:53:37.303 I ns/openshift-dns pod/dns-default-mwcnm Stopping container dns
Nov 08 00:53:37.312 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-64z97
Nov 08 00:53:37.312 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-k7jtb
Nov 08 00:53:37.317 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Stopping container oauth-proxy
Nov 08 00:53:37.324 I ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 Stopping container machine-config-daemon
Nov 08 00:53:37.326 I ns/openshift-ingress pod/router-default-56666d9db-6ck52 node/ created
Nov 08 00:53:37.342 I ns/openshift-image-registry pod/node-ca-k7jtb Stopping container node-ca
Nov 08 00:53:37.357 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager
Nov 08 00:53:37.357 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-64z97
Nov 08 00:53:37.357 I ns/openshift-image-registry pod/node-ca-k7jtb Marking for deletion Pod openshift-image-registry/node-ca-k7jtb
Nov 08 00:53:37.357 I ns/openshift-ingress replicaset/router-default-56666d9db Created pod: router-default-56666d9db-6ck52
Nov 08 00:53:37.360 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Stopping container prom-label-proxy
Nov 08 00:53:37.375 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Stopping container kube-rbac-proxy
Nov 08 00:53:37.384 I ns/openshift-ingress pod/router-default-56666d9db-6ck52 Successfully assigned openshift-ingress/router-default-56666d9db-6ck52 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:53:37.384 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Stopping container oauth-proxy
Nov 08 00:53:37.392 I ns/openshift-monitoring pod/alertmanager-main-2 Marking for deletion Pod openshift-monitoring/alertmanager-main-2
Nov 08 00:53:37.431 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb node/ created
Nov 08 00:53:37.442 I ns/openshift-monitoring replicaset/thanos-querier-5ddfc789fd Created pod: thanos-querier-5ddfc789fd-kwmdb
Nov 08 00:53:37.469 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Successfully assigned openshift-monitoring/thanos-querier-5ddfc789fd-kwmdb to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:53:37.502 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb Stopping container thanos-querier
Nov 08 00:53:37.644 W clusteroperator/ingress changed Available to False: IngressUnavailable: Not all ingress controllers are available.
Nov 08 00:53:37.644 W clusteroperator/ingress changed Progressing to True: Reconciling: Not all ingress controllers are available.
Nov 08 00:53:37.677 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy
Nov 08 00:53:37.869 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader
Nov 08 00:53:39.234 W ns/openshift-ingress pod/router-default-56666d9db-qht49 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:53:39.234 W ns/openshift-ingress pod/router-default-56666d9db-qht49 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=router container stopped being ready
Nov 08 00:53:39.249 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:53:39.249 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready
Nov 08 00:53:39.249 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready
Nov 08 00:53:39.249 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=thanos-querier container stopped being ready
Nov 08 00:53:39.249 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=thanos-querier container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=prom-label-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.265 E ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.285 W ns/openshift-dns pod/dns-default-mwcnm node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:53:39.285 W ns/openshift-dns pod/dns-default-mwcnm node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=dns-node-resolver container stopped being ready
Nov 08 00:53:39.285 W ns/openshift-dns pod/dns-default-mwcnm node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=dns container stopped being ready
Nov 08 00:53:39.307 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:53:39.307 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=alertmanager container stopped being ready
Nov 08 00:53:39.307 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container stopped being ready
Nov 08 00:53:39.307 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=config-reloader container stopped being ready
Nov 08 00:53:39.323 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.323 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.323 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=alertmanager container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 08 00:53:39.345 W ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:53:39.345 W ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=machine-config-daemon container stopped being ready
Nov 08 00:53:39.345 W ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready
Nov 08 00:53:39.375 W ns/openshift-image-registry pod/node-ca-k7jtb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:53:39.375 W ns/openshift-image-registry pod/node-ca-k7jtb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready
Nov 08 00:53:42.923 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:53:42.923 W ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:53:42.923 W ns/openshift-image-registry pod/node-ca-k7jtb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:53:42.923 W ns/openshift-ingress pod/router-default-56666d9db-qht49 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:53:42.923 W ns/openshift-dns pod/dns-default-mwcnm node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute
Nov 08 00:53:45.195 I ns/openshift-ingress pod/router-default-56666d9db-6ck52 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7ece12ae9ebfcce0c5b8d8a2eff009a1706a070e9021e96112dbbe6711795a35" already present on machine
Nov 08 00:53:45.303 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bf187f44c7273f1328d614d1dcd9f701c5f66cf8849143e34c8d1f88d5e2ce55" already present on machine
Nov 08 00:53:45.370 I ns/openshift-ingress pod/router-default-56666d9db-6ck52 Created container router
Nov 08 00:53:45.414 I ns/openshift-ingress pod/router-default-56666d9db-6ck52 Started container router
Nov 08 00:53:45.485 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Created container thanos-querier
Nov 08 00:53:45.519 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Started container thanos-querier
Nov 08 00:53:45.526 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:53:45.705 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Created container oauth-proxy
Nov 08 00:53:45.739 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Started container oauth-proxy
Nov 08 00:53:45.746 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5366da9d7c15f912b609cdbb2851d01ea863fb22d452081914d923d8208c39c8" already present on machine
Nov 08 00:53:45.923 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Created container kube-rbac-proxy
Nov 08 00:53:45.963 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Started container kube-rbac-proxy
Nov 08 00:53:45.969 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b8aefd089984c2e7d9dd7a162bd8e23bb9e492b494f6c285c2a8272f8230ff9e" already present on machine
Nov 08 00:53:46.152 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Created container prom-label-proxy
Nov 08 00:53:46.179 I ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-kwmdb Started container prom-label-proxy
Nov 08 00:53:46.556 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:53:46.580 W ns/openshift-image-registry pod/node-ca-k7jtb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:53:46.608 I ns/openshift-monitoring pod/alertmanager-main-2 node/ created
Nov 08 00:53:46.614 W ns/openshift-monitoring pod/thanos-querier-5ddfc789fd-cvpnb node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:53:46.624 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-2 in StatefulSet alertmanager-main successful (3 times)
Nov 08 00:53:46.629 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully assigned openshift-monitoring/alertmanager-main-2 to ci-op--bqfmn-w-c-vrsgm.c.openshift-gce-devel-ci.internal
Nov 08 00:53:46.646 W ns/openshift-dns pod/dns-default-mwcnm node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:53:46.674 W ns/openshift-machine-config-operator pod/machine-config-daemon-64z97 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:53:46.701 W ns/openshift-ingress pod/router-default-56666d9db-qht49 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:53:54.469 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:78ba454f09557364dabd918265784493287ed242fb51ac2ffde695ae78823736" already present on machine
Nov 08 00:53:54.655 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager
Nov 08 00:53:54.692 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager
Nov 08 00:53:54.699 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:db10cd1f5efd9e9e6f2b0eea0a665f53399e25de374cf916ad7d8141f8ac643d" already present on machine
Nov 08 00:53:55.130 I ns/openshift-monitoring pod/alertmanager-main-2 Created container config-reloader
Nov 08 00:53:55.158 I ns/openshift-monitoring pod/alertmanager-main-2 Started container config-reloader
Nov 08 00:53:55.164 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:53:55.344 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager-proxy
Nov 08 00:53:55.375 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager-proxy
Nov 08 00:54:02.477 W clusteroperator/ingress changed Available to True: desired and current number of IngressControllers are equal
Nov 08 00:54:02.477 W clusteroperator/ingress changed Progressing to False: desired and current number of IngressControllers are equal
Nov 08 00:54:40.216 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (10 times)
Nov 08 00:54:41.209 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (3 times)
Nov 08 00:54:42.271 I ns/openshift-dns pod/dns-default-f7fmn node/ created
Nov 08 00:54:42.292 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-f7fmn
Nov 08 00:54:42.294 I ns/openshift-image-registry pod/node-ca-9pmk6 node/ created
Nov 08 00:54:42.298 I ns/openshift-dns pod/dns-default-f7fmn Successfully assigned openshift-dns/dns-default-f7fmn to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:54:42.302 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-9pmk6
Nov 08 00:54:42.312 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx node/ created
Nov 08 00:54:42.322 I ns/openshift-image-registry pod/node-ca-9pmk6 Successfully assigned openshift-image-registry/node-ca-9pmk6 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:54:42.324 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-ms4lx
Nov 08 00:54:42.336 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Successfully assigned openshift-machine-config-operator/machine-config-daemon-ms4lx to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:54:42.409 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 00:54:43.184 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 00:54:43.373 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Created container machine-config-daemon
Nov 08 00:54:43.413 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Started container machine-config-daemon
Nov 08 00:54:43.422 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:54:43.633 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Created container oauth-proxy
Nov 08 00:54:43.692 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Started container oauth-proxy
Nov 08 00:54:51.035 I ns/openshift-image-registry pod/node-ca-9pmk6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 00:54:51.224 I ns/openshift-image-registry pod/node-ca-9pmk6 Created container node-ca
Nov 08 00:54:51.257 I ns/openshift-image-registry pod/node-ca-9pmk6 Started container node-ca
Nov 08 00:54:55.771 I ns/openshift-dns pod/dns-default-f7fmn Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 00:54:55.948 I ns/openshift-dns pod/dns-default-f7fmn Created container dns
Nov 08 00:54:55.979 I ns/openshift-dns pod/dns-default-f7fmn Started container dns
Nov 08 00:54:55.991 I ns/openshift-dns pod/dns-default-f7fmn Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 00:54:56.137 I ns/openshift-dns pod/dns-default-f7fmn Created container dns-node-resolver
Nov 08 00:54:56.170 I ns/openshift-dns pod/dns-default-f7fmn Started container dns-node-resolver
Nov 08 00:55:07.457 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 00:55:57.379 W ns/openshift-image-registry pod/node-ca-9pmk6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:55:57.384 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-ms4lx
Nov 08 00:55:57.386 W ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 00:55:57.389 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-9pmk6
Nov 08 00:55:57.389 W ns/openshift-dns pod/dns-default-f7fmn node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 00:55:57.395 I ns/openshift-image-registry pod/node-ca-9pmk6 Stopping container node-ca
Nov 08 00:55:57.398 I ns/openshift-image-registry pod/node-ca-9pmk6 Marking for deletion Pod openshift-image-registry/node-ca-9pmk6
Nov 08 00:55:57.401 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Stopping container oauth-proxy
Nov 08 00:55:57.412 I ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx Stopping container machine-config-daemon
Nov 08 00:55:57.414 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-ms4lx
Nov 08 00:55:57.414 I ns/openshift-dns pod/dns-default-f7fmn Marking for deletion Pod openshift-dns/dns-default-f7fmn
Nov 08 00:55:57.424 I ns/openshift-dns pod/dns-default-f7fmn Stopping container dns-node-resolver
Nov 08 00:55:57.429 I ns/openshift-dns pod/dns-default-f7fmn Stopping container dns
Nov 08 00:55:58.751 W ns/openshift-image-registry pod/node-ca-9pmk6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:55:58.751 W ns/openshift-image-registry pod/node-ca-9pmk6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready
Nov 08 00:55:58.764 W ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:55:58.764 W ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready
Nov 08 00:55:58.764 W ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=machine-config-daemon container stopped being ready
Nov 08 00:55:58.789 W ns/openshift-dns pod/dns-default-f7fmn node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 00:55:58.789 W ns/openshift-dns pod/dns-default-f7fmn node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=dns-node-resolver container stopped being ready
Nov 08 00:55:58.789 W ns/openshift-dns pod/dns-default-f7fmn node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=dns container stopped being ready
Nov 08 00:56:06.559 W ns/openshift-image-registry pod/node-ca-9pmk6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:56:06.595 W ns/openshift-machine-config-operator pod/machine-config-daemon-ms4lx node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:56:06.646 W ns/openshift-dns pod/dns-default-f7fmn node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 00:57:02.404 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh node/ created
Nov 08 00:57:02.421 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-2vmfh
Nov 08 00:57:02.430 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Successfully assigned openshift-machine-config-operator/machine-config-daemon-2vmfh to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:57:02.449 I ns/openshift-image-registry pod/node-ca-6pw92 node/ created
Nov 08 00:57:02.450 I ns/openshift-dns pod/dns-default-r6pkg node/ created
Nov 08 00:57:02.459 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-6pw92
Nov 08 00:57:02.469 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-r6pkg
Nov 08 00:57:02.475 I ns/openshift-dns pod/dns-default-r6pkg Successfully assigned openshift-dns/dns-default-r6pkg to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:57:02.485 I ns/openshift-image-registry pod/node-ca-6pw92 Successfully assigned openshift-image-registry/node-ca-6pw92 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 00:57:02.647 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 00:57:03.236 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 00:57:03.406 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Created container machine-config-daemon
Nov 08 00:57:03.444 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Started container machine-config-daemon
Nov 08 00:57:03.448 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 00:57:03.636 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Created container oauth-proxy
Nov 08 00:57:03.698 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Started container oauth-proxy
Nov 08 00:57:10.446 I ns/openshift-image-registry pod/node-ca-6pw92 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 00:57:10.640 I ns/openshift-image-registry pod/node-ca-6pw92 Created container node-ca
Nov 08 00:57:10.674 I ns/openshift-image-registry pod/node-ca-6pw92 Started container node-ca
Nov 08 00:57:11.052 I ns/openshift-dns pod/dns-default-r6pkg Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 00:57:11.243 I ns/openshift-dns pod/dns-default-r6pkg Created container dns
Nov 08 00:57:11.274 I ns/openshift-dns pod/dns-default-r6pkg Started container dns
Nov 08 00:57:11.285 I ns/openshift-dns pod/dns-default-r6pkg Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 00:57:11.442 I ns/openshift-dns pod/dns-default-r6pkg Created container dns-node-resolver
Nov 08 00:57:11.477 I ns/openshift-dns pod/dns-default-r6pkg Started container dns-node-resolver
Nov 08 00:57:23.557 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 00:59:21.629 W node/node1 deleted
Nov 08 01:00:50.098 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (7 times)
Nov 08 01:00:50.694 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (2 times)
Nov 08 01:02:46.440 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (5 times)
Nov 08 01:02:47.035 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (5 times)
Nov 08 01:04:40.217 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (4 times)
Nov 08 01:04:40.811 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (21 times)
Nov 08 01:05:47.238 W ns/openshift-image-registry pod/node-ca-6pw92 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:05:47.238 W ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 01:05:47.244 W ns/openshift-dns pod/dns-default-r6pkg node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:05:47.245 I ns/openshift-image-registry pod/node-ca-6pw92 Marking for deletion Pod openshift-image-registry/node-ca-6pw92
Nov 08 01:05:47.252 I ns/openshift-image-registry pod/node-ca-6pw92 Stopping container node-ca
Nov 08 01:05:47.258 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Stopping container oauth-proxy
Nov 08 01:05:47.264 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-2vmfh
Nov 08 01:05:47.264 I ns/openshift-dns pod/dns-default-r6pkg Marking for deletion Pod openshift-dns/dns-default-r6pkg
Nov 08 01:05:47.267 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Stopping container machine-config-daemon
Nov 08 01:05:47.280 I ns/openshift-dns pod/dns-default-r6pkg Stopping container dns-node-resolver
Nov 08 01:05:47.282 I ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-2vmfh
Nov 08 01:05:47.282 I ns/openshift-dns daemonset/dns-default Deleted pod: dns-default-r6pkg
Nov 08 01:05:47.294 I ns/openshift-dns pod/dns-default-r6pkg Stopping container dns
Nov 08 01:05:48.404 E ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:05:49.401 W ns/openshift-machine-config-operator pod/machine-config-daemon-2vmfh node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:05:49.428 W ns/openshift-image-registry pod/node-ca-6pw92 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:05:56.580 W ns/openshift-dns pod/dns-default-r6pkg node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:06:17.285 I ns/openshift-dns pod/dns-default-qg84m node/ created
Nov 08 01:06:17.285 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc node/ created
Nov 08 01:06:17.316 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-k7kgc
Nov 08 01:06:17.325 I ns/openshift-dns pod/dns-default-qg84m Successfully assigned openshift-dns/dns-default-qg84m to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:06:17.326 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Successfully assigned openshift-machine-config-operator/machine-config-daemon-k7kgc to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:06:17.328 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-qg84m
Nov 08 01:06:17.370 I ns/openshift-image-registry pod/node-ca-nvrdw node/ created
Nov 08 01:06:17.380 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-nvrdw
Nov 08 01:06:17.399 I ns/openshift-image-registry pod/node-ca-nvrdw Successfully assigned openshift-image-registry/node-ca-nvrdw to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:06:17.465 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 01:06:18.463 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 01:06:18.610 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Created container machine-config-daemon
Nov 08 01:06:18.645 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Started container machine-config-daemon
Nov 08 01:06:18.651 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 01:06:18.842 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Created container oauth-proxy
Nov 08 01:06:18.876 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Started container oauth-proxy
Nov 08 01:06:25.738 I ns/openshift-image-registry pod/node-ca-nvrdw Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 01:06:25.901 I ns/openshift-image-registry pod/node-ca-nvrdw Created container node-ca
Nov 08 01:06:25.932 I ns/openshift-image-registry pod/node-ca-nvrdw Started container node-ca
Nov 08 01:06:26.040 I ns/openshift-dns pod/dns-default-qg84m Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 01:06:26.208 I ns/openshift-dns pod/dns-default-qg84m Created container dns
Nov 08 01:06:26.239 I ns/openshift-dns pod/dns-default-qg84m Started container dns
Nov 08 01:06:26.246 I ns/openshift-dns pod/dns-default-qg84m Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 01:06:26.395 I ns/openshift-dns pod/dns-default-qg84m Created container dns-node-resolver
Nov 08 01:06:26.427 I ns/openshift-dns pod/dns-default-qg84m Started container dns-node-resolver
Nov 08 01:06:42.961 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 01:08:30.806 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "" to "OperatorSyncDegraded: failed syncing configuration objects: config maps [v4-0-config-user-idp-0-ca] in openshift-authentication not synced"
Nov 08 01:08:30.864 I ns/openshift-authentication-operator deployment/authentication-operator Created ConfigMap/v4-0-config-user-idp-0-ca -n openshift-authentication because it was missing
Nov 08 01:08:33.862 I ns/openshift-authentication-operator deployment/authentication-operator Updated ConfigMap/v4-0-config-system-cliconfig -n openshift-authentication: cause by changes in data.v4-0-config-system-cliconfig
Nov 08 01:08:34.492 I ns/openshift-authentication-operator deployment/authentication-operator Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed
Nov 08 01:08:34.516 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-5c56f5fddd to 1
Nov 08 01:08:34.545 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn node/ created
Nov 08 01:08:34.560 I ns/openshift-authentication replicaset/oauth-openshift-5c56f5fddd Created pod: oauth-openshift-5c56f5fddd-x4vdn
Nov 08 01:08:34.571 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Successfully assigned openshift-authentication/oauth-openshift-5c56f5fddd-x4vdn to ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal
Nov 08 01:08:34.922 W clusteroperator/authentication changed Progressing to True: ProgressingOAuthServerDeploymentNotReady: Progressing: deployment's observed generation did not reach the expected generation
Nov 08 01:08:34.930 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "OperatorSyncDegraded: failed syncing configuration objects: config maps [v4-0-config-user-idp-0-ca] in openshift-authentication not synced" to "",Progressing changed from False to True ("Progressing: deployment's observed generation did not reach the expected generation")
Nov 08 01:08:38.324 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing message changed from "Progressing: deployment's observed generation did not reach the expected generation" to "Progressing: not all deployment replicas are ready"
Nov 08 01:08:42.422 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:748d8c1c889ad7aaa90dca2ede7435bfd94c5af44a9dc4cb741032bddf8fd3ed"
Nov 08 01:08:45.997 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:748d8c1c889ad7aaa90dca2ede7435bfd94c5af44a9dc4cb741032bddf8fd3ed"
Nov 08 01:08:46.244 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Created container oauth-openshift
Nov 08 01:08:46.308 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Started container oauth-openshift
Nov 08 01:08:50.231 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-b9988c676 to 1
Nov 08 01:08:50.235 W ns/openshift-authentication pod/oauth-openshift-b9988c676-c6ht4 node/ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:08:50.250 I ns/openshift-authentication replicaset/oauth-openshift-b9988c676 Deleted pod: oauth-openshift-b9988c676-c6ht4
Nov 08 01:08:50.262 I ns/openshift-authentication pod/oauth-openshift-b9988c676-c6ht4 Stopping container oauth-openshift
Nov 08 01:08:50.266 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-5c56f5fddd to 2
Nov 08 01:08:50.315 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb node/ created
Nov 08 01:08:50.339 I ns/openshift-authentication replicaset/oauth-openshift-5c56f5fddd Created pod: oauth-openshift-5c56f5fddd-lrpqb
Nov 08 01:08:50.349 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb Successfully assigned openshift-authentication/oauth-openshift-5c56f5fddd-lrpqb to ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal
Nov 08 01:08:52.307 W ns/openshift-authentication pod/oauth-openshift-b9988c676-c6ht4 node/ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:08:58.193 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:748d8c1c889ad7aaa90dca2ede7435bfd94c5af44a9dc4cb741032bddf8fd3ed" already present on machine
Nov 08 01:08:58.425 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb Created container oauth-openshift
Nov 08 01:08:58.462 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb Started container oauth-openshift
Nov 08 01:09:05.580 W ns/openshift-authentication pod/oauth-openshift-b9988c676-cfhgl node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:09:05.582 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-b9988c676 to 0
Nov 08 01:09:05.601 I ns/openshift-authentication replicaset/oauth-openshift-b9988c676 Deleted pod: oauth-openshift-b9988c676-cfhgl
Nov 08 01:09:05.609 I ns/openshift-authentication pod/oauth-openshift-b9988c676-cfhgl Stopping container oauth-openshift
Nov 08 01:09:06.866 W clusteroperator/authentication changed Progressing to False
Nov 08 01:09:06.873 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing changed from True to False ("")
Nov 08 01:09:07.161 W ns/openshift-authentication pod/oauth-openshift-b9988c676-cfhgl node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 01:09:07.161 W ns/openshift-authentication pod/oauth-openshift-b9988c676-cfhgl node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal container=oauth-openshift container stopped being ready
Nov 08 01:09:08.187 W ns/openshift-authentication pod/oauth-openshift-b9988c676-cfhgl node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:09:28.537 I ns/openshift-authentication-operator deployment/authentication-operator Deleted target configmap openshift-authentication/v4-0-config-user-idp-0-ca because source config does not exist
Nov 08 01:09:29.730 I ns/openshift-authentication-operator deployment/authentication-operator Updated ConfigMap/v4-0-config-system-cliconfig -n openshift-authentication: cause by changes in data.v4-0-config-system-cliconfig (2 times)
Nov 08 01:09:30.566 I ns/openshift-authentication-operator deployment/authentication-operator Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed (2 times)
Nov 08 01:09:30.588 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-76c6997bdf to 1
Nov 08 01:09:30.615 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-sqshs node/ created
Nov 08 01:09:30.626 I ns/openshift-authentication replicaset/oauth-openshift-76c6997bdf Created pod: oauth-openshift-76c6997bdf-sqshs
Nov 08 01:09:30.658 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-sqshs Successfully assigned openshift-authentication/oauth-openshift-76c6997bdf-sqshs to ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal
Nov 08 01:09:30.988 W clusteroperator/authentication changed Progressing to True: ProgressingOAuthServerDeploymentNotReady: Progressing: deployment's observed generation did not reach the expected generation
Nov 08 01:09:30.995 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing changed from False to True ("Progressing: deployment's observed generation did not reach the expected generation")
Nov 08 01:09:34.402 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing message changed from "Progressing: deployment's observed generation did not reach the expected generation" to "Progressing: not all deployment replicas are ready" (2 times)
Nov 08 01:09:38.602 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-sqshs Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:748d8c1c889ad7aaa90dca2ede7435bfd94c5af44a9dc4cb741032bddf8fd3ed" already present on machine
Nov 08 01:09:38.834 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-sqshs Created container oauth-openshift
Nov 08 01:09:38.873 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-sqshs Started container oauth-openshift
Nov 08 01:09:47.962 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-5c56f5fddd to 1
Nov 08 01:09:47.976 W ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:09:47.998 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb Stopping container oauth-openshift
Nov 08 01:09:47.998 I ns/openshift-authentication replicaset/oauth-openshift-5c56f5fddd Deleted pod: oauth-openshift-5c56f5fddd-lrpqb
Nov 08 01:09:48.013 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-76c6997bdf to 2
Nov 08 01:09:48.083 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-pwj4h node/ created
Nov 08 01:09:48.102 I ns/openshift-authentication replicaset/oauth-openshift-76c6997bdf Created pod: oauth-openshift-76c6997bdf-pwj4h
Nov 08 01:09:48.111 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-pwj4h Successfully assigned openshift-authentication/oauth-openshift-76c6997bdf-pwj4h to ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal
Nov 08 01:09:49.491 W ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 08 01:09:49.491 W ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal container=oauth-openshift container stopped being ready
Nov 08 01:09:50.503 W ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-lrpqb node/ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:09:50.718 I ns/openshift-sdn pod/sdn-vz9b4 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:26e239b1b6588f0a3bdfd921bb1e816f9d64e7a400fc334817521f3d9c67d2ce" already present on machine (2 times)
Nov 08 01:09:50.731 E ns/openshift-sdn pod/sdn-vz9b4 node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal container=sdn container exited with code 255 (Error): se\nI1108 01:09:42.741181    5574 vnids.go:162] Dissociate netid 6677712 from namespace "e2e-pv-6055"\nI1108 01:09:43.435269    5574 vnids.go:148] Associate netid 3999694 to namespace "e2e-taint-single-pod-1557" with mcEnabled false\nI1108 01:09:47.932752    5574 roundrobin.go:270] LoadBalancerRR: Setting endpoints for openshift-authentication/oauth-openshift:https to [10.128.0.48:6443 10.129.0.39:6443 10.130.0.44:6443]\nI1108 01:09:47.932894    5574 roundrobin.go:218] Delete endpoint 10.128.0.48:6443 for service "openshift-authentication/oauth-openshift:https"\nI1108 01:09:47.932994    5574 proxy.go:334] hybrid proxy: syncProxyRules start\nI1108 01:09:47.995758    5574 roundrobin.go:270] LoadBalancerRR: Setting endpoints for openshift-authentication/oauth-openshift:https to [10.128.0.48:6443 10.130.0.44:6443]\nI1108 01:09:47.995924    5574 roundrobin.go:218] Delete endpoint 10.129.0.39:6443 for service "openshift-authentication/oauth-openshift:https"\nI1108 01:09:48.191204    5574 proxy.go:337] hybrid proxy: mainProxy.syncProxyRules complete\nI1108 01:09:48.302814    5574 proxier.go:371] userspace proxy: processing 0 service events\nI1108 01:09:48.302915    5574 proxier.go:350] userspace syncProxyRules took 111.571773ms\nI1108 01:09:48.302935    5574 proxy.go:340] hybrid proxy: unidlingProxy.syncProxyRules complete\nI1108 01:09:48.302955    5574 proxy.go:334] hybrid proxy: syncProxyRules start\nI1108 01:09:48.581459    5574 proxy.go:337] hybrid proxy: mainProxy.syncProxyRules complete\nI1108 01:09:48.679497    5574 proxier.go:371] userspace proxy: processing 0 service events\nI1108 01:09:48.679528    5574 proxier.go:350] userspace syncProxyRules took 98.037241ms\nI1108 01:09:48.679539    5574 proxy.go:340] hybrid proxy: unidlingProxy.syncProxyRules complete\nI1108 01:09:49.810096    5574 ovs.go:169] Error executing ovs-ofctl: ovs-ofctl: /var/run/openvswitch/br0.mgmt: failed to open socket (Broken pipe)\nF1108 01:09:49.810246    5574 healthcheck.go:99] SDN healthcheck detected unhealthy OVS server, restarting: plugin is not setup\n
Nov 08 01:09:50.768 W clusteroperator/network changed Progressing to True: Deploying: DaemonSet "openshift-sdn/sdn" is not available (awaiting 1 nodes)
Nov 08 01:09:51.067 I ns/openshift-sdn pod/sdn-vz9b4 Created container sdn (3 times)
Nov 08 01:09:51.123 I ns/openshift-sdn pod/sdn-vz9b4 Started container sdn (3 times)
Nov 08 01:09:51.726 W ns/openshift-sdn pod/sdn-vz9b4 node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal container=sdn container restarted
Nov 08 01:09:52.820 I node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking.
Nov 08 01:09:54.028 W ns/openshift-ingress-operator pod/ingress-operator-7dd5f4bb6d-9pw8q network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network
Nov 08 01:09:54.438 I node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal status is now: NodeNotReady
Nov 08 01:09:54.442 W node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 01:09:54.491 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator Status for clusteroperator/kube-scheduler changed: Degraded message changed from "NodeControllerDegraded: All master node(s) are ready" to "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready"
Nov 08 01:09:54.524 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Status for clusteroperator/kube-apiserver changed: Degraded message changed from "NodeControllerDegraded: All master node(s) are ready" to "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready"
Nov 08 01:09:54.542 I ns/openshift-kube-controller-manager-operator deployment/kube-controller-manager-operator Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeControllerDegraded: All master node(s) are ready" to "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready"
Nov 08 01:09:54.542 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Status for clusteroperator/kube-apiserver changed: Degraded message changed from "NodeControllerDegraded: All master node(s) are ready" to "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready" (2 times)
Nov 08 01:09:55.328 I ns/openshift-machine-api machine/ci-op--bqfmn-m-2 Updated Machine ci-op--bqfmn-m-2 (6 times)
Nov 08 01:09:56.032 W ns/openshift-ingress-operator pod/ingress-operator-7dd5f4bb6d-9pw8q network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (2 times)
Nov 08 01:09:56.291 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-pwj4h Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:748d8c1c889ad7aaa90dca2ede7435bfd94c5af44a9dc4cb741032bddf8fd3ed" already present on machine
Nov 08 01:09:56.490 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-pwj4h Created container oauth-openshift
Nov 08 01:09:56.526 I ns/openshift-authentication pod/oauth-openshift-76c6997bdf-pwj4h Started container oauth-openshift
Nov 08 01:09:57.923 W node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal node is not ready
Nov 08 01:09:58.513 W clusteroperator/network changed Progressing to False
Nov 08 01:10:03.966 W ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:10:03.973 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-5c56f5fddd to 0
Nov 08 01:10:03.987 I ns/openshift-authentication replicaset/oauth-openshift-5c56f5fddd Deleted pod: oauth-openshift-5c56f5fddd-x4vdn
Nov 08 01:10:04.033 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Stopping container oauth-openshift
Nov 08 01:10:04.467 I node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Node ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal status is now: NodeReady (2 times)
Nov 08 01:10:04.468 W node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal condition Ready changed
Nov 08 01:10:04.586 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator Status for clusteroperator/kube-scheduler changed: Degraded message changed from "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready" to "NodeControllerDegraded: All master node(s) are ready"
Nov 08 01:10:04.617 I ns/openshift-kube-controller-manager-operator deployment/kube-controller-manager-operator Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready" to "NodeControllerDegraded: All master node(s) are ready"
Nov 08 01:10:04.617 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Status for clusteroperator/kube-apiserver changed: Degraded message changed from "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready" to "NodeControllerDegraded: All master node(s) are ready"
Nov 08 01:10:04.638 I ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn Cancelling deletion of Pod openshift-authentication/oauth-openshift-5c56f5fddd-x4vdn
Nov 08 01:10:04.648 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Status for clusteroperator/kube-apiserver changed: Degraded message changed from "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready" to "NodeControllerDegraded: All master node(s) are ready" (2 times)
Nov 08 01:10:04.657 I ns/openshift-marketplace pod/marketplace-operator-c97f779cb-vks7w Cancelling deletion of Pod openshift-marketplace/marketplace-operator-c97f779cb-vks7w
Nov 08 01:10:04.657 I ns/openshift-kube-controller-manager-operator deployment/kube-controller-manager-operator Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeControllerDegraded: The master node(s) \"ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal\" not ready" to "NodeControllerDegraded: All master node(s) are ready" (2 times)
Nov 08 01:10:04.670 I ns/openshift-console pod/downloads-65b75786c-z5bsk Cancelling deletion of Pod openshift-console/downloads-65b75786c-z5bsk
Nov 08 01:10:04.678 I ns/openshift-service-ca pod/apiservice-cabundle-injector-56b9fb8cfb-n25gk Cancelling deletion of Pod openshift-service-ca/apiservice-cabundle-injector-56b9fb8cfb-n25gk
Nov 08 01:10:04.686 I ns/openshift-ingress-operator pod/ingress-operator-7dd5f4bb6d-9pw8q Cancelling deletion of Pod openshift-ingress-operator/ingress-operator-7dd5f4bb6d-9pw8q
Nov 08 01:10:04.694 I ns/openshift-machine-config-operator pod/etcd-quorum-guard-cd87b6bd-r2j64 Cancelling deletion of Pod openshift-machine-config-operator/etcd-quorum-guard-cd87b6bd-r2j64
Nov 08 01:10:04.704 I ns/openshift-image-registry pod/cluster-image-registry-operator-544ff57d87-s2j9p Cancelling deletion of Pod openshift-image-registry/cluster-image-registry-operator-544ff57d87-s2j9p
Nov 08 01:10:04.712 I ns/openshift-machine-api pod/cluster-autoscaler-operator-6f7498554b-4fsbd Cancelling deletion of Pod openshift-machine-api/cluster-autoscaler-operator-6f7498554b-4fsbd
Nov 08 01:10:04.717 I ns/openshift-console pod/console-68ff5fcdf7-zshm9 Cancelling deletion of Pod openshift-console/console-68ff5fcdf7-zshm9
Nov 08 01:10:04.722 I ns/openshift-machine-config-operator pod/machine-config-controller-855c8898cc-gwj7w Cancelling deletion of Pod openshift-machine-config-operator/machine-config-controller-855c8898cc-gwj7w
Nov 08 01:10:04.728 I ns/openshift-monitoring pod/cluster-monitoring-operator-54848f8468-wrzbb Cancelling deletion of Pod openshift-monitoring/cluster-monitoring-operator-54848f8468-wrzbb
Nov 08 01:10:04.735 I ns/openshift-cluster-storage-operator pod/cluster-storage-operator-65b4d966c4-jdcqz Cancelling deletion of Pod openshift-cluster-storage-operator/cluster-storage-operator-65b4d966c4-jdcqz
Nov 08 01:10:04.740 I ns/openshift-machine-api pod/machine-api-controllers-8587db8dc5-6gmc7 Cancelling deletion of Pod openshift-machine-api/machine-api-controllers-8587db8dc5-6gmc7
Nov 08 01:10:04.750 I ns/openshift-operator-lifecycle-manager pod/packageserver-64c74d58b4-9mhth Cancelling deletion of Pod openshift-operator-lifecycle-manager/packageserver-64c74d58b4-9mhth
Nov 08 01:10:05.416 I ns/openshift-machine-api machine/ci-op--bqfmn-m-2 Updated Machine ci-op--bqfmn-m-2 (7 times)
Nov 08 01:10:05.796 W ns/openshift-authentication pod/oauth-openshift-5c56f5fddd-x4vdn node/ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:10:14.735 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "" to "RouteHealthDegraded: failed to GET route: net/http: TLS handshake timeout"
Nov 08 01:10:16.171 W clusteroperator/authentication changed Progressing to False
Nov 08 01:10:16.177 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "RouteHealthDegraded: failed to GET route: net/http: TLS handshake timeout" to "",Progressing changed from True to False ("")
Nov 08 01:10:44.135 W ns/openshift-dns pod/dns-default-qg84m node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:10:44.149 W ns/openshift-image-registry pod/node-ca-nvrdw node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:10:44.149 W ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 01:10:44.169 I ns/openshift-dns pod/dns-default-qg84m Marking for deletion Pod openshift-dns/dns-default-qg84m
Nov 08 01:10:44.176 I ns/openshift-dns pod/dns-default-qg84m Stopping container dns
Nov 08 01:10:44.176 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-nvrdw
Nov 08 01:10:44.176 I ns/openshift-dns pod/dns-default-qg84m Stopping container dns-node-resolver
Nov 08 01:10:44.176 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-k7kgc
Nov 08 01:10:44.180 I ns/openshift-image-registry pod/node-ca-nvrdw Stopping container node-ca
Nov 08 01:10:44.188 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Stopping container machine-config-daemon
Nov 08 01:10:44.188 I ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc Stopping container oauth-proxy
Nov 08 01:10:44.193 I ns/openshift-dns daemonset/dns-default Deleted pod: dns-default-qg84m
Nov 08 01:10:44.193 I ns/openshift-image-registry pod/node-ca-nvrdw Marking for deletion Pod openshift-image-registry/node-ca-nvrdw
Nov 08 01:10:45.074 E ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:10:50.095 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (28 times)
Nov 08 01:10:50.693 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (4 times)
Nov 08 01:10:56.555 W ns/openshift-image-registry pod/node-ca-nvrdw node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:10:56.583 W ns/openshift-dns pod/dns-default-qg84m node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:10:56.617 W ns/openshift-machine-config-operator pod/machine-config-daemon-k7kgc node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:12:06.601 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 node/ created
Nov 08 01:12:06.625 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-9bhn6
Nov 08 01:12:06.625 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Successfully assigned openshift-machine-config-operator/machine-config-daemon-9bhn6 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:12:06.638 I ns/openshift-image-registry pod/node-ca-7ssdj node/ created
Nov 08 01:12:06.656 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-7ssdj
Nov 08 01:12:06.657 I ns/openshift-dns pod/dns-default-4jqdj node/ created
Nov 08 01:12:06.660 I ns/openshift-image-registry pod/node-ca-7ssdj Successfully assigned openshift-image-registry/node-ca-7ssdj to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:12:06.678 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-4jqdj
Nov 08 01:12:06.689 I ns/openshift-dns pod/dns-default-4jqdj Successfully assigned openshift-dns/dns-default-4jqdj to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:12:06.786 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 01:12:07.617 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 01:12:07.762 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Created container machine-config-daemon
Nov 08 01:12:07.795 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Started container machine-config-daemon
Nov 08 01:12:07.803 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 01:12:07.962 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Created container oauth-proxy
Nov 08 01:12:08.004 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Started container oauth-proxy
Nov 08 01:12:14.428 I ns/openshift-image-registry pod/node-ca-7ssdj Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 01:12:14.593 I ns/openshift-image-registry pod/node-ca-7ssdj Created container node-ca
Nov 08 01:12:14.625 I ns/openshift-image-registry pod/node-ca-7ssdj Started container node-ca
Nov 08 01:12:14.980 I ns/openshift-dns pod/dns-default-4jqdj Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 01:12:15.145 I ns/openshift-dns pod/dns-default-4jqdj Created container dns
Nov 08 01:12:15.175 I ns/openshift-dns pod/dns-default-4jqdj Started container dns
Nov 08 01:12:15.179 I ns/openshift-dns pod/dns-default-4jqdj Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 01:12:15.327 I ns/openshift-dns pod/dns-default-4jqdj Created container dns-node-resolver
Nov 08 01:12:15.360 I ns/openshift-dns pod/dns-default-4jqdj Started container dns-node-resolver
Nov 08 01:12:27.864 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 01:12:46.439 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (7 times)
Nov 08 01:12:47.037 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (27 times)
Nov 08 01:13:15.894 I persistentvolume/pvc-774eec49-9724-4369-88ee-6a863d160fcd googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--bqfmn-dynamic-pvc-774eec49-9724-4369-88ee-6a863d160fcd' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--bqfmn-w-d-zzcqg', resourceInUseByAnotherResource
Nov 08 01:13:25.519 I persistentvolume/pvc-774eec49-9724-4369-88ee-6a863d160fcd googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--bqfmn-dynamic-pvc-774eec49-9724-4369-88ee-6a863d160fcd' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--bqfmn-w-d-zzcqg', resourceInUseByAnotherResource (2 times)
Nov 08 01:14:40.218 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (6 times)
Nov 08 01:14:40.813 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (5 times)
Nov 08 01:16:19.752 I persistentvolume/pvc-240ff9d7-fd1e-425d-b4e1-abd959180853 googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--bqfmn-dynamic-pvc-240ff9d7-fd1e-425d-b4e1-abd959180853' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--bqfmn-w-d-zzcqg', resourceInUseByAnotherResource
Nov 08 01:16:25.609 I persistentvolume/pvc-240ff9d7-fd1e-425d-b4e1-abd959180853 googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--bqfmn-dynamic-pvc-240ff9d7-fd1e-425d-b4e1-abd959180853' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--bqfmn-w-d-zzcqg', resourceInUseByAnotherResource (2 times)
Nov 08 01:17:47.208 W ns/openshift-dns pod/dns-default-4jqdj node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:17:47.210 W ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 01:17:47.211 W ns/openshift-image-registry pod/node-ca-7ssdj node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:17:47.213 I ns/openshift-image-registry pod/node-ca-7ssdj Marking for deletion Pod openshift-image-registry/node-ca-7ssdj
Nov 08 01:17:47.221 I ns/openshift-dns pod/dns-default-4jqdj Stopping container dns
Nov 08 01:17:47.235 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-9bhn6
Nov 08 01:17:47.236 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-7ssdj
Nov 08 01:17:47.241 I ns/openshift-dns pod/dns-default-4jqdj Stopping container dns-node-resolver
Nov 08 01:17:47.241 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Stopping container machine-config-daemon
Nov 08 01:17:47.251 I ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 Stopping container oauth-proxy
Nov 08 01:17:47.268 I ns/openshift-dns pod/dns-default-4jqdj Marking for deletion Pod openshift-dns/dns-default-4jqdj
Nov 08 01:17:47.268 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-9bhn6
Nov 08 01:17:47.268 I ns/openshift-image-registry pod/node-ca-7ssdj Stopping container node-ca
Nov 08 01:17:48.220 E ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:17:56.557 W ns/openshift-machine-config-operator pod/machine-config-daemon-9bhn6 node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:17:56.583 W ns/openshift-image-registry pod/node-ca-7ssdj node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:17:56.610 W ns/openshift-dns pod/dns-default-4jqdj node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:17:56.691 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg node/ created
Nov 08 01:17:56.711 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-gndjg
Nov 08 01:17:56.713 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Successfully assigned openshift-machine-config-operator/machine-config-daemon-gndjg to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:17:56.724 I ns/openshift-image-registry pod/node-ca-q2t4j node/ created
Nov 08 01:17:56.734 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-q2t4j
Nov 08 01:17:56.746 I ns/openshift-image-registry pod/node-ca-q2t4j Successfully assigned openshift-image-registry/node-ca-q2t4j to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:17:56.750 I ns/openshift-dns pod/dns-default-vn89b node/ created
Nov 08 01:17:56.762 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-vn89b
Nov 08 01:17:56.768 I ns/openshift-dns pod/dns-default-vn89b Successfully assigned openshift-dns/dns-default-vn89b to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:17:56.891 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 01:17:57.501 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 01:17:57.689 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Created container machine-config-daemon
Nov 08 01:17:57.727 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Started container machine-config-daemon
Nov 08 01:17:57.734 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 01:17:57.892 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Created container oauth-proxy
Nov 08 01:17:57.929 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Started container oauth-proxy
Nov 08 01:18:04.590 I ns/openshift-image-registry pod/node-ca-q2t4j Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 01:18:04.755 I ns/openshift-image-registry pod/node-ca-q2t4j Created container node-ca
Nov 08 01:18:04.797 I ns/openshift-image-registry pod/node-ca-q2t4j Started container node-ca
Nov 08 01:18:05.277 I ns/openshift-dns pod/dns-default-vn89b Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 01:18:05.455 I ns/openshift-dns pod/dns-default-vn89b Created container dns
Nov 08 01:18:05.539 I ns/openshift-dns pod/dns-default-vn89b Started container dns
Nov 08 01:18:05.577 I ns/openshift-dns pod/dns-default-vn89b Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 01:18:05.672 I ns/openshift-dns pod/dns-default-vn89b Created container dns-node-resolver
Nov 08 01:18:05.719 I ns/openshift-dns pod/dns-default-vn89b Started container dns-node-resolver
Nov 08 01:18:17.347 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 01:20:11.463 W ns/openshift-image-registry pod/node-ca-q2t4j node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:20:11.469 W ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 600s
Nov 08 01:20:11.484 W ns/openshift-dns pod/dns-default-vn89b node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 30s
Nov 08 01:20:11.491 I ns/openshift-image-registry pod/node-ca-q2t4j Stopping container node-ca
Nov 08 01:20:11.491 I ns/openshift-image-registry pod/node-ca-q2t4j Marking for deletion Pod openshift-image-registry/node-ca-q2t4j
Nov 08 01:20:11.491 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-q2t4j
Nov 08 01:20:11.500 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Stopping container oauth-proxy
Nov 08 01:20:11.519 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-gndjg
Nov 08 01:20:11.519 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-gndjg
Nov 08 01:20:11.519 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Stopping container machine-config-daemon
Nov 08 01:20:11.523 I ns/openshift-dns pod/dns-default-vn89b Stopping container dns
Nov 08 01:20:11.531 I ns/openshift-dns pod/dns-default-vn89b Stopping container dns-node-resolver
Nov 08 01:20:11.534 I ns/openshift-dns pod/dns-default-vn89b Marking for deletion Pod openshift-dns/dns-default-vn89b
Nov 08 01:20:11.551 I ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg Cancelling deletion of Pod openshift-machine-config-operator/machine-config-daemon-gndjg
Nov 08 01:20:11.561 I ns/openshift-dns pod/dns-default-vn89b Cancelling deletion of Pod openshift-dns/dns-default-vn89b
Nov 08 01:20:11.567 I ns/openshift-image-registry pod/node-ca-q2t4j Cancelling deletion of Pod openshift-image-registry/node-ca-q2t4j
Nov 08 01:20:12.598 E ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): 
Nov 08 01:20:12.681 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available.
Nov 08 01:20:16.560 W ns/openshift-machine-config-operator pod/machine-config-daemon-gndjg node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:20:16.580 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 node/ created
Nov 08 01:20:16.595 I ns/openshift-machine-config-operator daemonset/machine-config-daemon (combined from similar events): Created pod: machine-config-daemon-pl2q6
Nov 08 01:20:16.599 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Successfully assigned openshift-machine-config-operator/machine-config-daemon-pl2q6 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:20:16.602 W ns/openshift-image-registry pod/node-ca-q2t4j node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:20:16.635 W ns/openshift-dns pod/dns-default-vn89b node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted
Nov 08 01:20:16.654 I ns/openshift-image-registry pod/node-ca-rz6j6 node/ created
Nov 08 01:20:16.670 I ns/openshift-image-registry daemonset/node-ca (combined from similar events): Created pod: node-ca-rz6j6
Nov 08 01:20:16.672 I ns/openshift-image-registry pod/node-ca-rz6j6 Successfully assigned openshift-image-registry/node-ca-rz6j6 to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:20:16.678 I ns/openshift-dns pod/dns-default-6fhxc node/ created
Nov 08 01:20:16.696 I ns/openshift-dns pod/dns-default-6fhxc Successfully assigned openshift-dns/dns-default-6fhxc to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:20:16.698 I ns/openshift-dns daemonset/dns-default (combined from similar events): Created pod: dns-default-6fhxc
Nov 08 01:20:17.386 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:076edf7d71cc9dea037972fafa0d7a25c5cbb827a2a3bdbce97010657e751cb6" already present on machine
Nov 08 01:20:17.575 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Created container machine-config-daemon
Nov 08 01:20:17.614 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Started container machine-config-daemon
Nov 08 01:20:17.621 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4f711682524b64033a4ce9193bd2cd9621e08b4bae44cffffb4e397a43a8f945" already present on machine
Nov 08 01:20:17.803 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Created container oauth-proxy
Nov 08 01:20:17.841 I ns/openshift-machine-config-operator pod/machine-config-daemon-pl2q6 Started container oauth-proxy
Nov 08 01:20:24.888 I ns/openshift-dns pod/dns-default-6fhxc Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:614764223d37c466a211f456a9d02c5881d947e1f05d541689dc34a3ec19acf2" already present on machine
Nov 08 01:20:24.940 I ns/openshift-image-registry pod/node-ca-rz6j6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3191fc23d0d7e820100498c5085592219a747743f06fc059436945df6ab3e8ee" already present on machine
Nov 08 01:20:25.086 I ns/openshift-dns pod/dns-default-6fhxc Created container dns
Nov 08 01:20:25.119 I ns/openshift-dns pod/dns-default-6fhxc Started container dns
Nov 08 01:20:25.126 I ns/openshift-dns pod/dns-default-6fhxc Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cd150f3e4a29d160fae9610f1096180eaeb29ff29b63bf490b0022b4d3913e29" already present on machine
Nov 08 01:20:25.134 I ns/openshift-image-registry pod/node-ca-rz6j6 Created container node-ca
Nov 08 01:20:25.172 I ns/openshift-image-registry pod/node-ca-rz6j6 Started container node-ca
Nov 08 01:20:25.303 I ns/openshift-dns pod/dns-default-6fhxc Created container dns-node-resolver
Nov 08 01:20:25.337 I ns/openshift-dns pod/dns-default-6fhxc Started container dns-node-resolver
Nov 08 01:20:35.315 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal
Nov 08 01:20:50.096 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (9 times)
Nov 08 01:20:50.694 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (6 times)
Nov 08 01:22:46.440 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (30 times)
Nov 08 01:22:47.034 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (32 times)
Nov 08 01:24:40.219 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (31 times)
Nov 08 01:24:41.014 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--bqfmn-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-009" not found) (7 times)
Nov 08 01:29:24.782 I ns/kube-system pod/pod0-system-node-critical node/ created
Nov 08 01:29:24.794 I ns/kube-system pod/pod1-system-cluster-critical node/ created
Nov 08 01:29:24.800 W ns/kube-system pod/pod1-system-cluster-critical node/ graceful deletion within 0s
Nov 08 01:29:24.806 I ns/kube-system pod/pod0-system-node-critical Successfully assigned kube-system/pod0-system-node-critical to ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal
Nov 08 01:29:24.808 W ns/kube-system pod/pod1-system-cluster-critical node/ deleted
Nov 08 01:29:24.816 W ns/kube-system pod/pod0-system-node-critical node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal graceful deletion within 0s
Nov 08 01:29:24.816 W ns/kube-system pod/pod1-system-cluster-critical skip schedule deleting pod: kube-system/pod1-system-cluster-critical
Nov 08 01:29:24.816 W ns/kube-system pod/pod1-system-cluster-critical Binding rejected: Operation cannot be fulfilled on pods/binding "pod1-system-cluster-critical": pod pod1-system-cluster-critical is being deleted, cannot be assigned to a host
Nov 08 01:29:24.819 W ns/kube-system pod/pod0-system-node-critical node/ci-op--bqfmn-w-d-zzcqg.c.openshift-gce-devel-ci.internal deleted


				Click to see stdout/stderrfrom junit_e2e_20191108-013005.xml

Find was mentions in log files