Result | SUCCESS |
Tests | 1 failed / 57 succeeded |
Started | |
Elapsed | 1h44m |
Work namespace | ci-op-7nwf2qjz |
pod | 4.3.0-0.ci-2019-11-06-170148-gcp-serial |
go run hack/e2e.go -v -test --test_args='--ginkgo.focus=openshift\-tests\sMonitor\scluster\swhile\stests\sexecute$'
24 error level events were detected during this test run: Nov 06 17:45:55.784 E ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nrvcq node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=prometheus-adapter container exited with code 2 (Error): I1106 17:30:29.966258 1 adapter.go:93] successfully using in-cluster auth\nI1106 17:30:30.432441 1 secure_serving.go:116] Serving securely on [::]:6443\n Nov 06 17:45:55.847 E ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 17:45:56.943 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 17:30:05 Watching directory: "/etc/alertmanager/config"\n Nov 06 17:45:56.943 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/06 17:30:06 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:30:06 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 17:30:06 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 17:30:06 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 17:30:06 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 17:30:06 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:30:06 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 17:30:06 http.go:96: HTTPS: listening on [::]:9095\n Nov 06 17:58:47.745 E ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 18:12:36.307 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available Nov 06 18:13:20.836 E ns/openshift-marketplace pod/certified-operators-c6448b4f6-57dtq node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=certified-operators container exited with code 2 (Error): Nov 06 18:13:20.901 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 17:29:30 Watching directory: "/etc/alertmanager/config"\n Nov 06 18:13:20.901 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/06 17:29:30 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:29:30 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 17:29:30 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 17:29:30 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 17:29:30 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 17:29:30 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:29:30 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 17:29:30 http.go:96: HTTPS: listening on [::]:9095\n Nov 06 18:13:20.937 E ns/openshift-marketplace pod/community-operators-74b7658499-gkfj7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=community-operators container exited with code 2 (Error): Nov 06 18:13:21.075 E ns/openshift-ingress pod/router-default-ffd95fb7d-d8wrr node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=router container exited with code 2 (Error): Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T17:58:46.958Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T17:58:59.833Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:21.540Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:26.537Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:39.599Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:44.586Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:51.270Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:12:57.925Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:13:02.924Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:13:10.774Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:13:15.765Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n Nov 06 18:13:22.203 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 17:46:16 Watching directory: "/etc/alertmanager/config"\n Nov 06 18:13:22.203 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/06 17:46:16 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:46:16 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 17:46:16 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 17:46:16 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 17:46:16 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 17:46:16 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:46:16 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 17:46:16 http.go:96: HTTPS: listening on [::]:9095\n Nov 06 18:13:54.348 E ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=prometheus container exited with code 1 (Error): caller=main.go:657 msg="Starting TSDB ..."\nlevel=info ts=2019-11-06T18:13:47.337Z caller=web.go:450 component=web msg="Start listening for connections" address=127.0.0.1:9090\nlevel=info ts=2019-11-06T18:13:47.342Z caller=head.go:514 component=tsdb msg="replaying WAL, this may take awhile"\nlevel=info ts=2019-11-06T18:13:47.342Z caller=head.go:562 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:672 fs_type=XFS_SUPER_MAGIC\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:673 msg="TSDB started"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:743 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:526 msg="Stopping scrape discovery manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:540 msg="Stopping notify discovery manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:562 msg="Stopping scrape manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=manager.go:814 component="rule manager" msg="Stopping rule manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=manager.go:820 component="rule manager" msg="Rule manager stopped"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:522 msg="Scrape discovery manager stopped"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:536 msg="Notify discovery manager stopped"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:556 msg="Scrape manager stopped"\nlevel=info ts=2019-11-06T18:13:47.345Z caller=notifier.go:602 component=notifier msg="Stopping notification manager..."\nlevel=info ts=2019-11-06T18:13:47.345Z caller=main.go:727 msg="Notifier manager stopped"\nlevel=error ts=2019-11-06 Nov 06 18:14:18.722 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available Nov 06 18:20:37.078 E ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 06 18:20:37.078 E ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=machine-config-daemon container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 06 18:20:37.130 E ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=router container exited with code 2 (Error): Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:40.526Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:45.525Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:53.721Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:58.710Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:10.999Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:15.995Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:28.495Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:33.490Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:19:14.929Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:19:20.102Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:20:35.711Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n Nov 06 18:20:37.186 E ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=prometheus-adapter container exited with code 2 (Error): I1106 18:13:32.468020 1 adapter.go:93] successfully using in-cluster auth\nI1106 18:13:33.638890 1 secure_serving.go:116] Serving securely on [::]:6443\n Nov 06 18:20:37.226 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): -monitoring:alertmanager-main\n2019/11/06 18:13:54 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 18:13:54 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 18:13:54 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 18:13:54 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 18:13:54 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 18:13:54 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 18:13:54 http.go:96: HTTPS: listening on [::]:9095\n2019/11/06 18:13:57 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:13:59 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:00 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:02 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:02 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:04 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:05 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:07 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:09 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:12 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:14 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n Nov 06 18:20:37.226 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 18:13:54 Watching directory: "/etc/alertmanager/config"\n Nov 06 18:32:43.369 E ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 18:32:44.415 E ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=node-ca container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 06 18:35:52.977 E ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error):from junit_e2e_20191106-183713.xml
Timeline: Nov 06 17:42:49.584 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (16 times) Nov 06 17:42:49.588 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (17 times) Nov 06 17:44:22.138 W ns/openshift-sdn pod/ovs-7hmk5 Liveness probe failed: ovs-ofctl: talking to br0 (Broken pipe)\n Nov 06 17:44:41.308 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (8 times) Nov 06 17:44:41.316 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (9 times) Nov 06 17:44:41.324 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-001" not found) (10 times) Nov 06 17:44:41.700 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-002" not found) (3 times) Nov 06 17:44:42.099 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-003" not found) (3 times) Nov 06 17:45:54.799 W ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 17:45:54.799 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nrvcq Marking for deletion Pod openshift-monitoring/prometheus-adapter-74ccdcf5b7-nrvcq Nov 06 17:45:54.815 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-bwvbx Nov 06 17:45:54.815 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg Marking for deletion Pod openshift-monitoring/thanos-querier-6fcc94d644-t9jkg Nov 06 17:45:54.815 I ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx Stopping container machine-config-daemon Nov 06 17:45:54.817 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nrvcq node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:45:54.818 W ns/openshift-image-registry pod/node-ca-bkflb node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:45:54.820 W ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:45:54.821 W ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:45:54.821 W ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 120s Nov 06 17:45:54.822 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:45:54.832 I ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx Stopping container oauth-proxy Nov 06 17:45:54.838 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-bkflb Nov 06 17:45:54.838 I ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 Marking for deletion Pod openshift-monitoring/grafana-5b5648fddf-jl7x5 Nov 06 17:45:54.847 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nrvcq Stopping container prometheus-adapter Nov 06 17:45:54.854 I ns/openshift-monitoring pod/alertmanager-main-1 Marking for deletion Pod openshift-monitoring/alertmanager-main-1 Nov 06 17:45:54.863 I ns/openshift-image-registry pod/node-ca-bkflb Stopping container node-ca Nov 06 17:45:54.868 I ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-bwvbx Nov 06 17:45:54.868 I ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 Stopping container grafana-proxy Nov 06 17:45:54.874 I ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 Stopping container grafana Nov 06 17:45:54.875 I ns/openshift-image-registry pod/node-ca-bkflb Marking for deletion Pod openshift-image-registry/node-ca-bkflb Nov 06 17:45:54.885 I ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 Marking for deletion Pod openshift-ingress/router-default-ffd95fb7d-tqv24 Nov 06 17:45:54.885 I ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 Stopping container router Nov 06 17:45:54.891 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager-proxy Nov 06 17:45:54.900 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container config-reloader Nov 06 17:45:54.906 I ns/openshift-ingress pod/router-default-ffd95fb7d-6bfzp node/ created Nov 06 17:45:54.907 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn node/ created Nov 06 17:45:54.917 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 node/ created Nov 06 17:45:54.922 I ns/openshift-ingress replicaset/router-default-ffd95fb7d Created pod: router-default-ffd95fb7d-6bfzp Nov 06 17:45:54.932 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager Nov 06 17:45:54.939 I ns/openshift-monitoring replicaset/prometheus-adapter-74ccdcf5b7 Created pod: prometheus-adapter-74ccdcf5b7-tdhpn Nov 06 17:45:54.948 I ns/openshift-monitoring replicaset/grafana-5b5648fddf Created pod: grafana-5b5648fddf-xp2w8 Nov 06 17:45:54.953 I ns/openshift-ingress pod/router-default-ffd95fb7d-6bfzp Successfully assigned openshift-ingress/router-default-ffd95fb7d-6bfzp to ci-op--kg4jd-w-b-lnz6x.c.openshift-gce-devel-ci.internal Nov 06 17:45:54.953 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn Successfully assigned openshift-monitoring/prometheus-adapter-74ccdcf5b7-tdhpn to ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Nov 06 17:45:54.967 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Successfully assigned openshift-monitoring/grafana-5b5648fddf-xp2w8 to ci-op--kg4jd-w-b-lnz6x.c.openshift-gce-devel-ci.internal Nov 06 17:45:55.016 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ created Nov 06 17:45:55.025 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg Stopping container prom-label-proxy Nov 06 17:45:55.033 I ns/openshift-monitoring replicaset/thanos-querier-6fcc94d644 Created pod: thanos-querier-6fcc94d644-hjg29 Nov 06 17:45:55.053 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Successfully assigned openshift-monitoring/thanos-querier-6fcc94d644-hjg29 to ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Nov 06 17:45:55.178 W clusteroperator/ingress changed Available to False: IngressUnavailable: Not all ingress controllers are available. Nov 06 17:45:55.178 W clusteroperator/ingress changed Progressing to True: Reconciling: Not all ingress controllers are available. Nov 06 17:45:55.213 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg Stopping container kube-rbac-proxy Nov 06 17:45:55.414 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg Stopping container oauth-proxy Nov 06 17:45:55.611 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg Stopping container thanos-querier Nov 06 17:45:55.784 E ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nrvcq node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=prometheus-adapter container exited with code 2 (Error): I1106 17:30:29.966258 1 adapter.go:93] successfully using in-cluster auth\nI1106 17:30:30.432441 1 secure_serving.go:116] Serving securely on [::]:6443\n Nov 06 17:45:55.847 E ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 17:45:56.888 W ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 17:45:56.888 W ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=router container stopped being ready Nov 06 17:45:56.920 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 17:45:56.920 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready Nov 06 17:45:56.920 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready Nov 06 17:45:56.920 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready Nov 06 17:45:56.920 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=thanos-querier container stopped being ready Nov 06 17:45:56.932 W ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 17:45:56.932 W ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=grafana container stopped being ready Nov 06 17:45:56.932 W ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=grafana-proxy container stopped being ready Nov 06 17:45:56.943 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 17:30:05 Watching directory: "/etc/alertmanager/config"\n Nov 06 17:45:56.943 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/06 17:30:06 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:30:06 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 17:30:06 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 17:30:06 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 17:30:06 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 17:30:06 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:30:06 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 17:30:06 http.go:96: HTTPS: listening on [::]:9095\n Nov 06 17:45:56.973 W ns/openshift-image-registry pod/node-ca-bkflb node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 17:45:56.973 W ns/openshift-image-registry pod/node-ca-bkflb node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready Nov 06 17:46:01.321 W ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 17:46:01.321 W ns/openshift-image-registry pod/node-ca-bkflb node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 17:46:01.321 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 17:46:01.321 W ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 17:46:02.436 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:832221e63bc0b28b204f04a861c773b7f808afd7b18922a6983e5f95fbeb7ae3" Nov 06 17:46:02.947 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:2b9dabef8ad692d143307c77162da685f6bdfe94be7f09010a64b73400c96258" already present on machine Nov 06 17:46:03.037 I ns/openshift-ingress pod/router-default-ffd95fb7d-6bfzp Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:bd70de74d6dc01dedf586047151a4e6f4116a04f853d3fce65f353726a0c64e3" Nov 06 17:46:03.134 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Created container thanos-querier Nov 06 17:46:03.164 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Started container thanos-querier Nov 06 17:46:03.172 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 17:46:03.340 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0018a5c222e49be417badd06bc01698272be50e0f77fb463a1d88dd0afc51306" already present on machine Nov 06 17:46:03.402 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Created container oauth-proxy Nov 06 17:46:03.438 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Started container oauth-proxy Nov 06 17:46:03.448 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 17:46:03.547 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Created container grafana Nov 06 17:46:03.605 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Started container grafana Nov 06 17:46:03.611 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 17:46:03.683 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Created container kube-rbac-proxy Nov 06 17:46:03.720 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Started container kube-rbac-proxy Nov 06 17:46:03.726 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:27f4cef3c20b57e894d34b297136eed9b2864efbbd96f57d2bed7c8441947b3d" already present on machine Nov 06 17:46:03.904 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Created container grafana-proxy Nov 06 17:46:03.947 I ns/openshift-monitoring pod/grafana-5b5648fddf-xp2w8 Started container grafana-proxy Nov 06 17:46:03.954 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Created container prom-label-proxy Nov 06 17:46:03.994 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Started container prom-label-proxy Nov 06 17:46:05.416 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:832221e63bc0b28b204f04a861c773b7f808afd7b18922a6983e5f95fbeb7ae3" Nov 06 17:46:05.587 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn Created container prometheus-adapter Nov 06 17:46:05.618 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn Started container prometheus-adapter Nov 06 17:46:06.027 I ns/openshift-ingress pod/router-default-ffd95fb7d-6bfzp Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:bd70de74d6dc01dedf586047151a4e6f4116a04f853d3fce65f353726a0c64e3" Nov 06 17:46:06.226 I ns/openshift-ingress pod/router-default-ffd95fb7d-6bfzp Created container router Nov 06 17:46:06.256 I ns/openshift-ingress pod/router-default-ffd95fb7d-6bfzp Started container router Nov 06 17:46:07.600 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nrvcq node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.622 W ns/openshift-ingress pod/router-default-ffd95fb7d-tqv24 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.650 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-t9jkg node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.667 W ns/openshift-machine-config-operator pod/machine-config-daemon-bwvbx node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.684 W ns/openshift-image-registry pod/node-ca-bkflb node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.709 W ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.730 W ns/openshift-monitoring pod/grafana-5b5648fddf-jl7x5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:46:07.755 I ns/openshift-monitoring pod/alertmanager-main-1 node/ created Nov 06 17:46:07.763 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-1 in StatefulSet alertmanager-main successful Nov 06 17:46:07.773 I ns/openshift-monitoring pod/alertmanager-main-1 Successfully assigned openshift-monitoring/alertmanager-main-1 to ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Nov 06 17:46:15.952 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:4a4d954ea2a2f7eff16f8c11d2b808286ff722c878d4d05dccf4d545580f1d3a" already present on machine Nov 06 17:46:16.142 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager Nov 06 17:46:16.175 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager Nov 06 17:46:16.182 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:63ff8d9f806970af770724d4ba8bc521885de81e3e22212042a8b770a83f1b94" already present on machine Nov 06 17:46:16.644 I ns/openshift-monitoring pod/alertmanager-main-1 Created container config-reloader Nov 06 17:46:16.678 I ns/openshift-monitoring pod/alertmanager-main-1 Started container config-reloader Nov 06 17:46:16.683 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 17:46:16.858 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager-proxy Nov 06 17:46:16.885 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager-proxy Nov 06 17:46:23.432 W clusteroperator/ingress changed Available to True: desired and current number of IngressControllers are equal Nov 06 17:46:23.432 W clusteroperator/ingress changed Progressing to False: desired and current number of IngressControllers are equal Nov 06 17:46:59.831 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk node/ created Nov 06 17:46:59.835 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-sz7vk Nov 06 17:46:59.856 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Successfully assigned openshift-machine-config-operator/machine-config-daemon-sz7vk to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 17:46:59.860 I ns/openshift-image-registry pod/node-ca-djdlp node/ created Nov 06 17:46:59.864 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-djdlp Nov 06 17:46:59.882 I ns/openshift-image-registry pod/node-ca-djdlp Successfully assigned openshift-image-registry/node-ca-djdlp to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 17:47:00.545 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" already present on machine Nov 06 17:47:00.700 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Created container machine-config-daemon Nov 06 17:47:00.735 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Started container machine-config-daemon Nov 06 17:47:00.739 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 17:47:00.934 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Created container oauth-proxy Nov 06 17:47:00.978 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Started container oauth-proxy Nov 06 17:47:08.349 I ns/openshift-image-registry pod/node-ca-djdlp Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 17:47:08.506 I ns/openshift-image-registry pod/node-ca-djdlp Created container node-ca Nov 06 17:47:08.540 I ns/openshift-image-registry pod/node-ca-djdlp Started container node-ca Nov 06 17:50:58.182 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (18 times) Nov 06 17:50:58.191 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (7 times) Nov 06 17:52:49.583 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (3 times) Nov 06 17:52:49.587 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (4 times) Nov 06 17:54:41.304 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (3 times) Nov 06 17:54:41.898 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (19 times) Nov 06 17:55:59.921 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-sz7vk Nov 06 17:55:59.921 W ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 17:55:59.923 W ns/openshift-image-registry pod/node-ca-djdlp node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:55:59.939 I ns/openshift-image-registry pod/node-ca-djdlp Marking for deletion Pod openshift-image-registry/node-ca-djdlp Nov 06 17:55:59.939 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-sz7vk Nov 06 17:55:59.939 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Stopping container oauth-proxy Nov 06 17:55:59.944 I ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk Stopping container machine-config-daemon Nov 06 17:55:59.944 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-djdlp Nov 06 17:55:59.948 I ns/openshift-image-registry pod/node-ca-djdlp Stopping container node-ca Nov 06 17:56:01.345 W ns/openshift-image-registry pod/node-ca-djdlp node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 17:56:01.345 W ns/openshift-image-registry pod/node-ca-djdlp node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready Nov 06 17:56:01.355 W ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 17:56:01.356 W ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=machine-config-daemon container stopped being ready Nov 06 17:56:01.356 W ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready Nov 06 17:56:07.624 W ns/openshift-machine-config-operator pod/machine-config-daemon-sz7vk node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:56:07.650 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 node/ created Nov 06 17:56:07.651 W ns/openshift-image-registry pod/node-ca-djdlp node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:56:07.655 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-x9mx5 Nov 06 17:56:07.670 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Successfully assigned openshift-machine-config-operator/machine-config-daemon-x9mx5 to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 17:56:07.686 I ns/openshift-image-registry pod/node-ca-j2fdh node/ created Nov 06 17:56:07.690 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-j2fdh Nov 06 17:56:07.705 I ns/openshift-image-registry pod/node-ca-j2fdh Successfully assigned openshift-image-registry/node-ca-j2fdh to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 17:56:08.449 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" already present on machine Nov 06 17:56:08.629 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Created container machine-config-daemon Nov 06 17:56:08.666 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Started container machine-config-daemon Nov 06 17:56:08.670 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 17:56:08.894 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Created container oauth-proxy Nov 06 17:56:08.922 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Started container oauth-proxy Nov 06 17:56:15.357 I ns/openshift-image-registry pod/node-ca-j2fdh Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 17:56:15.521 I ns/openshift-image-registry pod/node-ca-j2fdh Created container node-ca Nov 06 17:56:15.559 I ns/openshift-image-registry pod/node-ca-j2fdh Started container node-ca Nov 06 17:58:46.901 I ns/openshift-image-registry pod/node-ca-j2fdh Marking for deletion Pod openshift-image-registry/node-ca-j2fdh Nov 06 17:58:46.904 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-j2fdh Nov 06 17:58:46.904 I ns/openshift-image-registry pod/node-ca-j2fdh Stopping container node-ca Nov 06 17:58:46.905 W ns/openshift-image-registry pod/node-ca-j2fdh node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 17:58:46.906 W ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 17:58:46.910 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-x9mx5 Nov 06 17:58:46.920 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-x9mx5 Nov 06 17:58:46.921 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Stopping container oauth-proxy Nov 06 17:58:46.945 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Cancelling deletion of Pod openshift-machine-config-operator/machine-config-daemon-x9mx5 Nov 06 17:58:46.945 I ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 Stopping container machine-config-daemon Nov 06 17:58:47.745 E ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 17:58:57.603 W ns/openshift-machine-config-operator pod/machine-config-daemon-x9mx5 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:58:57.621 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz node/ created Nov 06 17:58:57.624 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-bswkz Nov 06 17:58:57.625 W ns/openshift-image-registry pod/node-ca-j2fdh node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 17:58:57.647 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Successfully assigned openshift-machine-config-operator/machine-config-daemon-bswkz to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 17:58:57.658 I ns/openshift-image-registry pod/node-ca-9spwh node/ created Nov 06 17:58:57.666 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-9spwh Nov 06 17:58:57.680 I ns/openshift-image-registry pod/node-ca-9spwh Successfully assigned openshift-image-registry/node-ca-9spwh to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 17:58:58.446 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" already present on machine Nov 06 17:58:58.605 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Created container machine-config-daemon Nov 06 17:58:58.647 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Started container machine-config-daemon Nov 06 17:58:58.651 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 17:58:58.878 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Created container oauth-proxy Nov 06 17:58:58.917 I ns/openshift-machine-config-operator pod/machine-config-daemon-bswkz Started container oauth-proxy Nov 06 17:59:05.582 I ns/openshift-image-registry pod/node-ca-9spwh Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 17:59:05.748 I ns/openshift-image-registry pod/node-ca-9spwh Created container node-ca Nov 06 17:59:05.784 I ns/openshift-image-registry pod/node-ca-9spwh Started container node-ca Nov 06 18:00:14.886 I ns/kube-system pod/pod0-system-node-critical node/ created Nov 06 18:00:14.895 I ns/kube-system pod/pod1-system-cluster-critical node/ created Nov 06 18:00:14.912 I ns/kube-system pod/pod0-system-node-critical Successfully assigned kube-system/pod0-system-node-critical to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:00:14.912 I ns/kube-system pod/pod1-system-cluster-critical Successfully assigned kube-system/pod1-system-cluster-critical to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:00:14.915 W ns/kube-system pod/pod1-system-cluster-critical node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:00:14.922 W ns/kube-system pod/pod1-system-cluster-critical node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 18:00:14.932 W ns/kube-system pod/pod0-system-node-critical node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:00:14.942 W ns/kube-system pod/pod0-system-node-critical node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal deleted Nov 06 18:00:58.182 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (23 times) Nov 06 18:00:58.188 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (9 times) Nov 06 18:02:17.923 W ns/kube-system pod/pod0-system-node-critical Unable to attach or mount volumes: unmounted volumes=[default-token-hc6rz], unattached volumes=[default-token-hc6rz]: timed out waiting for the condition Nov 06 18:02:17.933 W ns/kube-system pod/pod1-system-cluster-critical Unable to attach or mount volumes: unmounted volumes=[default-token-hc6rz], unattached volumes=[default-token-hc6rz]: timed out waiting for the condition Nov 06 18:02:49.589 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (28 times) Nov 06 18:02:49.590 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (6 times) Nov 06 18:03:26.250 I persistentvolume/pvc-b14fd7c1-cf14-450a-a162-5977db8d1652 googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-d/disks/ci-op--kg4jd-dynamic-pvc-b14fd7c1-cf14-450a-a162-5977db8d1652' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op--kg4jd-w-d-sr9bq', resourceInUseByAnotherResource Nov 06 18:04:41.351 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (5 times) Nov 06 18:04:42.315 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (3 times) Nov 06 18:05:58.000 W ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp requeue in: 20s Nov 06 18:05:58.762 W ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp requeue in: 20s Nov 06 18:06:00.965 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg requeue in: 20s Nov 06 18:06:03.531 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s Nov 06 18:06:04.154 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp Nov 06 18:06:04.725 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg requeue in: 20s Nov 06 18:06:05.348 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s Nov 06 18:06:06.028 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp (2 times) Nov 06 18:06:06.594 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg requeue in: 20s (2 times) Nov 06 18:06:07.259 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk Nov 06 18:06:07.946 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp (3 times) Nov 06 18:06:08.526 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg requeue in: 20s (3 times) Nov 06 18:06:09.244 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk (2 times) Nov 06 18:06:09.922 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk (3 times) Nov 06 18:06:18.649 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp (4 times) Nov 06 18:06:21.476 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg Nov 06 18:06:22.004 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg (2 times) Nov 06 18:06:23.598 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg (3 times) Nov 06 18:06:24.286 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk (4 times) Nov 06 18:06:24.802 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg (4 times) Nov 06 18:07:46.845 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal status is now: NodeHasSufficientMemory (8 times) Nov 06 18:07:46.845 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal condition NetworkUnavailable changed Nov 06 18:07:46.862 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal status is now: NodeHasNoDiskPressure (8 times) Nov 06 18:07:46.862 I ns/openshift-multus pod/multus-7jm82 node/ created Nov 06 18:07:46.872 I ns/openshift-multus daemonset/multus Created pod: multus-7jm82 Nov 06 18:07:46.890 I ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z node/ created Nov 06 18:07:46.899 I ns/openshift-sdn pod/ovs-5dgr8 node/ created Nov 06 18:07:46.902 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-ssr4z Nov 06 18:07:46.909 W clusteroperator/network changed Progressing to True: Deploying: DaemonSet "openshift-multus/multus" is not available (awaiting 1 nodes) Nov 06 18:07:46.923 I ns/openshift-sdn daemonset/ovs Created pod: ovs-5dgr8 Nov 06 18:07:46.923 I ns/openshift-multus pod/multus-7jm82 Successfully assigned openshift-multus/multus-7jm82 to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:07:46.933 I ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z Successfully assigned openshift-cluster-node-tuning-operator/tuned-ssr4z to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:07:46.933 I ns/openshift-sdn pod/ovs-5dgr8 Successfully assigned openshift-sdn/ovs-5dgr8 to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:07:46.945 I ns/openshift-dns pod/dns-default-4kkr9 node/ created Nov 06 18:07:46.962 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-4kkr9 Nov 06 18:07:46.969 W clusteroperator/node-tuning changed Progressing to True: Reconciling: Working towards "4.3.0-0.ci-2019-11-06-170148" Nov 06 18:07:46.983 I ns/openshift-sdn pod/sdn-tx8lf node/ created Nov 06 18:07:46.986 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Nov 06 18:07:46.997 I ns/openshift-sdn daemonset/sdn Created pod: sdn-tx8lf Nov 06 18:07:46.997 I ns/openshift-dns pod/dns-default-4kkr9 Successfully assigned openshift-dns/dns-default-4kkr9 to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:07:47.009 I ns/openshift-sdn pod/sdn-tx8lf Successfully assigned openshift-sdn/sdn-tx8lf to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:07:47.046 I ns/openshift-monitoring pod/node-exporter-xvxcw node/ created Nov 06 18:07:47.055 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-xvxcw Nov 06 18:07:47.074 I ns/openshift-monitoring pod/node-exporter-xvxcw Successfully assigned openshift-monitoring/node-exporter-xvxcw to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:07:47.169 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available. Nov 06 18:07:47.207 W clusteroperator/node-tuning changed Progressing to False: AsExpected: Cluster version is "4.3.0-0.ci-2019-11-06-170148" Nov 06 18:07:47.578 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk (5 times) Nov 06 18:07:48.069 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (2 times) Nov 06 18:07:48.144 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal event: Registered Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal in Controller Nov 06 18:07:48.330 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk (6 times) Nov 06 18:07:50.077 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (3 times) Nov 06 18:07:52.073 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (4 times) Nov 06 18:07:54.072 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (5 times) Nov 06 18:07:56.072 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (6 times) Nov 06 18:07:58.069 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (7 times) Nov 06 18:07:59.023 I ns/openshift-sdn pod/ovs-5dgr8 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" Nov 06 18:07:59.045 I ns/openshift-monitoring pod/node-exporter-xvxcw Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:195b73ffa38c24c558372c7be3e5fc4633a51b229d901dd0c060045143e80a8d" Nov 06 18:07:59.059 I ns/openshift-sdn pod/sdn-tx8lf Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:07:59.113 I ns/openshift-multus pod/multus-7jm82 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:07:59.198 I ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a09dc9c52c07e6cee4eb210a0b468d413497f3e16fbeb8d4146e5b84ffef2dc" Nov 06 18:08:00.069 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (8 times) Nov 06 18:08:01.321 - 14s W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal node is not ready Nov 06 18:08:02.729 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (9 times) Nov 06 18:08:04.311 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (10 times) Nov 06 18:08:06.069 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (11 times) Nov 06 18:08:08.069 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (12 times) Nov 06 18:08:10.075 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (13 times) Nov 06 18:08:12.071 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (14 times) Nov 06 18:08:14.070 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (15 times) Nov 06 18:08:16.074 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (16 times) Nov 06 18:08:17.497 I ns/openshift-sdn pod/ovs-5dgr8 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" Nov 06 18:08:17.536 I ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a09dc9c52c07e6cee4eb210a0b468d413497f3e16fbeb8d4146e5b84ffef2dc" Nov 06 18:08:17.540 I ns/openshift-monitoring pod/node-exporter-xvxcw Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:195b73ffa38c24c558372c7be3e5fc4633a51b229d901dd0c060045143e80a8d" Nov 06 18:08:17.561 I ns/openshift-multus pod/multus-7jm82 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:08:17.564 I ns/openshift-sdn pod/sdn-tx8lf Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:08:17.833 I ns/openshift-sdn pod/ovs-5dgr8 Created container openvswitch Nov 06 18:08:17.837 I ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z Created container tuned Nov 06 18:08:17.898 I ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z Started container tuned Nov 06 18:08:17.908 I ns/openshift-sdn pod/ovs-5dgr8 Started container openvswitch Nov 06 18:08:17.911 I ns/openshift-sdn pod/sdn-tx8lf Created container install-cni-plugins Nov 06 18:08:17.922 I ns/openshift-monitoring pod/node-exporter-xvxcw Created container node-exporter Nov 06 18:08:17.925 I ns/openshift-multus pod/multus-7jm82 Created container cni-plugins Nov 06 18:08:18.097 I ns/openshift-multus pod/multus-7jm82 Started container cni-plugins Nov 06 18:08:18.298 I ns/openshift-monitoring pod/node-exporter-xvxcw Started container node-exporter Nov 06 18:08:18.497 I ns/openshift-monitoring pod/node-exporter-xvxcw Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" Nov 06 18:08:18.697 I ns/openshift-sdn pod/sdn-tx8lf Started container install-cni-plugins Nov 06 18:08:18.900 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (17 times) Nov 06 18:08:19.097 I ns/openshift-multus pod/multus-7jm82 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:54ec2b7eacfd3dbd354d10545c2eda3a2a2112b89137dd9fabb164c0f9313b31" Nov 06 18:08:19.307 I ns/openshift-sdn pod/sdn-tx8lf Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" already present on machine Nov 06 18:08:19.497 I ns/openshift-sdn pod/sdn-tx8lf Created container sdn Nov 06 18:08:19.697 I ns/openshift-sdn pod/sdn-tx8lf Started container sdn Nov 06 18:08:19.940 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking. Nov 06 18:08:20.070 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (18 times) Nov 06 18:08:20.850 I ns/openshift-monitoring pod/node-exporter-xvxcw Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" Nov 06 18:08:21.018 I ns/openshift-monitoring pod/node-exporter-xvxcw Created container kube-rbac-proxy Nov 06 18:08:21.115 I ns/openshift-monitoring pod/node-exporter-xvxcw Started container kube-rbac-proxy Nov 06 18:08:21.533 I ns/openshift-multus pod/multus-7jm82 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:54ec2b7eacfd3dbd354d10545c2eda3a2a2112b89137dd9fabb164c0f9313b31" Nov 06 18:08:21.685 I ns/openshift-multus pod/multus-7jm82 Created container kube-multus Nov 06 18:08:21.715 I ns/openshift-multus pod/multus-7jm82 Started container kube-multus Nov 06 18:08:22.071 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (19 times) Nov 06 18:08:24.071 W ns/openshift-dns pod/dns-default-4kkr9 network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (20 times) Nov 06 18:08:27.322 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal condition Ready changed Nov 06 18:08:27.365 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 node/ created Nov 06 18:08:27.379 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-dfjl9 Nov 06 18:08:27.390 I ns/openshift-image-registry pod/node-ca-dpq6t node/ created Nov 06 18:08:27.393 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Successfully assigned openshift-machine-config-operator/machine-config-daemon-dfjl9 to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:08:27.403 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-dpq6t Nov 06 18:08:27.423 I ns/openshift-image-registry pod/node-ca-dpq6t Successfully assigned openshift-image-registry/node-ca-dpq6t to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:08:28.075 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Updated Machine ci-op--kg4jd-w-d-4rskk (7 times) Nov 06 18:08:28.104 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" Nov 06 18:08:33.555 I ns/openshift-dns pod/dns-default-4kkr9 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:9a7e8297f7b70ee3a11fcbe4a78c59a5861e1afda5657a7437de6934bdc2458e" Nov 06 18:08:34.633 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" Nov 06 18:08:34.762 W clusteroperator/network changed Progressing to False Nov 06 18:08:34.812 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Created container machine-config-daemon Nov 06 18:08:34.872 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Started container machine-config-daemon Nov 06 18:08:34.876 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" Nov 06 18:08:35.664 I ns/openshift-image-registry pod/node-ca-dpq6t Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" Nov 06 18:08:38.389 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Setting node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal, currentConfig rendered-worker-f0324f2675d9cc2bb16336bce61aa68a to Done Nov 06 18:08:38.651 I ns/openshift-dns pod/dns-default-4kkr9 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:9a7e8297f7b70ee3a11fcbe4a78c59a5861e1afda5657a7437de6934bdc2458e" Nov 06 18:08:38.655 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" Nov 06 18:08:38.863 I ns/openshift-dns pod/dns-default-4kkr9 Created container dns Nov 06 18:08:38.869 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Created container oauth-proxy Nov 06 18:08:38.912 I ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 Started container oauth-proxy Nov 06 18:08:38.928 I ns/openshift-dns pod/dns-default-4kkr9 Started container dns Nov 06 18:08:38.931 I ns/openshift-dns pod/dns-default-4kkr9 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:37811ccc702371cbc91e86935f03b5d55040835fb8d653d211410587385034f8" Nov 06 18:08:41.215 I ns/openshift-image-registry pod/node-ca-dpq6t Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" Nov 06 18:08:41.542 I ns/openshift-image-registry pod/node-ca-dpq6t Created container node-ca Nov 06 18:08:41.583 I ns/openshift-image-registry pod/node-ca-dpq6t Started container node-ca Nov 06 18:08:51.307 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal Nov 06 18:08:52.123 I ns/openshift-ingress service/router-default Updated load balancer with new hosts Nov 06 18:10:58.182 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (8 times) Nov 06 18:10:58.192 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (10 times) Nov 06 18:12:23.183 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal condition NetworkUnavailable changed Nov 06 18:12:23.185 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal status is now: NodeHasSufficientMemory (8 times) Nov 06 18:12:23.203 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal status is now: NodeHasNoDiskPressure (8 times) Nov 06 18:12:23.212 I ns/openshift-multus daemonset/multus Created pod: multus-lhw9n Nov 06 18:12:23.214 I ns/openshift-multus pod/multus-lhw9n node/ created Nov 06 18:12:23.233 W clusteroperator/network changed Progressing to True: Deploying: DaemonSet "openshift-multus/multus" is not available (awaiting 1 nodes) Nov 06 18:12:23.247 I ns/openshift-sdn pod/ovs-cjz7b node/ created Nov 06 18:12:23.253 I ns/openshift-sdn daemonset/ovs Created pod: ovs-cjz7b Nov 06 18:12:23.255 I ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 node/ created Nov 06 18:12:23.270 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-krkn2 Nov 06 18:12:23.276 I ns/openshift-multus pod/multus-lhw9n Successfully assigned openshift-multus/multus-lhw9n to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:12:23.284 I ns/openshift-sdn pod/ovs-cjz7b Successfully assigned openshift-sdn/ovs-cjz7b to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:12:23.286 I ns/openshift-dns pod/dns-default-m7g9w node/ created Nov 06 18:12:23.290 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-m7g9w Nov 06 18:12:23.292 I ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 Successfully assigned openshift-cluster-node-tuning-operator/tuned-krkn2 to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:12:23.307 I ns/openshift-sdn pod/sdn-9fqxf node/ created Nov 06 18:12:23.318 I ns/openshift-sdn daemonset/sdn Created pod: sdn-9fqxf Nov 06 18:12:23.334 I ns/openshift-sdn pod/sdn-9fqxf Successfully assigned openshift-sdn/sdn-9fqxf to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:12:23.334 I ns/openshift-dns pod/dns-default-m7g9w Successfully assigned openshift-dns/dns-default-m7g9w to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:12:23.343 I ns/openshift-monitoring pod/node-exporter-9r97w node/ created Nov 06 18:12:23.347 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-9r97w Nov 06 18:12:23.379 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Nov 06 18:12:23.385 I ns/openshift-monitoring pod/node-exporter-9r97w Successfully assigned openshift-monitoring/node-exporter-9r97w to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:12:23.428 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available. Nov 06 18:12:24.005 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp (5 times) Nov 06 18:12:24.714 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp (6 times) Nov 06 18:12:25.422 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (2 times) Nov 06 18:12:27.722 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (3 times) Nov 06 18:12:28.175 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal event: Registered Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal in Controller Nov 06 18:12:29.938 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (4 times) Nov 06 18:12:31.321 - 29s W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal node is not ready Nov 06 18:12:31.423 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (5 times) Nov 06 18:12:33.423 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (6 times) Nov 06 18:12:35.423 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (7 times) Nov 06 18:12:35.576 I ns/openshift-sdn pod/ovs-cjz7b Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" Nov 06 18:12:35.614 I ns/openshift-sdn pod/sdn-9fqxf Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:12:35.699 I ns/openshift-monitoring pod/node-exporter-9r97w Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:195b73ffa38c24c558372c7be3e5fc4633a51b229d901dd0c060045143e80a8d" Nov 06 18:12:35.716 I ns/openshift-multus pod/multus-lhw9n Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:12:35.769 I ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a09dc9c52c07e6cee4eb210a0b468d413497f3e16fbeb8d4146e5b84ffef2dc" Nov 06 18:12:36.095 I ns/openshift-multus pod/multus-rfmwh node/ created Nov 06 18:12:36.097 W node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal condition NetworkUnavailable changed Nov 06 18:12:36.100 I node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal status is now: NodeHasSufficientMemory (8 times) Nov 06 18:12:36.103 I ns/openshift-multus daemonset/multus Created pod: multus-rfmwh Nov 06 18:12:36.117 I ns/openshift-sdn pod/ovs-npxkg node/ created Nov 06 18:12:36.136 I ns/openshift-multus pod/multus-rfmwh Successfully assigned openshift-multus/multus-rfmwh to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:12:36.136 I node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal status is now: NodeHasNoDiskPressure (8 times) Nov 06 18:12:36.137 I ns/openshift-sdn daemonset/ovs Created pod: ovs-npxkg Nov 06 18:12:36.143 I ns/openshift-cluster-node-tuning-operator pod/tuned-bdz6b node/ created Nov 06 18:12:36.147 I ns/openshift-dns pod/dns-default-sz26z node/ created Nov 06 18:12:36.153 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-bdz6b Nov 06 18:12:36.163 I ns/openshift-sdn pod/ovs-npxkg Successfully assigned openshift-sdn/ovs-npxkg to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:12:36.167 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-sz26z Nov 06 18:12:36.177 I ns/openshift-sdn pod/sdn-v2h5l node/ created Nov 06 18:12:36.188 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Nov 06 18:12:36.193 I ns/openshift-sdn daemonset/sdn Created pod: sdn-v2h5l Nov 06 18:12:36.198 I ns/openshift-cluster-node-tuning-operator pod/tuned-bdz6b Successfully assigned openshift-cluster-node-tuning-operator/tuned-bdz6b to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:12:36.198 I ns/openshift-dns pod/dns-default-sz26z Successfully assigned openshift-dns/dns-default-sz26z to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:12:36.218 I ns/openshift-sdn pod/sdn-v2h5l Successfully assigned openshift-sdn/sdn-v2h5l to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:12:36.221 I ns/openshift-monitoring pod/node-exporter-shnb2 node/ created Nov 06 18:12:36.230 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-shnb2 Nov 06 18:12:36.246 I ns/openshift-monitoring pod/node-exporter-shnb2 Successfully assigned openshift-monitoring/node-exporter-shnb2 to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:12:36.307 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available Nov 06 18:12:36.655 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg (5 times) Nov 06 18:12:37.191 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg (6 times) Nov 06 18:12:37.593 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (8 times) Nov 06 18:12:38.177 I node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal event: Registered Node ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal in Controller Nov 06 18:12:38.309 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (2 times) Nov 06 18:12:40.654 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (3 times) Nov 06 18:12:41.649 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (9 times) Nov 06 18:12:43.423 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (10 times) Nov 06 18:12:43.428 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (4 times) Nov 06 18:12:45.307 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (5 times) Nov 06 18:12:45.426 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (11 times) Nov 06 18:12:46.321 - 14s W node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal node is not ready Nov 06 18:12:47.308 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (6 times) Nov 06 18:12:47.422 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (12 times) Nov 06 18:12:48.740 I ns/openshift-sdn pod/ovs-npxkg Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" Nov 06 18:12:48.798 I ns/openshift-sdn pod/sdn-v2h5l Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:12:48.867 I ns/openshift-cluster-node-tuning-operator pod/tuned-bdz6b Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a09dc9c52c07e6cee4eb210a0b468d413497f3e16fbeb8d4146e5b84ffef2dc" Nov 06 18:12:48.879 I ns/openshift-monitoring pod/node-exporter-shnb2 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:195b73ffa38c24c558372c7be3e5fc4633a51b229d901dd0c060045143e80a8d" Nov 06 18:12:48.886 I ns/openshift-multus pod/multus-rfmwh Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:12:49.308 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (7 times) Nov 06 18:12:49.423 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (13 times) Nov 06 18:12:49.582 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (6 times) Nov 06 18:12:49.589 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (8 times) Nov 06 18:12:51.312 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (8 times) Nov 06 18:12:51.424 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (14 times) Nov 06 18:12:52.712 I ns/openshift-sdn pod/ovs-cjz7b Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" Nov 06 18:12:52.842 I ns/openshift-multus pod/multus-lhw9n Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:12:52.850 I ns/openshift-monitoring pod/node-exporter-9r97w Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:195b73ffa38c24c558372c7be3e5fc4633a51b229d901dd0c060045143e80a8d" Nov 06 18:12:52.870 I ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a09dc9c52c07e6cee4eb210a0b468d413497f3e16fbeb8d4146e5b84ffef2dc" Nov 06 18:12:52.880 I ns/openshift-sdn pod/sdn-9fqxf Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:12:53.085 I ns/openshift-sdn pod/ovs-cjz7b Created container openvswitch Nov 06 18:12:53.129 I ns/openshift-multus pod/multus-lhw9n Created container cni-plugins Nov 06 18:12:53.164 I ns/openshift-sdn pod/ovs-cjz7b Started container openvswitch Nov 06 18:12:53.187 I ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 Created container tuned Nov 06 18:12:53.196 I ns/openshift-sdn pod/sdn-9fqxf Created container install-cni-plugins Nov 06 18:12:53.218 I ns/openshift-monitoring pod/node-exporter-9r97w Created container node-exporter Nov 06 18:12:53.225 I ns/openshift-multus pod/multus-lhw9n Started container cni-plugins Nov 06 18:12:53.312 I ns/openshift-sdn pod/sdn-9fqxf Started container install-cni-plugins Nov 06 18:12:53.331 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (9 times) Nov 06 18:12:53.512 I ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 Started container tuned Nov 06 18:12:53.713 I ns/openshift-monitoring pod/node-exporter-9r97w Started container node-exporter Nov 06 18:12:53.915 I ns/openshift-monitoring pod/node-exporter-9r97w Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" Nov 06 18:12:54.115 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (15 times) Nov 06 18:12:54.315 I ns/openshift-multus pod/multus-lhw9n Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:54ec2b7eacfd3dbd354d10545c2eda3a2a2112b89137dd9fabb164c0f9313b31" Nov 06 18:12:54.517 I ns/openshift-sdn pod/sdn-9fqxf Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" already present on machine Nov 06 18:12:54.713 I ns/openshift-sdn pod/sdn-9fqxf Created container sdn Nov 06 18:12:54.912 I ns/openshift-sdn pod/sdn-9fqxf Started container sdn Nov 06 18:12:55.135 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking. Nov 06 18:12:55.314 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (10 times) Nov 06 18:12:55.423 W ns/openshift-dns pod/dns-default-m7g9w network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (16 times) Nov 06 18:12:56.860 I ns/openshift-monitoring pod/node-exporter-9r97w Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" Nov 06 18:12:56.867 I ns/openshift-multus pod/multus-lhw9n Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:54ec2b7eacfd3dbd354d10545c2eda3a2a2112b89137dd9fabb164c0f9313b31" Nov 06 18:12:57.054 I ns/openshift-monitoring pod/node-exporter-9r97w Created container kube-rbac-proxy Nov 06 18:12:57.060 I ns/openshift-multus pod/multus-lhw9n Created container kube-multus Nov 06 18:12:57.088 I ns/openshift-monitoring pod/node-exporter-9r97w Started container kube-rbac-proxy Nov 06 18:12:57.094 I ns/openshift-multus pod/multus-lhw9n Started container kube-multus Nov 06 18:12:57.313 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (11 times) Nov 06 18:12:59.313 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (12 times) Nov 06 18:13:01.312 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (13 times) Nov 06 18:13:03.260 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal condition Ready changed Nov 06 18:13:03.312 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 node/ created Nov 06 18:13:03.317 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-f96x2 Nov 06 18:13:03.328 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (14 times) Nov 06 18:13:03.340 I ns/openshift-image-registry pod/node-ca-lhw7l node/ created Nov 06 18:13:03.343 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Successfully assigned openshift-machine-config-operator/machine-config-daemon-f96x2 to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:13:03.348 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-lhw7l Nov 06 18:13:03.361 I ns/openshift-image-registry pod/node-ca-lhw7l Successfully assigned openshift-image-registry/node-ca-lhw7l to ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Nov 06 18:13:03.996 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" Nov 06 18:13:04.029 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Updated Machine ci-op--kg4jd-w-b-526wp (7 times) Nov 06 18:13:04.720 I ns/openshift-dns pod/dns-default-m7g9w Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:9a7e8297f7b70ee3a11fcbe4a78c59a5861e1afda5657a7437de6934bdc2458e" Nov 06 18:13:05.308 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (15 times) Nov 06 18:13:05.548 I ns/openshift-sdn pod/ovs-npxkg Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" Nov 06 18:13:06.729 I ns/openshift-multus pod/multus-rfmwh Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:13:06.734 I ns/openshift-monitoring pod/node-exporter-shnb2 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:195b73ffa38c24c558372c7be3e5fc4633a51b229d901dd0c060045143e80a8d" Nov 06 18:13:06.750 I ns/openshift-cluster-node-tuning-operator pod/tuned-bdz6b Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a09dc9c52c07e6cee4eb210a0b468d413497f3e16fbeb8d4146e5b84ffef2dc" Nov 06 18:13:06.758 I ns/openshift-sdn pod/sdn-v2h5l Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:222704af43f638d68cf1c99fb3281bcca7ea299c95a4202ad1980c0326b35a90" Nov 06 18:13:07.002 I ns/openshift-sdn pod/ovs-npxkg Created container openvswitch Nov 06 18:13:07.038 I ns/openshift-monitoring pod/node-exporter-shnb2 Created container node-exporter Nov 06 18:13:07.070 I ns/openshift-cluster-node-tuning-operator pod/tuned-bdz6b Created container tuned Nov 06 18:13:07.096 I ns/openshift-sdn pod/sdn-v2h5l Created container install-cni-plugins Nov 06 18:13:07.104 I ns/openshift-sdn pod/ovs-npxkg Started container openvswitch Nov 06 18:13:07.110 I ns/openshift-monitoring pod/node-exporter-shnb2 Started container node-exporter Nov 06 18:13:07.125 I ns/openshift-monitoring pod/node-exporter-shnb2 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" Nov 06 18:13:07.130 I ns/openshift-multus pod/multus-rfmwh Created container cni-plugins Nov 06 18:13:07.327 I ns/openshift-cluster-node-tuning-operator pod/tuned-bdz6b Started container tuned Nov 06 18:13:07.528 I ns/openshift-sdn pod/sdn-v2h5l Started container install-cni-plugins Nov 06 18:13:07.732 I ns/openshift-multus pod/multus-rfmwh Started container cni-plugins Nov 06 18:13:07.930 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (16 times) Nov 06 18:13:08.129 I ns/openshift-sdn pod/sdn-v2h5l Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:0a60705c375cf1796e81cc7a5d459e5faa1de63b24e624ab87aa3fea6feb3348" already present on machine Nov 06 18:13:08.328 I ns/openshift-multus pod/multus-rfmwh Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:54ec2b7eacfd3dbd354d10545c2eda3a2a2112b89137dd9fabb164c0f9313b31" Nov 06 18:13:08.529 I ns/openshift-sdn pod/sdn-v2h5l Created container sdn Nov 06 18:13:08.728 I ns/openshift-sdn pod/sdn-v2h5l Started container sdn Nov 06 18:13:09.309 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (17 times) Nov 06 18:13:09.404 I node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal openshift-sdn done initializing node networking. Nov 06 18:13:09.671 I ns/openshift-monitoring pod/node-exporter-shnb2 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" Nov 06 18:13:09.831 I ns/openshift-monitoring pod/node-exporter-shnb2 Created container kube-rbac-proxy Nov 06 18:13:09.860 I ns/openshift-monitoring pod/node-exporter-shnb2 Started container kube-rbac-proxy Nov 06 18:13:10.403 I ns/openshift-multus pod/multus-rfmwh Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:54ec2b7eacfd3dbd354d10545c2eda3a2a2112b89137dd9fabb164c0f9313b31" Nov 06 18:13:10.557 I ns/openshift-multus pod/multus-rfmwh Created container kube-multus Nov 06 18:13:10.592 I ns/openshift-multus pod/multus-rfmwh Started container kube-multus Nov 06 18:13:10.947 I ns/openshift-dns pod/dns-default-m7g9w Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:9a7e8297f7b70ee3a11fcbe4a78c59a5861e1afda5657a7437de6934bdc2458e" Nov 06 18:13:10.967 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" Nov 06 18:13:11.137 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Created container machine-config-daemon Nov 06 18:13:11.148 I ns/openshift-dns pod/dns-default-m7g9w Created container dns Nov 06 18:13:11.176 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Started container machine-config-daemon Nov 06 18:13:11.184 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" Nov 06 18:13:11.190 I ns/openshift-dns pod/dns-default-m7g9w Started container dns Nov 06 18:13:11.197 I ns/openshift-dns pod/dns-default-m7g9w Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:37811ccc702371cbc91e86935f03b5d55040835fb8d653d211410587385034f8" Nov 06 18:13:11.307 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (18 times) Nov 06 18:13:11.756 I ns/openshift-image-registry pod/node-ca-lhw7l Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" Nov 06 18:13:13.308 W ns/openshift-dns pod/dns-default-sz26z network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (19 times) Nov 06 18:13:13.916 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" Nov 06 18:13:14.099 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Created container oauth-proxy Nov 06 18:13:14.132 I ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 Started container oauth-proxy Nov 06 18:13:14.340 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Setting node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal, currentConfig rendered-worker-f0324f2675d9cc2bb16336bce61aa68a to Done Nov 06 18:13:15.668 I ns/openshift-dns pod/dns-default-m7g9w Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:37811ccc702371cbc91e86935f03b5d55040835fb8d653d211410587385034f8" Nov 06 18:13:15.820 I ns/openshift-dns pod/dns-default-m7g9w Created container dns-node-resolver Nov 06 18:13:15.862 I ns/openshift-dns pod/dns-default-m7g9w Started container dns-node-resolver Nov 06 18:13:16.161 W node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal condition Ready changed Nov 06 18:13:16.184 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ created Nov 06 18:13:16.192 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-m2gmn Nov 06 18:13:16.205 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Successfully assigned openshift-machine-config-operator/machine-config-daemon-m2gmn to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:13:16.210 I ns/openshift-image-registry pod/node-ca-zstvb node/ created Nov 06 18:13:16.219 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-zstvb Nov 06 18:13:16.233 I ns/openshift-image-registry pod/node-ca-zstvb Successfully assigned openshift-image-registry/node-ca-zstvb to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:13:16.795 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-rkgcg Updated Machine ci-op--kg4jd-w-c-rkgcg (7 times) Nov 06 18:13:16.881 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" Nov 06 18:13:17.133 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Node "ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal" drained Nov 06 18:13:17.290 I ns/openshift-image-registry pod/node-ca-lhw7l Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" Nov 06 18:13:17.426 I ns/openshift-image-registry pod/node-ca-lhw7l Created container node-ca Nov 06 18:13:17.462 I ns/openshift-image-registry pod/node-ca-lhw7l Started container node-ca Nov 06 18:13:18.276 W ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp requeue in: 20s Nov 06 18:13:18.410 W ns/openshift-marketplace pod/certified-operators-c6448b4f6-57dtq node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.413 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 120s Nov 06 18:13:18.431 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-57dtq Stopping container certified-operators Nov 06 18:13:18.432 W ns/openshift-marketplace pod/community-operators-74b7658499-gkfj7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.460 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager Nov 06 18:13:18.460 W ns/openshift-image-registry pod/image-registry-9b9765ddf-b7n4q node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.461 W ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.461 W ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 120s Nov 06 18:13:18.463 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.463 W ns/openshift-ingress pod/router-default-ffd95fb7d-d8wrr node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.464 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.467 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 18:13:18.468 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-kktz7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.478 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:18.488 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f node/ created Nov 06 18:13:18.494 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy Nov 06 18:13:18.502 I ns/openshift-marketplace replicaset/certified-operators-c6448b4f6 Created pod: certified-operators-c6448b4f6-9qx7f Nov 06 18:13:18.516 I ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd node/ created Nov 06 18:13:18.534 I ns/openshift-marketplace replicaset/community-operators-74b7658499 Created pod: community-operators-74b7658499-pvhcd Nov 06 18:13:18.543 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader Nov 06 18:13:18.548 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Successfully assigned openshift-marketplace/certified-operators-c6448b4f6-9qx7f to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.548 I ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Successfully assigned openshift-marketplace/community-operators-74b7658499-pvhcd to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.557 I ns/openshift-marketplace pod/community-operators-74b7658499-gkfj7 Stopping container community-operators Nov 06 18:13:18.576 I ns/openshift-image-registry replicaset/image-registry-9b9765ddf Created pod: image-registry-9b9765ddf-cpgbq Nov 06 18:13:18.579 I ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq node/ created Nov 06 18:13:18.584 I ns/openshift-image-registry pod/image-registry-9b9765ddf-b7n4q Stopping container registry Nov 06 18:13:18.595 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q node/ created Nov 06 18:13:18.605 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd Stopping container kube-state-metrics Nov 06 18:13:18.606 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x node/ created Nov 06 18:13:18.608 I ns/openshift-ingress replicaset/router-default-ffd95fb7d Created pod: router-default-ffd95fb7d-pd54q Nov 06 18:13:18.613 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq node/ created Nov 06 18:13:18.616 I ns/openshift-monitoring replicaset/prometheus-adapter-74ccdcf5b7 Created pod: prometheus-adapter-74ccdcf5b7-47k9x Nov 06 18:13:18.634 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd Stopping container kube-rbac-proxy-self Nov 06 18:13:18.641 I ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq Successfully assigned openshift-image-registry/image-registry-9b9765ddf-cpgbq to ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.650 I ns/openshift-monitoring replicaset/kube-state-metrics-66b55b47c7 Created pod: kube-state-metrics-66b55b47c7-qrzpq Nov 06 18:13:18.660 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Successfully assigned openshift-ingress/router-default-ffd95fb7d-pd54q to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.660 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd Stopping container kube-rbac-proxy-main Nov 06 18:13:18.687 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager-proxy Nov 06 18:13:18.687 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Successfully assigned openshift-monitoring/kube-state-metrics-66b55b47c7-qrzpq to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.687 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Successfully assigned openshift-monitoring/prometheus-adapter-74ccdcf5b7-47k9x to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.713 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container config-reloader Nov 06 18:13:18.727 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z node/ created Nov 06 18:13:18.759 W clusteroperator/image-registry changed Available to False: NoReplicasAvailable: The deployment does not have available replicas Nov 06 18:13:18.759 W clusteroperator/image-registry changed Progressing to True: DeploymentNotCompleted: The deployment has not completed Nov 06 18:13:18.770 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Successfully assigned openshift-marketplace/redhat-operators-557b5c89cf-6k27z to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.780 I ns/openshift-marketplace replicaset/redhat-operators-557b5c89cf Created pod: redhat-operators-557b5c89cf-6k27z Nov 06 18:13:18.793 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb node/ created Nov 06 18:13:18.809 I ns/openshift-monitoring replicaset/openshift-state-metrics-59b4fb5c69 Created pod: openshift-state-metrics-59b4fb5c69-dgkfb Nov 06 18:13:18.811 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx node/ created Nov 06 18:13:18.838 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager Nov 06 18:13:18.851 I ns/openshift-monitoring replicaset/thanos-querier-6fcc94d644 Created pod: thanos-querier-6fcc94d644-2vhtx Nov 06 18:13:18.851 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Successfully assigned openshift-monitoring/openshift-state-metrics-59b4fb5c69-dgkfb to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.857 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Successfully assigned openshift-monitoring/thanos-querier-6fcc94d644-2vhtx to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:18.906 W clusteroperator/image-registry changed Available to True: Ready: The registry is ready Nov 06 18:13:18.906 W clusteroperator/image-registry changed Progressing to False: Ready: The registry is ready Nov 06 18:13:18.931 W clusteroperator/image-registry changed Available to False: NoReplicasAvailable: The deployment does not have available replicas Nov 06 18:13:18.931 W clusteroperator/image-registry changed Progressing to True: DeploymentNotCompleted: The deployment has not completed Nov 06 18:13:18.971 W clusteroperator/ingress changed Available to False: IngressUnavailable: Not all ingress controllers are available. Nov 06 18:13:18.971 W clusteroperator/ingress changed Progressing to True: Reconciling: Not all ingress controllers are available. Nov 06 18:13:19.029 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn Stopping container prometheus-adapter Nov 06 18:13:19.223 I ns/openshift-ingress pod/router-default-ffd95fb7d-d8wrr Stopping container router Nov 06 18:13:19.427 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Stopping container prom-label-proxy Nov 06 18:13:19.629 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Stopping container kube-rbac-proxy Nov 06 18:13:19.822 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Stopping container oauth-proxy Nov 06 18:13:20.022 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 Stopping container thanos-querier Nov 06 18:13:20.225 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container kube-rbac-proxy Nov 06 18:13:20.422 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prometheus-proxy Nov 06 18:13:20.628 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container thanos-sidecar Nov 06 18:13:20.823 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container rules-configmap-reloader Nov 06 18:13:20.836 E ns/openshift-marketplace pod/certified-operators-c6448b4f6-57dtq node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=certified-operators container exited with code 2 (Error): Nov 06 18:13:20.858 W ns/openshift-image-registry pod/image-registry-9b9765ddf-b7n4q node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:20.858 W ns/openshift-image-registry pod/image-registry-9b9765ddf-b7n4q node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=registry container stopped being ready Nov 06 18:13:20.877 W ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:20.877 W ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-state-metrics container stopped being ready Nov 06 18:13:20.877 W ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy-self container stopped being ready Nov 06 18:13:20.877 W ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy-main container stopped being ready Nov 06 18:13:20.901 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 17:29:30 Watching directory: "/etc/alertmanager/config"\n Nov 06 18:13:20.901 E ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/06 17:29:30 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:29:30 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 17:29:30 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 17:29:30 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 17:29:30 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 17:29:30 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:29:30 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 17:29:30 http.go:96: HTTPS: listening on [::]:9095\n Nov 06 18:13:20.937 E ns/openshift-marketplace pod/community-operators-74b7658499-gkfj7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=community-operators container exited with code 2 (Error): Nov 06 18:13:20.958 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:20.958 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready Nov 06 18:13:20.958 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready Nov 06 18:13:20.958 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=thanos-querier container stopped being ready Nov 06 18:13:20.958 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready Nov 06 18:13:20.974 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-kktz7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:20.974 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-kktz7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=redhat-operators container stopped being ready Nov 06 18:13:21.030 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prometheus-config-reloader Nov 06 18:13:21.075 E ns/openshift-ingress pod/router-default-ffd95fb7d-d8wrr node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=router container exited with code 2 (Error): Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T17:58:46.958Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T17:58:59.833Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:21.540Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:26.537Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:39.599Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:44.586Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:08:51.270Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:12:57.925Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:13:02.924Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:13:10.774Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:13:15.765Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n Nov 06 18:13:21.094 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:21.094 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=prometheus-adapter container stopped being ready Nov 06 18:13:21.222 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prometheus Nov 06 18:13:21.421 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prom-label-proxy Nov 06 18:13:21.623 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-kktz7 Stopping container redhat-operators Nov 06 18:13:21.822 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz Stopping container openshift-state-metrics Nov 06 18:13:22.022 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz Stopping container kube-rbac-proxy-self Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=prom-label-proxy container stopped being ready Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=thanos-sidecar container stopped being ready Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=rules-configmap-reloader container stopped being ready Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy container stopped being ready Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=prometheus-config-reloader container stopped being ready Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=prometheus container stopped being ready Nov 06 18:13:22.084 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=prometheus-proxy container stopped being ready Nov 06 18:13:22.123 W ns/openshift-marketplace pod/certified-operators-c6448b4f6-57dtq node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.140 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:22.140 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy-main container stopped being ready Nov 06 18:13:22.140 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=openshift-state-metrics container stopped being ready Nov 06 18:13:22.140 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=kube-rbac-proxy-self container stopped being ready Nov 06 18:13:22.186 W ns/openshift-monitoring pod/alertmanager-main-2 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.203 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 17:46:16 Watching directory: "/etc/alertmanager/config"\n Nov 06 18:13:22.203 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): 2019/11/06 17:46:16 provider.go:117: Defaulting client-id to system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:46:16 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 17:46:16 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 17:46:16 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 17:46:16 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 17:46:16 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 17:46:16 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 17:46:16 http.go:96: HTTPS: listening on [::]:9095\n Nov 06 18:13:22.223 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz Stopping container kube-rbac-proxy-main Nov 06 18:13:22.237 I ns/openshift-monitoring pod/alertmanager-main-2 node/ created Nov 06 18:13:22.249 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-2 in StatefulSet alertmanager-main successful Nov 06 18:13:22.262 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully assigned openshift-monitoring/alertmanager-main-2 to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:22.275 W ns/openshift-image-registry pod/image-registry-9b9765ddf-b7n4q node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.312 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-kktz7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.342 W ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-dmhxd node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.380 W ns/openshift-marketplace pod/community-operators-74b7658499-gkfj7 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.412 W ns/openshift-monitoring pod/thanos-querier-6fcc94d644-hjg29 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:22.829 I ns/openshift-dns pod/dns-default-sz26z Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:9a7e8297f7b70ee3a11fcbe4a78c59a5861e1afda5657a7437de6934bdc2458e" Nov 06 18:13:23.619 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" Nov 06 18:13:23.773 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Created container machine-config-daemon Nov 06 18:13:23.805 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Started container machine-config-daemon Nov 06 18:13:23.810 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" Nov 06 18:13:24.636 I ns/openshift-image-registry pod/node-ca-zstvb Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" Nov 06 18:13:26.287 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:7cecc44f74abd6d635eef4b5e3ab69016ccdb48dcccaf50d265cdd50384b373b" Nov 06 18:13:26.388 I ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 18:13:26.473 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:bd70de74d6dc01dedf586047151a4e6f4116a04f853d3fce65f353726a0c64e3" Nov 06 18:13:26.562 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:2b9dabef8ad692d143307c77162da685f6bdfe94be7f09010a64b73400c96258" already present on machine Nov 06 18:13:26.567 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 18:13:26.568 I ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq Created container registry Nov 06 18:13:26.603 I ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq Started container registry Nov 06 18:13:26.634 W clusteroperator/network changed Progressing to False Nov 06 18:13:26.690 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:7cecc44f74abd6d635eef4b5e3ab69016ccdb48dcccaf50d265cdd50384b373b" Nov 06 18:13:26.825 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Created container thanos-querier Nov 06 18:13:26.830 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Created container kube-rbac-proxy-main Nov 06 18:13:26.869 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Started container kube-rbac-proxy-main Nov 06 18:13:26.873 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 18:13:26.882 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Started container thanos-querier Nov 06 18:13:26.884 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:13:26.940 I node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal status is now: NodeNotSchedulable Nov 06 18:13:27.084 I ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:7cecc44f74abd6d635eef4b5e3ab69016ccdb48dcccaf50d265cdd50384b373b" Nov 06 18:13:27.110 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Created container kube-rbac-proxy-self Nov 06 18:13:27.163 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Started container kube-rbac-proxy-self Nov 06 18:13:27.165 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:239d068ee44f8d3c0889df2b9f74920b6eade4b0c36a5842b414865b85798f30" Nov 06 18:13:27.222 I node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Setting node ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal, currentConfig rendered-worker-f0324f2675d9cc2bb16336bce61aa68a to Done Nov 06 18:13:27.337 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:832221e63bc0b28b204f04a861c773b7f808afd7b18922a6983e5f95fbeb7ae3" Nov 06 18:13:27.363 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Created container oauth-proxy Nov 06 18:13:27.563 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Started container oauth-proxy Nov 06 18:13:27.763 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 18:13:27.963 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 18:13:28.163 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Created container kube-rbac-proxy Nov 06 18:13:28.364 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Started container kube-rbac-proxy Nov 06 18:13:28.563 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:27f4cef3c20b57e894d34b297136eed9b2864efbbd96f57d2bed7c8441947b3d" already present on machine Nov 06 18:13:28.763 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Created container kube-rbac-proxy-main Nov 06 18:13:28.964 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Started container kube-rbac-proxy-main Nov 06 18:13:29.168 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 18:13:29.366 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Created container prom-label-proxy Nov 06 18:13:29.564 I ns/openshift-monitoring pod/thanos-querier-6fcc94d644-2vhtx Started container prom-label-proxy Nov 06 18:13:29.764 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Created container kube-rbac-proxy-self Nov 06 18:13:29.967 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Started container kube-rbac-proxy-self Nov 06 18:13:30.163 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:2ff626cae8e48e7075928c30aaf339add647c8b3a7c9cbb62581ef0e91a2a9f9" Nov 06 18:13:31.001 I ns/openshift-dns pod/dns-default-sz26z Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:9a7e8297f7b70ee3a11fcbe4a78c59a5861e1afda5657a7437de6934bdc2458e" Nov 06 18:13:31.054 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" Nov 06 18:13:31.061 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:bd70de74d6dc01dedf586047151a4e6f4116a04f853d3fce65f353726a0c64e3" Nov 06 18:13:31.321 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 18:13:31.321 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 18:13:31.321 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 18:13:31.652 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:4a4d954ea2a2f7eff16f8c11d2b808286ff722c878d4d05dccf4d545580f1d3a" already present on machine Nov 06 18:13:31.688 W ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-qm6tz node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:31.721 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-tdhpn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:31.764 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:31.799 W ns/openshift-ingress pod/router-default-ffd95fb7d-d8wrr node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:31.836 W ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:31.869 I ns/openshift-monitoring pod/prometheus-k8s-0 node/ created Nov 06 18:13:31.891 I ns/openshift-monitoring statefulset/prometheus-k8s create Pod prometheus-k8s-0 in StatefulSet prometheus-k8s successful Nov 06 18:13:31.892 I ns/openshift-monitoring pod/alertmanager-main-1 node/ created Nov 06 18:13:31.898 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:832221e63bc0b28b204f04a861c773b7f808afd7b18922a6983e5f95fbeb7ae3" Nov 06 18:13:31.899 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully assigned openshift-monitoring/prometheus-k8s-0 to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:31.907 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-1 in StatefulSet alertmanager-main successful (2 times) Nov 06 18:13:31.919 I ns/openshift-monitoring pod/alertmanager-main-1 Successfully assigned openshift-monitoring/alertmanager-main-1 to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:13:32.131 I ns/openshift-dns pod/dns-default-sz26z Created container dns Nov 06 18:13:32.292 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Created container oauth-proxy Nov 06 18:13:32.300 I ns/openshift-dns pod/dns-default-sz26z Started container dns Nov 06 18:13:32.308 I ns/openshift-dns pod/dns-default-sz26z Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:37811ccc702371cbc91e86935f03b5d55040835fb8d653d211410587385034f8" Nov 06 18:13:32.314 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Created container router Nov 06 18:13:32.321 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Created container prometheus-adapter Nov 06 18:13:32.395 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Started container router Nov 06 18:13:32.400 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Started container oauth-proxy Nov 06 18:13:32.437 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Started container prometheus-adapter Nov 06 18:13:32.643 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Node "ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal" drained Nov 06 18:13:33.152 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager Nov 06 18:13:33.529 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl requeue in: 20s Nov 06 18:13:33.608 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager Nov 06 18:13:33.632 W ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:13:33.640 I ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq Stopping container registry Nov 06 18:13:33.687 I ns/openshift-image-registry pod/image-registry-9b9765ddf-zsthn node/ created Nov 06 18:13:33.723 I ns/openshift-image-registry replicaset/image-registry-9b9765ddf Created pod: image-registry-9b9765ddf-zsthn Nov 06 18:13:33.731 I ns/openshift-image-registry pod/image-registry-9b9765ddf-zsthn Successfully assigned openshift-image-registry/image-registry-9b9765ddf-zsthn to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:13:33.782 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:63ff8d9f806970af770724d4ba8bc521885de81e3e22212042a8b770a83f1b94" already present on machine Nov 06 18:13:34.057 I ns/openshift-image-registry pod/node-ca-zstvb Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" Nov 06 18:13:34.286 I ns/openshift-image-registry pod/node-ca-zstvb Created container node-ca Nov 06 18:13:34.323 I ns/openshift-image-registry pod/node-ca-zstvb Started container node-ca Nov 06 18:13:35.158 W ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:13:35.557 I ns/openshift-monitoring pod/alertmanager-main-2 Created container config-reloader Nov 06 18:13:35.660 I ns/openshift-monitoring pod/alertmanager-main-2 Started container config-reloader Nov 06 18:13:37.287 I ns/openshift-dns pod/dns-default-sz26z Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:37811ccc702371cbc91e86935f03b5d55040835fb8d653d211410587385034f8" Nov 06 18:13:37.573 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal status is now: NodeNotSchedulable Nov 06 18:13:38.408 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:13:41.035 I ns/openshift-monitoring pod/alertmanager-main-1 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:4a4d954ea2a2f7eff16f8c11d2b808286ff722c878d4d05dccf4d545580f1d3a" Nov 06 18:13:41.056 I ns/openshift-monitoring pod/prometheus-k8s-0 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:becb624e216762b01c94fcd4a8c830ae10cd415a7225928187c2d94b69ff4b9f" Nov 06 18:13:42.842 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:7cecc44f74abd6d635eef4b5e3ab69016ccdb48dcccaf50d265cdd50384b373b" Nov 06 18:13:42.846 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:239d068ee44f8d3c0889df2b9f74920b6eade4b0c36a5842b414865b85798f30" Nov 06 18:13:43.320 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:7cecc44f74abd6d635eef4b5e3ab69016ccdb48dcccaf50d265cdd50384b373b" Nov 06 18:13:43.384 I ns/openshift-image-registry pod/image-registry-9b9765ddf-zsthn Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 18:13:43.387 I ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:7cecc44f74abd6d635eef4b5e3ab69016ccdb48dcccaf50d265cdd50384b373b" Nov 06 18:13:43.560 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:2ff626cae8e48e7075928c30aaf339add647c8b3a7c9cbb62581ef0e91a2a9f9" Nov 06 18:13:43.587 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager-proxy Nov 06 18:13:43.706 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager-proxy Nov 06 18:13:43.906 I ns/openshift-monitoring pod/alertmanager-main-1 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:4a4d954ea2a2f7eff16f8c11d2b808286ff722c878d4d05dccf4d545580f1d3a" Nov 06 18:13:43.926 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Created container kube-state-metrics Nov 06 18:13:43.967 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Created container certified-operators Nov 06 18:13:44.087 W ns/openshift-image-registry pod/image-registry-9b9765ddf-cpgbq node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:13:44.089 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager Nov 06 18:13:44.125 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager Nov 06 18:13:44.132 I ns/openshift-monitoring pod/alertmanager-main-1 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:63ff8d9f806970af770724d4ba8bc521885de81e3e22212042a8b770a83f1b94" Nov 06 18:13:44.136 I ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Started container certified-operators Nov 06 18:13:44.161 I ns/openshift-image-registry pod/image-registry-9b9765ddf-zsthn Created container registry Nov 06 18:13:44.187 I ns/openshift-monitoring pod/kube-state-metrics-66b55b47c7-qrzpq Started container kube-state-metrics Nov 06 18:13:44.198 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Created container redhat-operators Nov 06 18:13:44.361 I ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Started container redhat-operators Nov 06 18:13:44.373 I ns/openshift-image-registry pod/image-registry-9b9765ddf-zsthn Started container registry Nov 06 18:13:44.401 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Created container openshift-state-metrics Nov 06 18:13:44.520 I ns/openshift-monitoring pod/openshift-state-metrics-59b4fb5c69-dgkfb Started container openshift-state-metrics Nov 06 18:13:44.652 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Node "ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal" drained Nov 06 18:13:44.718 I ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Created container community-operators Nov 06 18:13:44.919 I ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Started container community-operators Nov 06 18:13:45.856 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s Nov 06 18:13:45.917 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Node "ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal" drained (2 times) Nov 06 18:13:46.775 W ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp requeue in: 20s (2 times) Nov 06 18:13:46.836 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Node "ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal" drained (2 times) Nov 06 18:13:47.053 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:becb624e216762b01c94fcd4a8c830ae10cd415a7225928187c2d94b69ff4b9f" Nov 06 18:13:47.265 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus Nov 06 18:13:47.312 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus Nov 06 18:13:47.316 I ns/openshift-monitoring pod/prometheus-k8s-0 Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:36810c9ef9cc21340d53741d9b9c9ec022c6dd6d194beaaba1ac8e54170a11df" Nov 06 18:13:47.626 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl requeue in: 20s (2 times) Nov 06 18:13:47.688 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Node "ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal" drained (2 times) Nov 06 18:13:48.457 W clusteroperator/ingress changed Available to True: desired and current number of IngressControllers are equal Nov 06 18:13:48.457 W clusteroperator/ingress changed Progressing to False: desired and current number of IngressControllers are equal Nov 06 18:13:48.540 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s (2 times) Nov 06 18:13:50.469 W clusteroperator/image-registry changed Available to True: Ready: The registry is ready Nov 06 18:13:50.469 W clusteroperator/image-registry changed Progressing to False: Ready: The registry is ready Nov 06 18:13:50.564 W clusteroperator/image-registry changed Available to False: NoReplicasAvailable: The deployment does not have available replicas Nov 06 18:13:50.564 W clusteroperator/image-registry changed Progressing to True: DeploymentNotCompleted: The deployment has not completed Nov 06 18:13:50.598 W clusteroperator/image-registry changed Available to True: Ready: The registry is ready Nov 06 18:13:50.598 W clusteroperator/image-registry changed Progressing to False: Ready: The registry is ready Nov 06 18:13:51.056 W clusteroperator/dns changed Degraded to False: AsExpected: All desired DNS DaemonSets available and operand Namespace exists Nov 06 18:13:52.064 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:36810c9ef9cc21340d53741d9b9c9ec022c6dd6d194beaaba1ac8e54170a11df" Nov 06 18:13:52.516 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus-config-reloader Nov 06 18:13:52.552 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus-config-reloader Nov 06 18:13:52.556 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:63ff8d9f806970af770724d4ba8bc521885de81e3e22212042a8b770a83f1b94" already present on machine Nov 06 18:13:52.917 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container rules-configmap-reloader Nov 06 18:13:52.952 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container rules-configmap-reloader Nov 06 18:13:52.955 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:2b9dabef8ad692d143307c77162da685f6bdfe94be7f09010a64b73400c96258" already present on machine Nov 06 18:13:53.171 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container thanos-sidecar Nov 06 18:13:53.209 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container thanos-sidecar Nov 06 18:13:53.214 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:13:53.414 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus-proxy Nov 06 18:13:53.450 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus-proxy Nov 06 18:13:53.454 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:93a0c6d5321fe0afcc6d59b04b7110f17418390e71d2ba2320b3b2a71b632b56" already present on machine Nov 06 18:13:53.657 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container kube-rbac-proxy Nov 06 18:13:53.671 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Node "ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal" drained (3 times) Nov 06 18:13:53.693 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container kube-rbac-proxy Nov 06 18:13:53.696 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:27f4cef3c20b57e894d34b297136eed9b2864efbbd96f57d2bed7c8441947b3d" already present on machine Nov 06 18:13:53.869 I ns/openshift-monitoring pod/alertmanager-main-1 Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:63ff8d9f806970af770724d4ba8bc521885de81e3e22212042a8b770a83f1b94" Nov 06 18:13:53.886 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prom-label-proxy Nov 06 18:13:53.922 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prom-label-proxy Nov 06 18:13:54.339 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:becb624e216762b01c94fcd4a8c830ae10cd415a7225928187c2d94b69ff4b9f" already present on machine Nov 06 18:13:54.348 E ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=prometheus container exited with code 1 (Error): caller=main.go:657 msg="Starting TSDB ..."\nlevel=info ts=2019-11-06T18:13:47.337Z caller=web.go:450 component=web msg="Start listening for connections" address=127.0.0.1:9090\nlevel=info ts=2019-11-06T18:13:47.342Z caller=head.go:514 component=tsdb msg="replaying WAL, this may take awhile"\nlevel=info ts=2019-11-06T18:13:47.342Z caller=head.go:562 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:672 fs_type=XFS_SUPER_MAGIC\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:673 msg="TSDB started"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:743 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:526 msg="Stopping scrape discovery manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:540 msg="Stopping notify discovery manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:562 msg="Stopping scrape manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=manager.go:814 component="rule manager" msg="Stopping rule manager..."\nlevel=info ts=2019-11-06T18:13:47.343Z caller=manager.go:820 component="rule manager" msg="Rule manager stopped"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:522 msg="Scrape discovery manager stopped"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:536 msg="Notify discovery manager stopped"\nlevel=info ts=2019-11-06T18:13:47.343Z caller=main.go:556 msg="Scrape manager stopped"\nlevel=info ts=2019-11-06T18:13:47.345Z caller=notifier.go:602 component=notifier msg="Stopping notification manager..."\nlevel=info ts=2019-11-06T18:13:47.345Z caller=main.go:727 msg="Notifier manager stopped"\nlevel=error ts=2019-11-06 Nov 06 18:13:54.400 I ns/openshift-monitoring pod/alertmanager-main-1 Created container config-reloader Nov 06 18:13:54.431 I ns/openshift-monitoring pod/alertmanager-main-1 Started container config-reloader Nov 06 18:13:54.437 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:13:54.456 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl requeue in: 20s (3 times) Nov 06 18:13:54.548 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus (2 times) Nov 06 18:13:54.608 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager-proxy Nov 06 18:13:54.639 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager-proxy Nov 06 18:13:54.956 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Readiness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 Nov 06 18:13:55.353 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal container=prometheus container restarted Nov 06 18:13:56.861 W ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Liveness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 06 18:13:56.956 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 Nov 06 18:13:57.341 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 06 18:13:58.957 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 Nov 06 18:14:00.956 W ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Readiness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 Nov 06 18:14:02.937 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 06 18:14:03.209 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal condition DiskPressure changed Nov 06 18:14:03.209 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal condition MemoryPressure changed Nov 06 18:14:03.209 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal condition PIDPressure changed Nov 06 18:14:03.209 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal condition Ready changed Nov 06 18:14:03.216 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal status is now: NodeNotReady Nov 06 18:14:03.316 W clusteroperator/network changed Progressing to True: Deploying: DaemonSet "openshift-multus/multus" is not available (awaiting 1 nodes) Nov 06 18:14:03.381 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Node "ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal" drained (3 times) Nov 06 18:14:04.140 W ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp requeue in: 20s (3 times) Nov 06 18:14:05.916 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Node "ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal" drained (3 times) Nov 06 18:14:06.776 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s (3 times) Nov 06 18:14:06.837 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Node "ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal" drained (4 times) Nov 06 18:14:06.919 W ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Liveness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n (2 times) Nov 06 18:14:06.958 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 (2 times) Nov 06 18:14:07.274 I ns/openshift-machine-api machine/ci-op--kg4jd-w-b-526wp Deleted machine ci-op--kg4jd-w-b-526wp Nov 06 18:14:07.354 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n (2 times) Nov 06 18:14:07.378 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal event: DeletingNode Nov 06 18:14:07.380 W node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:08.316 I node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal event: Removing Node ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal from Controller Nov 06 18:14:08.320 W node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal condition DiskPressure changed Nov 06 18:14:08.320 W node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal condition MemoryPressure changed Nov 06 18:14:08.320 W node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal condition PIDPressure changed Nov 06 18:14:08.320 W node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal condition Ready changed Nov 06 18:14:08.333 I node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal status is now: NodeNotReady Nov 06 18:14:08.509 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Node "ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal" drained (4 times) Nov 06 18:14:08.960 W ns/openshift-marketplace pod/redhat-operators-557b5c89cf-6k27z Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 (2 times) Nov 06 18:14:09.230 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl requeue in: 20s (4 times) Nov 06 18:14:10.957 W ns/openshift-marketplace pod/certified-operators-c6448b4f6-9qx7f Readiness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 (2 times) Nov 06 18:14:13.239 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n (2 times) Nov 06 18:14:13.807 W ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.812 W ns/openshift-cluster-node-tuning-operator pod/tuned-krkn2 node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.826 W ns/openshift-sdn pod/sdn-9fqxf node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.833 W ns/openshift-sdn pod/sdn-9fqxf node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.837 W ns/openshift-monitoring pod/node-exporter-9r97w node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.843 W ns/openshift-monitoring pod/node-exporter-9r97w node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.852 W ns/openshift-image-registry pod/node-ca-lhw7l node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.860 W ns/openshift-monitoring endpoints/node-exporter Failed to update endpoint openshift-monitoring/node-exporter: Operation cannot be fulfilled on endpoints "node-exporter": the object has been modified; please apply your changes to the latest version and try again Nov 06 18:14:13.862 W ns/openshift-image-registry pod/node-ca-lhw7l node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.873 W ns/openshift-multus pod/multus-lhw9n node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.881 W ns/openshift-multus pod/multus-lhw9n node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.887 W ns/openshift-sdn pod/ovs-cjz7b node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.895 W ns/openshift-sdn pod/ovs-cjz7b node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.905 W ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.920 W ns/openshift-machine-config-operator pod/machine-config-daemon-f96x2 node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:13.931 W ns/openshift-dns pod/dns-default-m7g9w node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:13.935 W ns/openshift-dns pod/dns-default-m7g9w node/ci-op--kg4jd-w-b-526wp.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:14.528 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Node "ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal" drained (5 times) Nov 06 18:14:15.221 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl requeue in: 20s (5 times) Nov 06 18:14:16.321 - 29s W node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal node is not ready Nov 06 18:14:16.965 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 (3 times) Nov 06 18:14:18.466 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal condition DiskPressure changed Nov 06 18:14:18.466 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal condition MemoryPressure changed Nov 06 18:14:18.466 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal condition PIDPressure changed Nov 06 18:14:18.466 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal condition Ready changed Nov 06 18:14:18.475 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal status is now: NodeNotReady Nov 06 18:14:18.685 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Node "ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal" drained (4 times) Nov 06 18:14:18.722 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available Nov 06 18:14:19.562 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s (4 times) Nov 06 18:14:24.958 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Readiness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 (2 times) Nov 06 18:14:26.825 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Node "ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal" drained (5 times) Nov 06 18:14:26.961 W ns/openshift-marketplace pod/community-operators-74b7658499-pvhcd Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 (4 times) Nov 06 18:14:27.659 W ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk requeue in: 20s (5 times) Nov 06 18:14:31.321 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal node is not ready Nov 06 18:14:35.281 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Node "ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal" drained (6 times) Nov 06 18:14:36.042 W ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl requeue in: 20s (6 times) Nov 06 18:14:41.307 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (7 times) Nov 06 18:14:41.901 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (32 times) Nov 06 18:14:45.010 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal event: DeletingNode Nov 06 18:14:45.015 W node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:48.228 I ns/openshift-machine-api machine/ci-op--kg4jd-w-d-4rskk Deleted machine ci-op--kg4jd-w-d-4rskk Nov 06 18:14:48.588 I node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal event: Removing Node ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal from Controller Nov 06 18:14:50.052 I ns/openshift-ingress service/router-default Updated load balancer with new hosts (2 times) Nov 06 18:14:50.304 I node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal event: DeletingNode Nov 06 18:14:50.313 W node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:53.587 I node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal Node ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal event: Removing Node ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal from Controller Nov 06 18:14:53.954 W ns/openshift-image-registry pod/node-ca-dpq6t node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:53.959 W ns/openshift-image-registry pod/node-ca-dpq6t node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:53.970 W ns/openshift-dns pod/dns-default-2hffn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:53.975 W ns/openshift-dns pod/dns-default-2hffn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:53.984 W ns/openshift-sdn pod/sdn-xq5lm node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:53.996 W ns/openshift-sdn pod/sdn-xq5lm node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.007 W ns/openshift-image-registry pod/node-ca-287kk node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.018 W ns/openshift-image-registry pod/node-ca-287kk node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.025 W ns/openshift-monitoring pod/node-exporter-52wfv node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.034 W ns/openshift-monitoring pod/node-exporter-52wfv node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.043 W ns/openshift-multus pod/multus-ht2rn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.051 W ns/openshift-multus pod/multus-ht2rn node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.064 W ns/openshift-sdn pod/ovs-5dgr8 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.075 W ns/openshift-sdn pod/ovs-5dgr8 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.090 W clusteroperator/network changed Progressing to False Nov 06 18:14:54.090 W ns/openshift-machine-config-operator pod/machine-config-daemon-zmnmt node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.099 W ns/openshift-machine-config-operator pod/machine-config-daemon-zmnmt node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.104 W ns/openshift-cluster-node-tuning-operator pod/tuned-8jh57 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.112 W ns/openshift-cluster-node-tuning-operator pod/tuned-8jh57 node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.114 W ns/openshift-machine-config-operator endpoints/machine-config-daemon Failed to update endpoint openshift-machine-config-operator/machine-config-daemon: Operation cannot be fulfilled on endpoints "machine-config-daemon": the object has been modified; please apply your changes to the latest version and try again Nov 06 18:14:54.118 W clusteroperator/dns changed Degraded to False: AsExpected: All desired DNS DaemonSets available and operand Namespace exists Nov 06 18:14:54.118 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal Nov 06 18:14:54.131 W ns/openshift-multus pod/multus-7jm82 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.133 W ns/openshift-multus pod/multus-7jm82 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.138 W ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.142 W ns/openshift-cluster-node-tuning-operator pod/tuned-ssr4z node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.149 W ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.158 W ns/openshift-machine-config-operator pod/machine-config-daemon-dfjl9 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.167 W ns/openshift-monitoring pod/node-exporter-xvxcw node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.176 W ns/openshift-monitoring pod/node-exporter-xvxcw node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.184 W ns/openshift-sdn pod/ovs-9tbpr node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.187 W ns/openshift-sdn pod/ovs-9tbpr node/ci-op--kg4jd-w-c-9tdzl.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.195 W ns/openshift-sdn pod/sdn-tx8lf node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.200 W ns/openshift-sdn pod/sdn-tx8lf node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:54.210 W ns/openshift-dns pod/dns-default-4kkr9 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal graceful deletion within 0s Nov 06 18:14:54.222 W ns/openshift-dns pod/dns-default-4kkr9 node/ci-op--kg4jd-w-d-4rskk.c.openshift-gce-devel-ci.internal deleted Nov 06 18:14:56.538 I ns/openshift-machine-api machine/ci-op--kg4jd-w-c-9tdzl Deleted machine ci-op--kg4jd-w-c-9tdzl Nov 06 18:17:14.600 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "" to "OperatorSyncDegraded: failed syncing configuration objects: config maps [v4-0-config-user-idp-0-ca] in openshift-authentication not synced" Nov 06 18:17:15.085 I ns/openshift-authentication-operator deployment/authentication-operator Created ConfigMap/v4-0-config-user-idp-0-ca -n openshift-authentication because it was missing Nov 06 18:17:17.681 I ns/openshift-authentication-operator deployment/authentication-operator Updated ConfigMap/v4-0-config-system-cliconfig -n openshift-authentication: cause by changes in data.v4-0-config-system-cliconfig Nov 06 18:17:18.333 I ns/openshift-authentication-operator deployment/authentication-operator Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed Nov 06 18:17:18.356 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-f74fd64bd to 1 Nov 06 18:17:18.378 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v node/ created Nov 06 18:17:18.383 I ns/openshift-authentication replicaset/oauth-openshift-f74fd64bd Created pod: oauth-openshift-f74fd64bd-s9p6v Nov 06 18:17:18.435 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v Successfully assigned openshift-authentication/oauth-openshift-f74fd64bd-s9p6v to ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Nov 06 18:17:18.725 W clusteroperator/authentication changed Progressing to True: ProgressingOAuthServerDeploymentNotReady: Progressing: deployment's observed generation did not reach the expected generation Nov 06 18:17:18.733 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "OperatorSyncDegraded: failed syncing configuration objects: config maps [v4-0-config-user-idp-0-ca] in openshift-authentication not synced" to "",Progressing changed from False to True ("Progressing: deployment's observed generation did not reach the expected generation") Nov 06 18:17:22.130 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing message changed from "Progressing: deployment's observed generation did not reach the expected generation" to "Progressing: not all deployment replicas are ready" Nov 06 18:17:26.431 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v Pulling image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:098c91095a31cb7688012c69626f5d18f574151106e92f788131d2dd3c2f5112" Nov 06 18:17:29.221 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:098c91095a31cb7688012c69626f5d18f574151106e92f788131d2dd3c2f5112" Nov 06 18:17:29.505 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v Created container oauth-openshift Nov 06 18:17:29.563 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v Started container oauth-openshift Nov 06 18:17:35.499 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-87dccc9c4 to 1 Nov 06 18:17:35.502 W ns/openshift-authentication pod/oauth-openshift-87dccc9c4-7glb6 node/ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:17:35.515 I ns/openshift-authentication replicaset/oauth-openshift-87dccc9c4 Deleted pod: oauth-openshift-87dccc9c4-7glb6 Nov 06 18:17:35.522 I ns/openshift-authentication pod/oauth-openshift-87dccc9c4-7glb6 Stopping container oauth-openshift Nov 06 18:17:35.529 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-f74fd64bd to 2 Nov 06 18:17:35.560 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx node/ created Nov 06 18:17:35.566 I ns/openshift-authentication replicaset/oauth-openshift-f74fd64bd Created pod: oauth-openshift-f74fd64bd-5mglx Nov 06 18:17:35.581 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx Successfully assigned openshift-authentication/oauth-openshift-f74fd64bd-5mglx to ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Nov 06 18:17:36.851 W ns/openshift-authentication pod/oauth-openshift-87dccc9c4-7glb6 node/ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:17:36.851 W ns/openshift-authentication pod/oauth-openshift-87dccc9c4-7glb6 node/ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal container=oauth-openshift container stopped being ready Nov 06 18:17:37.861 W ns/openshift-authentication pod/oauth-openshift-87dccc9c4-7glb6 node/ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal deleted Nov 06 18:17:43.730 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:098c91095a31cb7688012c69626f5d18f574151106e92f788131d2dd3c2f5112" already present on machine Nov 06 18:17:43.995 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx Created container oauth-openshift Nov 06 18:17:44.040 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx Started container oauth-openshift Nov 06 18:17:46.436 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "" to "RouteHealthDegraded: failed to GET route: net/http: TLS handshake timeout" Nov 06 18:17:47.898 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Degraded message changed from "RouteHealthDegraded: failed to GET route: net/http: TLS handshake timeout" to "" Nov 06 18:17:53.681 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-87dccc9c4 to 0 Nov 06 18:17:53.688 W ns/openshift-authentication pod/oauth-openshift-87dccc9c4-h6kkh node/ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:17:53.696 I ns/openshift-authentication replicaset/oauth-openshift-87dccc9c4 Deleted pod: oauth-openshift-87dccc9c4-h6kkh Nov 06 18:17:53.709 I ns/openshift-authentication pod/oauth-openshift-87dccc9c4-h6kkh Stopping container oauth-openshift Nov 06 18:17:54.474 W clusteroperator/authentication changed Progressing to False Nov 06 18:17:54.483 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing changed from True to False ("") Nov 06 18:17:57.977 W ns/openshift-authentication pod/oauth-openshift-87dccc9c4-h6kkh node/ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal deleted Nov 06 18:18:01.035 I ns/openshift-authentication-operator deployment/authentication-operator Updated ConfigMap/v4-0-config-system-cliconfig -n openshift-authentication: cause by changes in data.v4-0-config-system-cliconfig (2 times) Nov 06 18:18:01.877 I ns/openshift-authentication-operator deployment/authentication-operator Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed (2 times) Nov 06 18:18:01.899 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-645c5dcb54 to 1 Nov 06 18:18:01.923 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-6zccd node/ created Nov 06 18:18:01.930 I ns/openshift-authentication replicaset/oauth-openshift-645c5dcb54 Created pod: oauth-openshift-645c5dcb54-6zccd Nov 06 18:18:01.946 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-6zccd Successfully assigned openshift-authentication/oauth-openshift-645c5dcb54-6zccd to ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Nov 06 18:18:02.287 W clusteroperator/authentication changed Progressing to True: ProgressingOAuthServerDeploymentNotReady: Progressing: deployment's observed generation did not reach the expected generation Nov 06 18:18:02.294 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing changed from False to True ("Progressing: deployment's observed generation did not reach the expected generation") Nov 06 18:18:05.689 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing message changed from "Progressing: deployment's observed generation did not reach the expected generation" to "Progressing: not all deployment replicas are ready" (2 times) Nov 06 18:18:09.330 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-6zccd Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:098c91095a31cb7688012c69626f5d18f574151106e92f788131d2dd3c2f5112" already present on machine Nov 06 18:18:09.524 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-6zccd Created container oauth-openshift Nov 06 18:18:09.547 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-6zccd Started container oauth-openshift Nov 06 18:18:10.971 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-f74fd64bd to 1 Nov 06 18:18:10.972 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx node/ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:18:10.981 I ns/openshift-authentication replicaset/oauth-openshift-f74fd64bd Deleted pod: oauth-openshift-f74fd64bd-5mglx Nov 06 18:18:10.986 I ns/openshift-authentication deployment/oauth-openshift Scaled up replica set oauth-openshift-645c5dcb54 to 2 Nov 06 18:18:10.997 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx Stopping container oauth-openshift Nov 06 18:18:11.013 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-hwhfz node/ created Nov 06 18:18:11.018 I ns/openshift-authentication replicaset/oauth-openshift-645c5dcb54 Created pod: oauth-openshift-645c5dcb54-hwhfz Nov 06 18:18:11.038 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-hwhfz Successfully assigned openshift-authentication/oauth-openshift-645c5dcb54-hwhfz to ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Nov 06 18:18:17.979 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-5mglx node/ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal deleted Nov 06 18:18:19.070 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-hwhfz Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:098c91095a31cb7688012c69626f5d18f574151106e92f788131d2dd3c2f5112" already present on machine Nov 06 18:18:19.314 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-hwhfz Created container oauth-openshift Nov 06 18:18:19.353 I ns/openshift-authentication pod/oauth-openshift-645c5dcb54-hwhfz Started container oauth-openshift Nov 06 18:18:28.468 I ns/openshift-authentication deployment/oauth-openshift Scaled down replica set oauth-openshift-f74fd64bd to 0 Nov 06 18:18:28.470 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v node/ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:18:28.488 I ns/openshift-authentication replicaset/oauth-openshift-f74fd64bd Deleted pod: oauth-openshift-f74fd64bd-s9p6v Nov 06 18:18:28.497 I ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v Stopping container oauth-openshift Nov 06 18:18:29.568 W clusteroperator/authentication changed Progressing to False Nov 06 18:18:29.578 I ns/openshift-authentication-operator deployment/authentication-operator Status for clusteroperator/authentication changed: Progressing changed from True to False ("") (2 times) Nov 06 18:18:30.049 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v node/ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:18:30.049 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v node/ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal container=oauth-openshift container stopped being ready Nov 06 18:18:31.321 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v node/ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 18:18:33.119 W ns/openshift-authentication pod/oauth-openshift-f74fd64bd-s9p6v node/ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal deleted Nov 06 18:20:35.625 W ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 18:20:35.633 W ns/openshift-image-registry pod/node-ca-zstvb node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:20:35.634 W ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 120s Nov 06 18:20:35.635 W ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:20:35.638 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:20:35.650 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Stopping container machine-config-daemon Nov 06 18:20:35.650 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-m2gmn Nov 06 18:20:35.650 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-zstvb Nov 06 18:20:35.664 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-m2gmn Nov 06 18:20:35.666 I ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn Stopping container oauth-proxy Nov 06 18:20:35.666 I ns/openshift-image-registry pod/node-ca-zstvb Marking for deletion Pod openshift-image-registry/node-ca-zstvb Nov 06 18:20:35.669 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Marking for deletion Pod openshift-ingress/router-default-ffd95fb7d-pd54q Nov 06 18:20:35.676 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Marking for deletion Pod openshift-monitoring/prometheus-adapter-74ccdcf5b7-47k9x Nov 06 18:20:35.680 I ns/openshift-image-registry pod/node-ca-zstvb Stopping container node-ca Nov 06 18:20:35.698 I ns/openshift-monitoring pod/alertmanager-main-1 Marking for deletion Pod openshift-monitoring/alertmanager-main-1 (2 times) Nov 06 18:20:35.698 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager-proxy Nov 06 18:20:35.705 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container config-reloader Nov 06 18:20:35.715 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nlrbf node/ created Nov 06 18:20:35.721 I ns/openshift-monitoring replicaset/prometheus-adapter-74ccdcf5b7 Created pod: prometheus-adapter-74ccdcf5b7-nlrbf Nov 06 18:20:35.730 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager Nov 06 18:20:35.751 I ns/openshift-ingress pod/router-default-ffd95fb7d-zk8l8 node/ created Nov 06 18:20:35.756 I ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q Stopping container router Nov 06 18:20:35.761 I ns/openshift-ingress replicaset/router-default-ffd95fb7d Created pod: router-default-ffd95fb7d-zk8l8 Nov 06 18:20:35.779 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nlrbf Successfully assigned openshift-monitoring/prometheus-adapter-74ccdcf5b7-nlrbf to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:20:35.779 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x Stopping container prometheus-adapter Nov 06 18:20:35.801 I ns/openshift-ingress pod/router-default-ffd95fb7d-zk8l8 Successfully assigned openshift-ingress/router-default-ffd95fb7d-zk8l8 to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:20:35.983 W clusteroperator/ingress changed Available to False: IngressUnavailable: Not all ingress controllers are available. Nov 06 18:20:35.983 W clusteroperator/ingress changed Progressing to True: Reconciling: Not all ingress controllers are available. Nov 06 18:20:37.052 W ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:20:37.052 W ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready Nov 06 18:20:37.052 W ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=machine-config-daemon container stopped being ready Nov 06 18:20:37.078 E ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 06 18:20:37.078 E ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=machine-config-daemon container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 06 18:20:37.104 W ns/openshift-image-registry pod/node-ca-zstvb node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:20:37.104 W ns/openshift-image-registry pod/node-ca-zstvb node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready Nov 06 18:20:37.130 E ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=router container exited with code 2 (Error): Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:40.526Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:45.525Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:53.721Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:17:58.710Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:10.999Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:15.995Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:28.495Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:18:33.490Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:19:14.929Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:19:20.102Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n2019-11-06T18:20:35.711Z INFO router.template template/router.go:548 router reloaded {"output": " - Checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n"}\n Nov 06 18:20:37.186 E ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=prometheus-adapter container exited with code 2 (Error): I1106 18:13:32.468020 1 adapter.go:93] successfully using in-cluster auth\nI1106 18:13:33.638890 1 secure_serving.go:116] Serving securely on [::]:6443\n Nov 06 18:20:37.226 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=alertmanager-proxy container exited with code 2 (Error): -monitoring:alertmanager-main\n2019/11/06 18:13:54 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token\n2019/11/06 18:13:54 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates.\n2019/11/06 18:13:54 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:9093/"\n2019/11/06 18:13:54 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics"\n2019/11/06 18:13:54 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:openshift-monitoring:alertmanager-main\n2019/11/06 18:13:54 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled\n2019/11/06 18:13:54 http.go:96: HTTPS: listening on [::]:9095\n2019/11/06 18:13:57 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:13:59 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:00 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:02 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:02 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:04 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:05 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:07 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:09 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:12 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n2019/11/06 18:14:14 reverseproxy.go:447: http: proxy error: dial tcp [::1]:9093: connect: connection refused\n Nov 06 18:20:37.226 E ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=config-reloader container exited with code 2 (Error): 2019/11/06 18:13:54 Watching directory: "/etc/alertmanager/config"\n Nov 06 18:20:38.073 W ns/openshift-ingress pod/router-default-ffd95fb7d-pd54q node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:20:38.127 W ns/openshift-image-registry pod/node-ca-zstvb node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:20:38.175 W ns/openshift-monitoring pod/alertmanager-main-1 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:20:38.216 I ns/openshift-monitoring pod/alertmanager-main-1 node/ created Nov 06 18:20:38.223 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-1 in StatefulSet alertmanager-main successful (3 times) Nov 06 18:20:38.248 I ns/openshift-monitoring pod/alertmanager-main-1 Successfully assigned openshift-monitoring/alertmanager-main-1 to ci-op--kg4jd-w-d-sr9bq.c.openshift-gce-devel-ci.internal Nov 06 18:20:43.329 W ns/openshift-machine-config-operator pod/machine-config-daemon-m2gmn node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:20:43.359 W ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-47k9x node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:20:43.659 I ns/openshift-ingress pod/router-default-ffd95fb7d-zk8l8 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:bd70de74d6dc01dedf586047151a4e6f4116a04f853d3fce65f353726a0c64e3" already present on machine Nov 06 18:20:43.870 I ns/openshift-ingress pod/router-default-ffd95fb7d-zk8l8 Created container router Nov 06 18:20:43.907 I ns/openshift-ingress pod/router-default-ffd95fb7d-zk8l8 Started container router Nov 06 18:20:44.101 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nlrbf Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:832221e63bc0b28b204f04a861c773b7f808afd7b18922a6983e5f95fbeb7ae3" already present on machine Nov 06 18:20:44.334 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nlrbf Created container prometheus-adapter Nov 06 18:20:44.372 I ns/openshift-monitoring pod/prometheus-adapter-74ccdcf5b7-nlrbf Started container prometheus-adapter Nov 06 18:20:45.694 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:4a4d954ea2a2f7eff16f8c11d2b808286ff722c878d4d05dccf4d545580f1d3a" already present on machine Nov 06 18:20:45.920 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager Nov 06 18:20:45.982 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager Nov 06 18:20:45.986 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:63ff8d9f806970af770724d4ba8bc521885de81e3e22212042a8b770a83f1b94" already present on machine Nov 06 18:20:46.511 I ns/openshift-monitoring pod/alertmanager-main-1 Created container config-reloader Nov 06 18:20:46.548 I ns/openshift-monitoring pod/alertmanager-main-1 Started container config-reloader Nov 06 18:20:46.550 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:20:46.756 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager-proxy Nov 06 18:20:46.795 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager-proxy Nov 06 18:20:54.800 W clusteroperator/ingress changed Available to True: desired and current number of IngressControllers are equal Nov 06 18:20:54.800 W clusteroperator/ingress changed Progressing to False: desired and current number of IngressControllers are equal Nov 06 18:20:58.457 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (10 times) Nov 06 18:20:58.465 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (12 times) Nov 06 18:21:02.666 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t node/ created Nov 06 18:21:02.672 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-nzf8t Nov 06 18:21:02.687 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Successfully assigned openshift-machine-config-operator/machine-config-daemon-nzf8t to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:21:02.689 I ns/openshift-image-registry pod/node-ca-fd9mt node/ created Nov 06 18:21:02.695 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-fd9mt Nov 06 18:21:02.707 I ns/openshift-image-registry pod/node-ca-fd9mt Successfully assigned openshift-image-registry/node-ca-fd9mt to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:21:03.365 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" already present on machine Nov 06 18:21:03.513 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Created container machine-config-daemon Nov 06 18:21:03.547 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Started container machine-config-daemon Nov 06 18:21:03.552 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:21:03.715 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Created container oauth-proxy Nov 06 18:21:03.752 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Started container oauth-proxy Nov 06 18:21:10.420 I ns/openshift-image-registry pod/node-ca-fd9mt Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 18:21:10.574 I ns/openshift-image-registry pod/node-ca-fd9mt Created container node-ca Nov 06 18:21:10.605 I ns/openshift-image-registry pod/node-ca-fd9mt Started container node-ca Nov 06 18:22:25.345 I persistentvolume/pvc-cb45d96d-be0a-4e46-9a0f-6ffbde528b36 googleapi: Error 400: The disk resource 'projects/openshift-gce-devel-ci/zones/us-east1-c/disks/ci-op--kg4jd-dynamic-pvc-cb45d96d-be0a-4e46-9a0f-6ffbde528b36' is already being used by 'projects/openshift-gce-devel-ci/zones/us-east1-c/instances/ci-op--kg4jd-w-c-rkgcg', resourceInUseByAnotherResource Nov 06 18:22:46.135 W node/node1 deleted Nov 06 18:22:49.588 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (41 times) Nov 06 18:22:49.595 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (42 times) Nov 06 18:23:36.217 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-nzf8t Nov 06 18:23:36.219 W ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 18:23:36.226 W ns/openshift-image-registry pod/node-ca-fd9mt node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:23:36.228 I ns/openshift-image-registry pod/node-ca-fd9mt Marking for deletion Pod openshift-image-registry/node-ca-fd9mt Nov 06 18:23:36.231 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Stopping container machine-config-daemon Nov 06 18:23:36.232 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-nzf8t Nov 06 18:23:36.237 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-fd9mt Nov 06 18:23:36.238 I ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t Stopping container oauth-proxy Nov 06 18:23:36.243 I ns/openshift-image-registry pod/node-ca-fd9mt Stopping container node-ca Nov 06 18:23:37.579 W ns/openshift-image-registry pod/node-ca-fd9mt node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:23:37.579 W ns/openshift-image-registry pod/node-ca-fd9mt node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready Nov 06 18:23:37.595 W ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:23:37.595 W ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=machine-config-daemon container stopped being ready Nov 06 18:23:37.595 W ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container stopped being ready Nov 06 18:23:43.332 W ns/openshift-machine-config-operator pod/machine-config-daemon-nzf8t node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:23:43.345 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl node/ created Nov 06 18:23:43.349 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-zszcl Nov 06 18:23:43.366 W ns/openshift-image-registry pod/node-ca-fd9mt node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:23:43.370 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Successfully assigned openshift-machine-config-operator/machine-config-daemon-zszcl to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:23:43.394 I ns/openshift-image-registry pod/node-ca-jnnxf node/ created Nov 06 18:23:43.399 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-jnnxf Nov 06 18:23:43.409 I ns/openshift-image-registry pod/node-ca-jnnxf Successfully assigned openshift-image-registry/node-ca-jnnxf to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:23:44.065 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" already present on machine Nov 06 18:23:44.209 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Created container machine-config-daemon Nov 06 18:23:44.245 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Started container machine-config-daemon Nov 06 18:23:44.252 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:23:44.439 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Created container oauth-proxy Nov 06 18:23:44.471 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Started container oauth-proxy Nov 06 18:23:51.819 I ns/openshift-image-registry pod/node-ca-jnnxf Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 18:23:51.979 I ns/openshift-image-registry pod/node-ca-jnnxf Created container node-ca Nov 06 18:23:52.015 I ns/openshift-image-registry pod/node-ca-jnnxf Started container node-ca Nov 06 18:24:41.310 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (8 times) Nov 06 18:24:41.902 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (5 times) Nov 06 18:30:58.184 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (12 times) Nov 06 18:30:58.193 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-2.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (14 times) Nov 06 18:32:42.825 W ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 18:32:42.827 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-zszcl Nov 06 18:32:42.831 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-zszcl Nov 06 18:32:42.836 I ns/openshift-image-registry pod/node-ca-jnnxf Marking for deletion Pod openshift-image-registry/node-ca-jnnxf Nov 06 18:32:42.839 W ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:32:42.844 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-jnnxf Nov 06 18:32:42.850 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Stopping container oauth-proxy Nov 06 18:32:42.856 I ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl Stopping container machine-config-daemon Nov 06 18:32:42.866 I ns/openshift-image-registry pod/node-ca-jnnxf Stopping container node-ca Nov 06 18:32:43.369 E ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 18:32:44.402 W ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:32:44.402 W ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready Nov 06 18:32:44.415 E ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=node-ca container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 06 18:32:46.321 W ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 18:32:49.583 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal (combined from similar events): Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (49 times) Nov 06 18:32:49.588 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-0.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-000" not found) (10 times) Nov 06 18:32:53.362 W ns/openshift-image-registry pod/node-ca-jnnxf node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:32:53.394 W ns/openshift-machine-config-operator pod/machine-config-daemon-zszcl node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:33:47.845 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk node/ created Nov 06 18:33:47.856 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-gx2jk Nov 06 18:33:47.864 I ns/openshift-image-registry pod/node-ca-zmhm5 node/ created Nov 06 18:33:47.870 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-zmhm5 Nov 06 18:33:47.879 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Successfully assigned openshift-machine-config-operator/machine-config-daemon-gx2jk to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:33:47.884 I ns/openshift-image-registry pod/node-ca-zmhm5 Successfully assigned openshift-image-registry/node-ca-zmhm5 to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:33:48.832 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:41f4a7f7d21f7dccf4683c4bbc12996ad0cd9eb321203734503ad2cf47dbd396" already present on machine Nov 06 18:33:48.963 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Created container machine-config-daemon Nov 06 18:33:48.998 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Started container machine-config-daemon Nov 06 18:33:49.004 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:56c87990597a959642670546823fa8e1965e6f9e744abeab33f2d6f018a8deb7" already present on machine Nov 06 18:33:49.166 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Created container oauth-proxy Nov 06 18:33:49.200 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Started container oauth-proxy Nov 06 18:33:55.506 I ns/openshift-image-registry pod/node-ca-zmhm5 Container image "registry.svc.ci.openshift.org/ocp/4.3-2019-11-06-170148@sha256:f6eb3311f979eda2afbf98b9b8240237215076246bf2ea5d4a35f62aa09ff192" already present on machine Nov 06 18:33:55.669 I ns/openshift-image-registry pod/node-ca-zmhm5 Created container node-ca Nov 06 18:33:55.701 I ns/openshift-image-registry pod/node-ca-zmhm5 Started container node-ca Nov 06 18:34:41.319 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert" not found) (10 times) Nov 06 18:34:41.926 W ns/openshift-kube-apiserver pod/kube-apiserver-ci-op--kg4jd-m-1.c.openshift-gce-devel-ci.internal Removed file for secret: /%!(EXTRA *errors.StatusError=secrets "user-serving-cert-008" not found) (7 times) Nov 06 18:35:52.416 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-gx2jk Nov 06 18:35:52.418 W ns/openshift-image-registry pod/node-ca-zmhm5 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 30s Nov 06 18:35:52.419 W ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal graceful deletion within 600s Nov 06 18:35:52.421 I ns/openshift-image-registry pod/node-ca-zmhm5 Marking for deletion Pod openshift-image-registry/node-ca-zmhm5 Nov 06 18:35:52.431 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-zmhm5 Nov 06 18:35:52.447 I ns/openshift-image-registry pod/node-ca-zmhm5 Stopping container node-ca Nov 06 18:35:52.449 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-gx2jk Nov 06 18:35:52.454 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Stopping container oauth-proxy Nov 06 18:35:52.465 I ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk Stopping container machine-config-daemon Nov 06 18:35:52.977 E ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=oauth-proxy container exited with code 143 (Error): Nov 06 18:35:53.996 W ns/openshift-machine-config-operator pod/machine-config-daemon-gx2jk node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:35:54.012 W ns/openshift-image-registry pod/node-ca-zmhm5 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 06 18:35:54.012 W ns/openshift-image-registry pod/node-ca-zmhm5 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal container=node-ca container stopped being ready Nov 06 18:36:01.321 W ns/openshift-image-registry pod/node-ca-zmhm5 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal pod has been pending longer than a minute Nov 06 18:36:03.333 W ns/openshift-image-registry pod/node-ca-zmhm5 node/ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal deleted Nov 06 18:37:13.363 I ns/openshift-machine-config-operator pod/machine-config-daemon-l2glv node/ created Nov 06 18:37:13.367 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-l2glv Nov 06 18:37:13.394 I ns/openshift-machine-config-operator pod/machine-config-daemon-l2glv Successfully assigned openshift-machine-config-operator/machine-config-daemon-l2glv to ci-op--kg4jd-w-c-rkgcg.c.openshift-gce-devel-ci.internal Nov 06 18:37:13.395 I ns/openshift-image-registry pod/node-ca-8hsbf node/ created Nov 06 18:37:13.404 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-8hsbf
Find was mentions in log files
openshift-tests [Feature:Image][triggers][Serial] ImageStream API TestImageStreamMappingCreate [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream API TestImageStreamTagLifecycleHook [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream API TestImageStreamWithoutDockerImageConfig [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream admission TestImageStreamAdmitSpecUpdate [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream admission TestImageStreamAdmitStatusUpdate [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream admission TestImageStreamTagsAdmission [Suite:openshift/conformance/serial]
openshift-tests [Feature:Machines][Serial] Managed cluster should grow and decrease when scaling different machineSets simultaneously [Suite:openshift/conformance/serial]
openshift-tests [Feature:OpenShiftAuthorization][Serial] authorization TestAuthorizationResourceAccessReview should succeed [Suite:openshift/conformance/serial]
openshift-tests [Feature:ProjectAPI][Serial] TestUnprivilegedNewProjectDenied [Suite:openshift/conformance/serial]
openshift-tests [Serial] API data in etcd should be stored at the correct location and version for all resources [Suite:openshift/conformance/serial]
openshift-tests [Serial] [Feature:OAuthServer] [RequestHeaders] [IdP] test RequestHeaders IdP [Suite:openshift/conformance/serial]
openshift-tests [Suite:openshift/oauth][Serial] ldap group sync can sync groups from ldap [Suite:openshift/conformance/serial]
openshift-tests [k8s.io] [sig-node] kubelet [k8s.io] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. [Suite:openshift/conformance/serial] [Suite:k8s] [Serial]
openshift-tests [sig-api-machinery] Namespaces [Serial] should always delete fast (ALL of 100 namespaces in 150 seconds) [Feature:ComprehensiveNamespaceDraining] [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-api-machinery] Namespaces [Serial] should delete fast enough (90 percent of 100 namespaces in 150 seconds) [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-api-machinery] Namespaces [Serial] should ensure that all pods are removed when a namespace is deleted [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-api-machinery] Namespaces [Serial] should ensure that all services are removed when a namespace is deleted [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should not update pod when spec was updated and update strategy is OnDelete [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should retry creating failed daemon pods [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should rollback without unnecessary restarts [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should run and stop complex daemon [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should run and stop complex daemon with node affinity [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should run and stop simple daemon [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should update pod when spec was updated and update strategy is RollingUpdate [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-cli] Kubectl client Kubectl taint [Serial] should remove all the taints with the same key off a node [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-cli] Kubectl client Kubectl taint [Serial] should update the taint on a node [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-network] Service endpoints latency should not be very high [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s] [Serial]
openshift-tests [sig-scheduling] NoExecuteTaintManager Multiple Pods [Serial] evicts pods with minTolerationSeconds [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Multiple Pods [Serial] only evicts pods without tolerations from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] doesn't evict pod with tolerations from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] eventually evict pod with finite tolerations from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] evicts pods from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] removing taint cancels eviction [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] PodPriorityResolution [Serial] validates critical system priorities are created and resolved [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that NodeAffinity is respected if not matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that NodeSelector is respected if matching [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that NodeSelector is respected if not matching [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that required NodeAffinity setting is respected if matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that taints-tolerations is respected if matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPriorities [Serial] Pod should avoid nodes that have avoidPod annotation [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPriorities [Serial] Pod should be preferably scheduled to nodes pod can tolerate [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPriorities [Serial] Pod should be scheduled to node that don't match the PodAntiAffinity terms [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] EmptyDir wrapper volumes should not cause race condition when used for configmaps [Serial] [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-storage] PersistentVolumes-local Pods sharing a single local PV [Serial] all pods should be running [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] PersistentVolumes-local Stress with local volumes [Serial] should be able to process many pods and reuse local volumes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create volume metrics in Volume Manager [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create volume metrics with the correct PVC ref [Suite:openshift/conformance/serial] [Suite:k8s]
operator All images are built and tagged into stable
operator Create the release image containing all images built by this job
operator Find all of the input images from ocp/4.3:${component} and tag them into the output image stream
operator Run template e2e-gcp-serial - e2e-gcp-serial container lease
operator Run template e2e-gcp-serial - e2e-gcp-serial container setup
operator Run template e2e-gcp-serial - e2e-gcp-serial container teardown
operator Run template e2e-gcp-serial - e2e-gcp-serial container test
openshift-tests [registry][Serial][Suite:openshift/registry/serial] Image signature workflow can push a signed image to openshift registry and verify it [Suite:openshift/conformance/serial]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create bound pv/pvc count metrics for pvc controller after creating both pv and pvc [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create none metrics for pvc controller before creating any PV or PVC [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create unbound pv count metrics for pvc controller after creating pv only [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create unbound pvc count metrics for pvc controller after creating pvc only [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create metrics for total number of volumes in A/D Controller [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create metrics for total time taken in volume operations in P/V Controller [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create prometheus metrics for volume provisioning and attach/detach [Suite:openshift/conformance/serial] [Suite:k8s]