release-openshift-origin-installer-e2e-aws-serial-4.1 #1222

• artifacts
• build log

Test Failures

openshift-tests Monitor cluster while tests execute 1h6m

go run hack/e2e.go -v -test --test_args='--ginkgo.focus=openshift\-tests\sMonitor\scluster\swhile\stests\sexecute$'

3 error level events were detected during this test run:

Nov 06 17:14:18.237 E ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal container=node-ca container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 06 17:14:45.494 E ns/openshift-console pod/downloads-8497d7b5c9-cx6fl node/ip-10-0-129-112.ec2.internal container=download-server container exited with code 137 (Error): 
Nov 06 17:48:39.658 E ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated

				

Timeline:

Nov 06 17:02:35.193 I ns/openshift-kube-scheduler pod/revision-pruner-6-ip-10-0-151-127.ec2.internal node/ip-10-0-151-127.ec2.internal created
Nov 06 17:02:35.194 I ns/openshift-kube-scheduler pod/openshift-kube-scheduler-ip-10-0-128-73.ec2.internal node/ip-10-0-128-73.ec2.internal created
Nov 06 17:02:35.194 I ns/openshift-kube-scheduler pod/installer-6-ip-10-0-128-73.ec2.internal node/ip-10-0-128-73.ec2.internal created
Nov 06 17:02:43.223 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Updated node "ip-10-0-128-73.ec2.internal" from revision 3 to 5
Nov 06 17:02:43.225 W clusteroperator/kube-apiserver changed Progressing to False: AsExpected: Progressing: 3 nodes are at revision 5
Nov 06 17:02:43.968 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Updated ConfigMap/revision-status-5 -n openshift-kube-apiserver: cause by changes in data.status
Nov 06 17:02:47.564 I ns/openshift-kube-apiserver pod/revision-pruner-5-ip-10-0-128-73.ec2.internal node/ip-10-0-128-73.ec2.internal created
Nov 06 17:02:47.570 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Created Pod/revision-pruner-5-ip-10-0-128-73.ec2.internal -n openshift-kube-apiserver because it was missing
Nov 06 17:02:55.341 I ns/openshift-kube-apiserver pod/revision-pruner-5-ip-10-0-128-73.ec2.internal Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:850acd17bd122e5a3d3932c3179dd83fa74e534611a60ac1b2a46106bdd95332" already present on machine
Nov 06 17:02:55.545 I ns/openshift-kube-apiserver pod/revision-pruner-5-ip-10-0-128-73.ec2.internal Created container pruner
Nov 06 17:02:55.588 I ns/openshift-kube-apiserver pod/revision-pruner-5-ip-10-0-128-73.ec2.internal Started container pruner
Nov 06 17:03:17.986 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator (combined from similar events): Status for clusteroperator/kube-scheduler changed: Degraded message changed from "StaticPodsDegraded: nodes/ip-10-0-128-73.ec2.internal pods/openshift-kube-scheduler-ip-10-0-128-73.ec2.internal container=\"scheduler\" is not ready" to "" (12 times)
Nov 06 17:03:18.175 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator Updated node "ip-10-0-128-73.ec2.internal" from revision 5 to 6
Nov 06 17:03:18.178 W clusteroperator/kube-scheduler changed Progressing to False: AsExpected: Progressing: 3 nodes are at revision 6
Nov 06 17:03:18.187 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator (combined from similar events): Status for clusteroperator/kube-scheduler changed: Progressing changed from True to False ("Progressing: 3 nodes are at revision 6"),Available message changed from "Available: 3 nodes are active; 1 nodes are at revision 5; 2 nodes are at revision 6" to "Available: 3 nodes are active; 3 nodes are at revision 6" (13 times)
Nov 06 17:03:19.369 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator Updated ConfigMap/revision-status-6 -n openshift-kube-scheduler: cause by changes in data.status
Nov 06 17:03:24.571 I ns/openshift-kube-scheduler pod/revision-pruner-6-ip-10-0-128-73.ec2.internal node/ip-10-0-128-73.ec2.internal created
Nov 06 17:03:24.577 I ns/openshift-kube-scheduler-operator deployment/openshift-kube-scheduler-operator Created Pod/revision-pruner-6-ip-10-0-128-73.ec2.internal -n openshift-kube-scheduler because it was missing
Nov 06 17:03:32.823 I ns/openshift-kube-scheduler pod/revision-pruner-6-ip-10-0-128-73.ec2.internal Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:627e5d02ea526b260fa13cdcabf86528b8812cc319eafec034a22e3ea5107f79" already present on machine
Nov 06 17:03:32.987 I ns/openshift-kube-scheduler pod/revision-pruner-6-ip-10-0-128-73.ec2.internal Created container pruner
Nov 06 17:03:33.024 I ns/openshift-kube-scheduler pod/revision-pruner-6-ip-10-0-128-73.ec2.internal Started container pruner
Nov 06 17:06:30.672 W ns/openshift-image-registry pod/node-ca-vtblp node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:06:30.672 W ns/openshift-machine-config-operator pod/machine-config-daemon-n8ddd node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 17:06:30.685 I ns/openshift-image-registry pod/node-ca-vtblp Stopping container node-ca
Nov 06 17:06:30.685 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-vtblp
Nov 06 17:06:30.690 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb Marking for deletion Pod openshift-monitoring/telemeter-client-7949cc65b7-8hltb
Nov 06 17:06:30.696 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-xzx82 node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:06:30.698 W ns/openshift-console pod/downloads-8497d7b5c9-d5ljw node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:06:30.701 W ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:06:30.708 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:06:30.711 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-n8ddd
Nov 06 17:06:30.711 I ns/openshift-machine-config-operator pod/machine-config-daemon-n8ddd Stopping container machine-config-daemon
Nov 06 17:06:30.711 I ns/openshift-console pod/downloads-8497d7b5c9-d5ljw Marking for deletion Pod openshift-console/downloads-8497d7b5c9-d5ljw
Nov 06 17:06:30.711 W ns/openshift-monitoring pod/alertmanager-main-2 node/ip-10-0-129-112.ec2.internal graceful deletion within 0s
Nov 06 17:06:30.728 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-xzx82 Stopping container prometheus-adapter
Nov 06 17:06:30.728 I ns/openshift-image-registry pod/node-ca-vtblp Marking for deletion Pod openshift-image-registry/node-ca-vtblp
Nov 06 17:06:30.773 W ns/openshift-monitoring pod/alertmanager-main-2 node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:06:30.778 I ns/openshift-console pod/downloads-8497d7b5c9-d5ljw Stopping container download-server
Nov 06 17:06:30.779 I ns/openshift-machine-config-operator pod/machine-config-daemon-n8ddd Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-n8ddd
Nov 06 17:06:30.791 I ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 Stopping container grafana-proxy
Nov 06 17:06:30.798 I ns/openshift-monitoring pod/alertmanager-main-2 Marking for deletion Pod openshift-monitoring/alertmanager-main-2
Nov 06 17:06:30.800 I ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 Stopping container grafana
Nov 06 17:06:30.803 I ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 Marking for deletion Pod openshift-monitoring/grafana-6dd6898db8-98ml5
Nov 06 17:06:30.808 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb Stopping container kube-rbac-proxy
Nov 06 17:06:30.812 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-xzx82 Marking for deletion Pod openshift-monitoring/prometheus-adapter-858d8d596b-xzx82
Nov 06 17:06:30.822 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb Stopping container reload
Nov 06 17:06:30.822 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh node/ created
Nov 06 17:06:30.827 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl node/ created
Nov 06 17:06:30.844 I ns/openshift-monitoring replicaset/prometheus-adapter-858d8d596b Created pod: prometheus-adapter-858d8d596b-vstsh
Nov 06 17:06:30.844 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb Stopping container telemeter-client
Nov 06 17:06:30.854 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh Successfully assigned openshift-monitoring/prometheus-adapter-858d8d596b-vstsh to ip-10-0-129-112.ec2.internal
Nov 06 17:06:30.869 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy
Nov 06 17:06:30.869 I ns/openshift-console replicaset/downloads-8497d7b5c9 Created pod: downloads-8497d7b5c9-cx6fl
Nov 06 17:06:30.869 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl Successfully assigned openshift-console/downloads-8497d7b5c9-cx6fl to ip-10-0-129-112.ec2.internal
Nov 06 17:06:30.875 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ created
Nov 06 17:06:30.875 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ created
Nov 06 17:06:30.888 I ns/openshift-monitoring replicaset/grafana-6dd6898db8 Created pod: grafana-6dd6898db8-dvrdc
Nov 06 17:06:30.897 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader
Nov 06 17:06:30.909 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Successfully assigned openshift-monitoring/grafana-6dd6898db8-dvrdc to ip-10-0-129-112.ec2.internal
Nov 06 17:06:30.914 I ns/openshift-monitoring pod/alertmanager-main-2 node/ created
Nov 06 17:06:30.924 I ns/openshift-monitoring replicaset/telemeter-client-7949cc65b7 Created pod: telemeter-client-7949cc65b7-7ljbw
Nov 06 17:06:30.935 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Successfully assigned openshift-monitoring/telemeter-client-7949cc65b7-7ljbw to ip-10-0-129-112.ec2.internal
Nov 06 17:06:30.935 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-2 in StatefulSet alertmanager-main successful
Nov 06 17:06:30.952 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully assigned openshift-monitoring/alertmanager-main-2 to ip-10-0-129-112.ec2.internal
Nov 06 17:06:31.080 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager
Nov 06 17:06:31.283 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager (2 times)
Nov 06 17:06:31.483 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy (2 times)
Nov 06 17:06:31.684 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader (2 times)
Nov 06 17:06:39.226 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:06:39.226 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb node/ip-10-0-129-112.ec2.internal container=telemeter-client container stopped being ready
Nov 06 17:06:39.226 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb node/ip-10-0-129-112.ec2.internal container=kube-rbac-proxy container stopped being ready
Nov 06 17:06:39.226 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb node/ip-10-0-129-112.ec2.internal container=reload container stopped being ready
Nov 06 17:06:39.842 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-8hltb node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:06:41.425 W ns/openshift-image-registry pod/node-ca-vtblp node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:06:41.425 W ns/openshift-image-registry pod/node-ca-vtblp node/ip-10-0-129-112.ec2.internal container=node-ca container stopped being ready
Nov 06 17:06:41.632 W ns/openshift-image-registry pod/node-ca-vtblp node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:06:41.665 I ns/openshift-image-registry pod/node-ca-l5th7 node/ created
Nov 06 17:06:41.672 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-l5th7
Nov 06 17:06:41.679 I ns/openshift-image-registry pod/node-ca-l5th7 Successfully assigned openshift-image-registry/node-ca-l5th7 to ip-10-0-129-112.ec2.internal
Nov 06 17:06:42.027 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-xzx82 node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:06:42.027 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-xzx82 node/ip-10-0-129-112.ec2.internal container=prometheus-adapter container stopped being ready
Nov 06 17:06:42.631 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-xzx82 node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:06:43.626 W ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:06:43.626 W ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 node/ip-10-0-129-112.ec2.internal container=grafana container stopped being ready
Nov 06 17:06:43.626 W ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 node/ip-10-0-129-112.ec2.internal container=grafana-proxy container stopped being ready
Nov 06 17:06:43.868 W ns/openshift-monitoring pod/grafana-6dd6898db8-98ml5 node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:06:44.224 W ns/openshift-machine-config-operator pod/machine-config-daemon-n8ddd node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:06:44.224 W ns/openshift-machine-config-operator pod/machine-config-daemon-n8ddd node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container stopped being ready
Nov 06 17:06:44.431 W ns/openshift-machine-config-operator pod/machine-config-daemon-n8ddd node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:06:44.443 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw node/ created
Nov 06 17:06:44.451 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-g4shw
Nov 06 17:06:44.461 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw Successfully assigned openshift-machine-config-operator/machine-config-daemon-g4shw to ip-10-0-129-112.ec2.internal
Nov 06 17:06:44.658 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:af71e807058ebf53c38cecc886e2ab46bca1ba965f532480f936600c6bbc766a" already present on machine
Nov 06 17:06:44.811 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl Created container download-server
Nov 06 17:06:44.840 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl Started container download-server
Nov 06 17:06:44.992 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:8571fc8f990c3b6aef603a1d326fdee326c06f3593815555f25b785cc50604ee" already present on machine
Nov 06 17:06:45.175 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Created container grafana
Nov 06 17:06:45.203 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Started container grafana
Nov 06 17:06:45.208 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:378ecf916293697cd1382f8069ce6e7f501e98e565e42959e79af78a333c4855" already present on machine
Nov 06 17:06:45.333 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:3908d0f7d26a85d82c988764802416c7eff10825de2ca4acd171b300b593e04f" already present on machine
Nov 06 17:06:45.369 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Created container grafana-proxy
Nov 06 17:06:45.422 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Started container grafana-proxy
Nov 06 17:06:45.523 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh Created container prometheus-adapter
Nov 06 17:06:45.559 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh Started container prometheus-adapter
Nov 06 17:06:45.596 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:9dedc258e9d68545492b0e086fc630db4f13d74528fefa98f578f0117119c62d" already present on machine
Nov 06 17:06:45.855 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager
Nov 06 17:06:45.892 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager
Nov 06 17:06:45.899 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:392cb0fa312b208d969e66f74305943aa504b525c14310686d35d53c2d8be116" already present on machine
Nov 06 17:06:46.060 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 17:06:46.261 I ns/openshift-monitoring pod/alertmanager-main-2 Created container config-reloader
Nov 06 17:06:46.458 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:6a4ba83b4a17fc2298d447bbf7c19dfe9a661ac53272aac04467ce4fd14735ca" already present on machine
Nov 06 17:06:46.659 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw Created container machine-config-daemon
Nov 06 17:06:46.858 I ns/openshift-monitoring pod/alertmanager-main-2 Started container config-reloader
Nov 06 17:06:47.059 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:378ecf916293697cd1382f8069ce6e7f501e98e565e42959e79af78a333c4855" already present on machine
Nov 06 17:06:47.258 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw Started container machine-config-daemon
Nov 06 17:06:47.458 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Created container telemeter-client
Nov 06 17:06:47.660 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Started container telemeter-client
Nov 06 17:06:47.860 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:392cb0fa312b208d969e66f74305943aa504b525c14310686d35d53c2d8be116" already present on machine
Nov 06 17:06:48.059 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager-proxy
Nov 06 17:06:48.258 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager-proxy
Nov 06 17:06:48.458 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Created container reload
Nov 06 17:06:48.664 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Started container reload
Nov 06 17:06:48.858 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:23a3dc86d1cb4d468a756707f0ec389432823bd340272ac791c6b724416a1a97" already present on machine
Nov 06 17:06:49.058 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Created container kube-rbac-proxy
Nov 06 17:06:49.258 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Started container kube-rbac-proxy
Nov 06 17:06:50.299 I ns/openshift-image-registry pod/node-ca-l5th7 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 17:06:50.432 I ns/openshift-image-registry pod/node-ca-l5th7 Created container node-ca
Nov 06 17:06:50.458 I ns/openshift-image-registry pod/node-ca-l5th7 Started container node-ca
Nov 06 17:07:02.198 W ns/openshift-console pod/downloads-8497d7b5c9-d5ljw node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:07:02.198 W ns/openshift-console pod/downloads-8497d7b5c9-d5ljw node/ip-10-0-129-112.ec2.internal container=download-server container stopped being ready
Nov 06 17:07:05.029 W ns/openshift-console pod/downloads-8497d7b5c9-d5ljw node/ip-10-0-129-112.ec2.internal pod has been pending longer than a minute
Nov 06 17:07:08.838 W ns/openshift-console pod/downloads-8497d7b5c9-d5ljw node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:12:24.869 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l (9 times)
Nov 06 17:12:25.018 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz (9 times)
Nov 06 17:12:25.233 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd (9 times)
Nov 06 17:12:26.802 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-0 Updated machine ci-op-7b03hy11-52f51-hg859-master-0 (6 times)
Nov 06 17:12:28.192 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-1 Updated machine ci-op-7b03hy11-52f51-hg859-master-1 (6 times)
Nov 06 17:12:29.683 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-2 Updated machine ci-op-7b03hy11-52f51-hg859-master-2 (6 times)
Nov 06 17:14:14.698 W ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:14:14.701 W ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 17:14:14.706 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-l5th7
Nov 06 17:14:14.707 I ns/openshift-image-registry pod/node-ca-l5th7 Stopping container node-ca
Nov 06 17:14:14.710 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-g4shw
Nov 06 17:14:14.714 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-g4shw
Nov 06 17:14:14.719 I ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw Stopping container machine-config-daemon
Nov 06 17:14:14.721 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl Marking for deletion Pod openshift-console/downloads-8497d7b5c9-cx6fl
Nov 06 17:14:14.729 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:14:14.738 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh Stopping container prometheus-adapter
Nov 06 17:14:14.744 W ns/openshift-monitoring pod/alertmanager-main-2 node/ip-10-0-129-112.ec2.internal graceful deletion within 0s
Nov 06 17:14:14.747 I ns/openshift-image-registry pod/node-ca-l5th7 Marking for deletion Pod openshift-image-registry/node-ca-l5th7
Nov 06 17:14:14.747 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader
Nov 06 17:14:14.748 W ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:14:14.757 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:14:14.757 W ns/openshift-console pod/downloads-8497d7b5c9-cx6fl node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:14:14.757 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager
Nov 06 17:14:14.764 W ns/openshift-monitoring pod/alertmanager-main-2 node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:14:14.777 I ns/openshift-monitoring pod/alertmanager-main-2 Marking for deletion Pod openshift-monitoring/alertmanager-main-2 (2 times)
Nov 06 17:14:14.783 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy
Nov 06 17:14:14.785 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Marking for deletion Pod openshift-monitoring/telemeter-client-7949cc65b7-7ljbw
Nov 06 17:14:14.799 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Stopping container grafana
Nov 06 17:14:14.801 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Marking for deletion Pod openshift-monitoring/grafana-6dd6898db8-dvrdc
Nov 06 17:14:14.815 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh Marking for deletion Pod openshift-monitoring/prometheus-adapter-858d8d596b-vstsh
Nov 06 17:14:14.819 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-5fr7j node/ created
Nov 06 17:14:14.824 I ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc Stopping container grafana-proxy
Nov 06 17:14:14.834 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Stopping container kube-rbac-proxy
Nov 06 17:14:14.844 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-5fr7j Successfully assigned openshift-monitoring/prometheus-adapter-858d8d596b-5fr7j to ip-10-0-133-245.ec2.internal
Nov 06 17:14:14.844 I ns/openshift-monitoring replicaset/prometheus-adapter-858d8d596b Created pod: prometheus-adapter-858d8d596b-5fr7j
Nov 06 17:14:14.846 I ns/openshift-console pod/downloads-8497d7b5c9-xlgsb node/ created
Nov 06 17:14:14.846 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb node/ created
Nov 06 17:14:14.859 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Stopping container reload
Nov 06 17:14:14.866 I ns/openshift-monitoring replicaset/grafana-6dd6898db8 Created pod: grafana-6dd6898db8-m26wb
Nov 06 17:14:14.880 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk node/ created
Nov 06 17:14:14.883 I ns/openshift-console replicaset/downloads-8497d7b5c9 Created pod: downloads-8497d7b5c9-xlgsb
Nov 06 17:14:14.902 I ns/openshift-monitoring replicaset/telemeter-client-7949cc65b7 Created pod: telemeter-client-7949cc65b7-xkkfk
Nov 06 17:14:14.920 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw Stopping container telemeter-client
Nov 06 17:14:14.920 I ns/openshift-console pod/downloads-8497d7b5c9-xlgsb Successfully assigned openshift-console/downloads-8497d7b5c9-xlgsb to ip-10-0-133-245.ec2.internal
Nov 06 17:14:14.934 I ns/openshift-monitoring pod/alertmanager-main-2 node/ created
Nov 06 17:14:14.945 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Successfully assigned openshift-monitoring/grafana-6dd6898db8-m26wb to ip-10-0-133-245.ec2.internal
Nov 06 17:14:14.953 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Successfully assigned openshift-monitoring/telemeter-client-7949cc65b7-xkkfk to ip-10-0-133-245.ec2.internal
Nov 06 17:14:14.963 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-2 in StatefulSet alertmanager-main successful (2 times)
Nov 06 17:14:14.965 I ns/openshift-monitoring pod/alertmanager-main-2 Successfully assigned openshift-monitoring/alertmanager-main-2 to ip-10-0-133-245.ec2.internal
Nov 06 17:14:15.105 I ns/openshift-console pod/downloads-8497d7b5c9-cx6fl Stopping container download-server
Nov 06 17:14:15.312 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager (2 times)
Nov 06 17:14:15.507 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container alertmanager-proxy (2 times)
Nov 06 17:14:15.707 I ns/openshift-monitoring pod/alertmanager-main-2 Stopping container config-reloader (2 times)
Nov 06 17:14:17.036 W ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:14:17.036 W ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container stopped being ready
Nov 06 17:14:17.438 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:14:17.438 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ip-10-0-129-112.ec2.internal container=reload container stopped being ready
Nov 06 17:14:17.438 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ip-10-0-129-112.ec2.internal container=telemeter-client container stopped being ready
Nov 06 17:14:17.438 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ip-10-0-129-112.ec2.internal container=kube-rbac-proxy container stopped being ready
Nov 06 17:14:17.837 W ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:14:17.837 W ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal container=node-ca container stopped being ready
Nov 06 17:14:18.237 E ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal container=node-ca container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 06 17:14:18.638 W ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:14:18.638 W ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ip-10-0-129-112.ec2.internal container=grafana-proxy container stopped being ready
Nov 06 17:14:18.638 W ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ip-10-0-129-112.ec2.internal container=grafana container stopped being ready
Nov 06 17:14:19.036 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:14:19.036 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh node/ip-10-0-129-112.ec2.internal container=prometheus-adapter container stopped being ready
Nov 06 17:14:19.246 W ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-vstsh node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:14:19.852 W ns/openshift-monitoring pod/telemeter-client-7949cc65b7-7ljbw node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:14:20.029 W ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal pod has been pending longer than a minute
Nov 06 17:14:20.029 W ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw node/ip-10-0-129-112.ec2.internal pod has been pending longer than a minute
Nov 06 17:14:20.029 W ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ip-10-0-129-112.ec2.internal pod has been pending longer than a minute
Nov 06 17:14:20.444 W ns/openshift-image-registry pod/node-ca-l5th7 node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:14:21.051 W ns/openshift-monitoring pod/grafana-6dd6898db8-dvrdc node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:14:21.643 W ns/openshift-machine-config-operator pod/machine-config-daemon-g4shw node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:14:28.151 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-5fr7j Pulling image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:3908d0f7d26a85d82c988764802416c7eff10825de2ca4acd171b300b593e04f"
Nov 06 17:14:28.580 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Pulling image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:6a4ba83b4a17fc2298d447bbf7c19dfe9a661ac53272aac04467ce4fd14735ca"
Nov 06 17:14:28.641 I ns/openshift-console pod/downloads-8497d7b5c9-xlgsb Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:af71e807058ebf53c38cecc886e2ab46bca1ba965f532480f936600c6bbc766a" already present on machine
Nov 06 17:14:28.793 I ns/openshift-console pod/downloads-8497d7b5c9-xlgsb Created container download-server
Nov 06 17:14:28.817 I ns/openshift-console pod/downloads-8497d7b5c9-xlgsb Started container download-server
Nov 06 17:14:29.704 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:9dedc258e9d68545492b0e086fc630db4f13d74528fefa98f578f0117119c62d" already present on machine
Nov 06 17:14:29.798 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Pulling image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:8571fc8f990c3b6aef603a1d326fdee326c06f3593815555f25b785cc50604ee"
Nov 06 17:14:29.867 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager
Nov 06 17:14:29.889 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager
Nov 06 17:14:29.894 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:392cb0fa312b208d969e66f74305943aa504b525c14310686d35d53c2d8be116" already present on machine
Nov 06 17:14:30.095 I ns/openshift-monitoring pod/alertmanager-main-2 Created container config-reloader
Nov 06 17:14:30.115 I ns/openshift-monitoring pod/alertmanager-main-2 Started container config-reloader
Nov 06 17:14:30.120 I ns/openshift-monitoring pod/alertmanager-main-2 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:378ecf916293697cd1382f8069ce6e7f501e98e565e42959e79af78a333c4855" already present on machine
Nov 06 17:14:30.285 I ns/openshift-monitoring pod/alertmanager-main-2 Created container alertmanager-proxy
Nov 06 17:14:30.304 I ns/openshift-monitoring pod/alertmanager-main-2 Started container alertmanager-proxy
Nov 06 17:14:33.122 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-5fr7j Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:3908d0f7d26a85d82c988764802416c7eff10825de2ca4acd171b300b593e04f"
Nov 06 17:14:33.575 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-5fr7j Created container prometheus-adapter
Nov 06 17:14:33.649 I ns/openshift-monitoring pod/prometheus-adapter-858d8d596b-5fr7j Started container prometheus-adapter
Nov 06 17:14:34.332 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:6a4ba83b4a17fc2298d447bbf7c19dfe9a661ac53272aac04467ce4fd14735ca"
Nov 06 17:14:34.551 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Created container telemeter-client
Nov 06 17:14:34.584 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Started container telemeter-client
Nov 06 17:14:34.590 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:392cb0fa312b208d969e66f74305943aa504b525c14310686d35d53c2d8be116" already present on machine
Nov 06 17:14:34.883 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Created container reload
Nov 06 17:14:34.947 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Started container reload
Nov 06 17:14:34.951 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:23a3dc86d1cb4d468a756707f0ec389432823bd340272ac791c6b724416a1a97" already present on machine
Nov 06 17:14:35.193 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Created container kube-rbac-proxy
Nov 06 17:14:35.254 I ns/openshift-monitoring pod/telemeter-client-7949cc65b7-xkkfk Started container kube-rbac-proxy
Nov 06 17:14:38.064 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Successfully pulled image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:8571fc8f990c3b6aef603a1d326fdee326c06f3593815555f25b785cc50604ee"
Nov 06 17:14:38.260 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Created container grafana
Nov 06 17:14:38.289 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Started container grafana
Nov 06 17:14:38.295 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:378ecf916293697cd1382f8069ce6e7f501e98e565e42959e79af78a333c4855" already present on machine
Nov 06 17:14:38.646 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Created container grafana-proxy
Nov 06 17:14:38.691 I ns/openshift-monitoring pod/grafana-6dd6898db8-m26wb Started container grafana-proxy
Nov 06 17:14:45.494 E ns/openshift-console pod/downloads-8497d7b5c9-cx6fl node/ip-10-0-129-112.ec2.internal container=download-server container exited with code 137 (Error): 
Nov 06 17:14:48.836 W ns/openshift-console pod/downloads-8497d7b5c9-cx6fl node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:15:28.888 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 node/ created
Nov 06 17:15:28.894 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-nlcg6
Nov 06 17:15:28.905 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 Successfully assigned openshift-machine-config-operator/machine-config-daemon-nlcg6 to ip-10-0-129-112.ec2.internal
Nov 06 17:15:28.912 I ns/openshift-image-registry pod/node-ca-drb5l node/ created
Nov 06 17:15:28.919 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-drb5l
Nov 06 17:15:28.926 I ns/openshift-image-registry pod/node-ca-drb5l Successfully assigned openshift-image-registry/node-ca-drb5l to ip-10-0-129-112.ec2.internal
Nov 06 17:15:29.547 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 17:15:29.703 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 Created container machine-config-daemon
Nov 06 17:15:29.727 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 Started container machine-config-daemon
Nov 06 17:15:36.806 I ns/openshift-image-registry pod/node-ca-drb5l Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 17:15:36.944 I ns/openshift-image-registry pod/node-ca-drb5l Created container node-ca
Nov 06 17:15:36.964 I ns/openshift-image-registry pod/node-ca-drb5l Started container node-ca
Nov 06 17:22:24.817 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz (10 times)
Nov 06 17:22:24.969 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd (10 times)
Nov 06 17:22:26.371 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-0 Updated machine ci-op-7b03hy11-52f51-hg859-master-0 (7 times)
Nov 06 17:22:27.675 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-1 Updated machine ci-op-7b03hy11-52f51-hg859-master-1 (7 times)
Nov 06 17:22:29.000 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-2 Updated machine ci-op-7b03hy11-52f51-hg859-master-2 (7 times)
Nov 06 17:22:29.233 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l (10 times)
Nov 06 17:25:19.191 W ns/openshift-image-registry pod/node-ca-drb5l node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:25:19.192 W ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 17:25:19.202 I ns/openshift-image-registry pod/node-ca-drb5l Marking for deletion Pod openshift-image-registry/node-ca-drb5l
Nov 06 17:25:19.204 I ns/openshift-image-registry pod/node-ca-drb5l Stopping container node-ca
Nov 06 17:25:19.205 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-drb5l
Nov 06 17:25:19.208 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-nlcg6
Nov 06 17:25:19.212 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-nlcg6
Nov 06 17:25:19.215 I ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 Stopping container machine-config-daemon
Nov 06 17:25:28.836 W ns/openshift-image-registry pod/node-ca-drb5l node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:25:28.861 W ns/openshift-machine-config-operator pod/machine-config-daemon-nlcg6 node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:26:24.255 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj node/ created
Nov 06 17:26:24.267 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-z49qj
Nov 06 17:26:24.274 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj Successfully assigned openshift-machine-config-operator/machine-config-daemon-z49qj to ip-10-0-129-112.ec2.internal
Nov 06 17:26:24.278 I ns/openshift-image-registry pod/node-ca-mqgng node/ created
Nov 06 17:26:24.290 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-mqgng
Nov 06 17:26:24.293 I ns/openshift-image-registry pod/node-ca-mqgng Successfully assigned openshift-image-registry/node-ca-mqgng to ip-10-0-129-112.ec2.internal
Nov 06 17:26:24.940 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 17:26:25.069 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj Created container machine-config-daemon
Nov 06 17:26:25.095 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj Started container machine-config-daemon
Nov 06 17:26:32.017 I ns/openshift-image-registry pod/node-ca-mqgng Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 17:26:32.147 I ns/openshift-image-registry pod/node-ca-mqgng Created container node-ca
Nov 06 17:26:32.176 I ns/openshift-image-registry pod/node-ca-mqgng Started container node-ca
Nov 06 17:26:52.166 I ns/kube-system pod/pod0-system-node-critical node/ created
Nov 06 17:26:52.183 I ns/kube-system pod/pod1-system-cluster-critical node/ created
Nov 06 17:26:52.206 I ns/kube-system pod/pod0-system-node-critical Successfully assigned kube-system/pod0-system-node-critical to ip-10-0-129-112.ec2.internal
Nov 06 17:26:52.215 I ns/kube-system pod/pod1-system-cluster-critical Successfully assigned kube-system/pod1-system-cluster-critical to ip-10-0-129-112.ec2.internal
Nov 06 17:26:52.221 W ns/kube-system pod/pod1-system-cluster-critical node/ip-10-0-129-112.ec2.internal graceful deletion within 0s
Nov 06 17:26:52.228 W ns/kube-system pod/pod1-system-cluster-critical node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:26:52.253 W ns/kube-system pod/pod0-system-node-critical node/ip-10-0-129-112.ec2.internal graceful deletion within 0s
Nov 06 17:26:52.266 W ns/kube-system pod/pod0-system-node-critical node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:28:55.219 W ns/kube-system pod/pod0-system-node-critical Unable to mount volumes for pod "pod0-system-node-critical_kube-system(9f1b53bf-00ba-11ea-aef6-122443d5b077)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"pod0-system-node-critical". list of unmounted volumes=[default-token-btscc]. list of unattached volumes=[default-token-btscc]
Nov 06 17:28:55.229 W ns/kube-system pod/pod1-system-cluster-critical Unable to mount volumes for pod "pod1-system-cluster-critical_kube-system(9f1ec415-00ba-11ea-aef6-122443d5b077)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"pod1-system-cluster-critical". list of unmounted volumes=[default-token-btscc]. list of unattached volumes=[default-token-btscc]
Nov 06 17:30:15.118 W ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 17:30:15.122 W ns/openshift-image-registry pod/node-ca-mqgng node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:30:15.128 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-z49qj
Nov 06 17:30:15.128 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj Stopping container machine-config-daemon
Nov 06 17:30:15.135 I ns/openshift-image-registry pod/node-ca-mqgng Marking for deletion Pod openshift-image-registry/node-ca-mqgng
Nov 06 17:30:15.139 I ns/openshift-image-registry pod/node-ca-mqgng Stopping container node-ca
Nov 06 17:30:15.139 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-mqgng
Nov 06 17:30:15.142 I ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-z49qj
Nov 06 17:30:16.317 W ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:30:16.317 W ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container stopped being ready
Nov 06 17:30:20.029 W ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj node/ip-10-0-129-112.ec2.internal pod has been pending longer than a minute
Nov 06 17:30:28.835 W ns/openshift-machine-config-operator pod/machine-config-daemon-z49qj node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:30:28.849 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ created
Nov 06 17:30:28.854 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-x89ll
Nov 06 17:30:28.867 W ns/openshift-image-registry pod/node-ca-mqgng node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:30:28.873 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll Successfully assigned openshift-machine-config-operator/machine-config-daemon-x89ll to ip-10-0-129-112.ec2.internal
Nov 06 17:30:28.922 I ns/openshift-image-registry pod/node-ca-d58rt node/ created
Nov 06 17:30:28.931 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-d58rt
Nov 06 17:30:28.941 I ns/openshift-image-registry pod/node-ca-d58rt Successfully assigned openshift-image-registry/node-ca-d58rt to ip-10-0-129-112.ec2.internal
Nov 06 17:30:31.316 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 17:30:31.430 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll Created container machine-config-daemon
Nov 06 17:30:31.452 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll Started container machine-config-daemon
Nov 06 17:30:37.787 I ns/openshift-image-registry pod/node-ca-d58rt Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 17:30:37.937 I ns/openshift-image-registry pod/node-ca-d58rt Created container node-ca
Nov 06 17:30:37.959 I ns/openshift-image-registry pod/node-ca-d58rt Started container node-ca
Nov 06 17:32:24.789 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd (11 times)
Nov 06 17:32:26.319 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-0 Updated machine ci-op-7b03hy11-52f51-hg859-master-0 (8 times)
Nov 06 17:32:27.834 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-1 Updated machine ci-op-7b03hy11-52f51-hg859-master-1 (8 times)
Nov 06 17:32:29.093 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-2 Updated machine ci-op-7b03hy11-52f51-hg859-master-2 (8 times)
Nov 06 17:32:29.268 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l (11 times)
Nov 06 17:32:29.543 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz (11 times)
Nov 06 17:34:19.066 W persistentvolume/pvc-8fa82719-00bb-11ea-957b-1267e1b39625 Error deleting EBS volume "vol-09b37f2be9b347813" since volume is currently attached to "i-047ef3cbf8ee8bdbf"
Nov 06 17:42:26.032 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-0 Updated machine ci-op-7b03hy11-52f51-hg859-master-0 (9 times)
Nov 06 17:42:27.302 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-1 Updated machine ci-op-7b03hy11-52f51-hg859-master-1 (9 times)
Nov 06 17:42:28.684 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-2 Updated machine ci-op-7b03hy11-52f51-hg859-master-2 (9 times)
Nov 06 17:42:28.925 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l (12 times)
Nov 06 17:42:29.120 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz (12 times)
Nov 06 17:42:29.407 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd (12 times)
Nov 06 17:48:37.140 W ns/openshift-image-registry pod/node-ca-d58rt node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:48:37.141 W ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 17:48:37.152 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-d58rt
Nov 06 17:48:37.152 I ns/openshift-image-registry pod/node-ca-d58rt Marking for deletion Pod openshift-image-registry/node-ca-d58rt
Nov 06 17:48:37.154 I ns/openshift-image-registry pod/node-ca-d58rt Stopping container node-ca
Nov 06 17:48:37.163 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-x89ll
Nov 06 17:48:37.167 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll Stopping container machine-config-daemon
Nov 06 17:48:37.172 I ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-x89ll
Nov 06 17:48:39.256 W ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 17:48:39.256 W ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container stopped being ready
Nov 06 17:48:39.658 E ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated
Nov 06 17:48:48.846 W ns/openshift-machine-config-operator pod/machine-config-daemon-x89ll node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:48:48.872 W ns/openshift-image-registry pod/node-ca-d58rt node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:49:42.195 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc node/ created
Nov 06 17:49:42.209 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-s2jpc
Nov 06 17:49:42.216 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc Successfully assigned openshift-machine-config-operator/machine-config-daemon-s2jpc to ip-10-0-129-112.ec2.internal
Nov 06 17:49:42.222 I ns/openshift-image-registry pod/node-ca-ct8pv node/ created
Nov 06 17:49:42.229 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-ct8pv
Nov 06 17:49:42.238 I ns/openshift-image-registry pod/node-ca-ct8pv Successfully assigned openshift-image-registry/node-ca-ct8pv to ip-10-0-129-112.ec2.internal
Nov 06 17:49:42.880 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 17:49:43.012 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc Created container machine-config-daemon
Nov 06 17:49:43.041 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc Started container machine-config-daemon
Nov 06 17:49:50.491 I ns/openshift-image-registry pod/node-ca-ct8pv Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 17:49:50.638 I ns/openshift-image-registry pod/node-ca-ct8pv Created container node-ca
Nov 06 17:49:50.672 I ns/openshift-image-registry pod/node-ca-ct8pv Started container node-ca
Nov 06 17:52:26.022 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-0 Updated machine ci-op-7b03hy11-52f51-hg859-master-0 (10 times)
Nov 06 17:52:27.444 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-1 Updated machine ci-op-7b03hy11-52f51-hg859-master-1 (10 times)
Nov 06 17:52:28.791 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-2 Updated machine ci-op-7b03hy11-52f51-hg859-master-2 (10 times)
Nov 06 17:52:28.994 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l (13 times)
Nov 06 17:52:29.160 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz (13 times)
Nov 06 17:52:29.404 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd (13 times)
Nov 06 17:57:57.044 I ns/kube-system pod/critical-pod node/ created
Nov 06 17:57:57.050 W ns/kube-system pod/critical-pod 0/6 nodes are available: 3 Insufficient memory, 3 node(s) had taints that the pod didn't tolerate.
Nov 06 17:57:57.082 W ns/kube-system pod/critical-pod 0/6 nodes are available: 3 Insufficient memory, 3 node(s) had taints that the pod didn't tolerate. (2 times)
Nov 06 17:57:57.103 W ns/kube-system pod/critical-pod 0/6 nodes are available: 3 Insufficient memory, 3 node(s) had taints that the pod didn't tolerate. (3 times)
Nov 06 17:57:59.415 I ns/kube-system pod/critical-pod Successfully assigned kube-system/critical-pod to ip-10-0-129-112.ec2.internal
Nov 06 17:58:07.623 I ns/kube-system pod/critical-pod Container image "k8s.gcr.io/pause:3.1" already present on machine
Nov 06 17:58:07.771 I ns/kube-system pod/critical-pod Created container critical-pod
Nov 06 17:58:07.790 I ns/kube-system pod/critical-pod Started container critical-pod
Nov 06 17:58:09.164 W ns/kube-system pod/critical-pod node/ip-10-0-129-112.ec2.internal graceful deletion within 0s
Nov 06 17:58:09.169 W ns/kube-system pod/critical-pod node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:58:09.175 I ns/kube-system pod/critical-pod Stopping container critical-pod
Nov 06 17:58:09.428 I ns/kube-system pod/critical-pod Container image "k8s.gcr.io/pause:3.1" already present on machine (2 times)
Nov 06 17:58:09.674 W ns/kube-system pod/critical-pod Error: CreateContainer failed as the sandbox was stopped: 2b610d6f55ba0d8db40c9f66f36f4a71447f99f56f26066e48fd50401f663dee
Nov 06 17:59:23.674 W ns/openshift-image-registry pod/node-ca-ct8pv node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 17:59:23.676 W ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 17:59:23.685 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-ct8pv
Nov 06 17:59:23.685 I ns/openshift-image-registry pod/node-ca-ct8pv Stopping container node-ca
Nov 06 17:59:23.690 I ns/openshift-image-registry pod/node-ca-ct8pv Marking for deletion Pod openshift-image-registry/node-ca-ct8pv
Nov 06 17:59:23.692 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-s2jpc
Nov 06 17:59:23.693 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc Stopping container machine-config-daemon
Nov 06 17:59:23.695 I ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-s2jpc
Nov 06 17:59:28.837 W ns/openshift-machine-config-operator pod/machine-config-daemon-s2jpc node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:59:28.862 W ns/openshift-image-registry pod/node-ca-ct8pv node/ip-10-0-129-112.ec2.internal deleted
Nov 06 17:59:58.886 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t node/ created
Nov 06 17:59:58.894 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-h2m4t
Nov 06 17:59:58.900 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t Successfully assigned openshift-machine-config-operator/machine-config-daemon-h2m4t to ip-10-0-129-112.ec2.internal
Nov 06 17:59:58.909 I ns/openshift-image-registry pod/node-ca-jdpff node/ created
Nov 06 17:59:58.923 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-jdpff
Nov 06 17:59:58.930 I ns/openshift-image-registry pod/node-ca-jdpff Successfully assigned openshift-image-registry/node-ca-jdpff to ip-10-0-129-112.ec2.internal
Nov 06 17:59:59.600 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 17:59:59.741 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t Created container machine-config-daemon
Nov 06 17:59:59.771 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t Started container machine-config-daemon
Nov 06 18:00:07.219 I ns/openshift-image-registry pod/node-ca-jdpff Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 18:00:07.359 I ns/openshift-image-registry pod/node-ca-jdpff Created container node-ca
Nov 06 18:00:07.379 I ns/openshift-image-registry pod/node-ca-jdpff Started container node-ca
Nov 06 18:00:18.377 W ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t node/ip-10-0-129-112.ec2.internal graceful deletion within 600s
Nov 06 18:00:18.380 W ns/openshift-image-registry pod/node-ca-jdpff node/ip-10-0-129-112.ec2.internal graceful deletion within 30s
Nov 06 18:00:18.385 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t Stopping container machine-config-daemon
Nov 06 18:00:18.388 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-jdpff
Nov 06 18:00:18.389 I ns/openshift-image-registry pod/node-ca-jdpff Marking for deletion Pod openshift-image-registry/node-ca-jdpff
Nov 06 18:00:18.391 I ns/openshift-image-registry pod/node-ca-jdpff Stopping container node-ca
Nov 06 18:00:18.394 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-h2m4t
Nov 06 18:00:18.394 I ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-h2m4t
Nov 06 18:00:19.770 W ns/openshift-image-registry pod/node-ca-jdpff node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 18:00:19.770 W ns/openshift-image-registry pod/node-ca-jdpff node/ip-10-0-129-112.ec2.internal container=node-ca container stopped being ready
Nov 06 18:00:19.784 W ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t node/ip-10-0-129-112.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated
Nov 06 18:00:19.784 W ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t node/ip-10-0-129-112.ec2.internal container=machine-config-daemon container stopped being ready
Nov 06 18:00:28.834 W ns/openshift-machine-config-operator pod/machine-config-daemon-h2m4t node/ip-10-0-129-112.ec2.internal deleted
Nov 06 18:00:28.841 I ns/openshift-machine-config-operator pod/machine-config-daemon-jmxrc node/ created
Nov 06 18:00:28.851 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-jmxrc
Nov 06 18:00:28.852 I ns/openshift-machine-config-operator pod/machine-config-daemon-jmxrc Successfully assigned openshift-machine-config-operator/machine-config-daemon-jmxrc to ip-10-0-129-112.ec2.internal
Nov 06 18:00:28.866 W ns/openshift-image-registry pod/node-ca-jdpff node/ip-10-0-129-112.ec2.internal deleted
Nov 06 18:00:28.891 I ns/openshift-image-registry pod/node-ca-4qmbc node/ created
Nov 06 18:00:28.901 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-4qmbc
Nov 06 18:00:28.908 I ns/openshift-image-registry pod/node-ca-4qmbc Successfully assigned openshift-image-registry/node-ca-4qmbc to ip-10-0-129-112.ec2.internal
Nov 06 18:00:30.369 I ns/openshift-machine-config-operator pod/machine-config-daemon-jmxrc Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:bce339317b00fe5e8395efb40189c1dbfe38e9938a3cba7cd3566679446249a3" already present on machine
Nov 06 18:00:30.513 I ns/openshift-machine-config-operator pod/machine-config-daemon-jmxrc Created container machine-config-daemon
Nov 06 18:00:30.551 I ns/openshift-machine-config-operator pod/machine-config-daemon-jmxrc Started container machine-config-daemon
Nov 06 18:00:37.968 I ns/openshift-image-registry pod/node-ca-4qmbc Container image "registry.svc.ci.openshift.org/ocp/4.1-2019-11-06-163148@sha256:fb3eb50b4cdc7158c9e157a263f6154726ee21d9b8d0986dbfb59ddf09046401" already present on machine
Nov 06 18:00:38.108 I ns/openshift-image-registry pod/node-ca-4qmbc Created container node-ca
Nov 06 18:00:38.130 I ns/openshift-image-registry pod/node-ca-4qmbc Started container node-ca
Nov 06 18:02:26.082 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-2 Updated machine ci-op-7b03hy11-52f51-hg859-master-2 (11 times)
Nov 06 18:02:26.245 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1b-82t2l (14 times)
Nov 06 18:02:26.454 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-g7rjz (14 times)
Nov 06 18:02:26.603 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd Updated machine ci-op-7b03hy11-52f51-hg859-worker-us-east-1a-5t5gd (14 times)
Nov 06 18:02:27.906 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-0 Updated machine ci-op-7b03hy11-52f51-hg859-master-0 (11 times)
Nov 06 18:02:29.188 I ns/openshift-machine-api machine/ci-op-7b03hy11-52f51-hg859-master-1 Updated machine ci-op-7b03hy11-52f51-hg859-master-1 (11 times)


				Click to see stdout/stderrfrom junit_e2e_20191106-180928.xml

Find was mentions in log files