| Result | SUCCESS |
| Tests | 1 failed / 56 succeeded |
| Started | |
| Elapsed | 1h55m |
| Work namespace | ci-op-d9nlcxqc |
| pod | 4.2.0-0.nightly-2019-11-05-111920-aws-serial |
go run hack/e2e.go -v -test --test_args='--ginkgo.focus=openshift\-tests\sMonitor\scluster\swhile\stests\sexecute$'
11 error level events were detected during this test run: Nov 05 11:59:07.436 E ns/openshift-marketplace pod/redhat-operators-f5fd5d65-9wm5g node/ip-10-0-139-89.ec2.internal container=redhat-operators container exited with code 2 (Error): Nov 05 11:59:08.363 E ns/openshift-marketplace pod/community-operators-7c4757bbd5-cjnrh node/ip-10-0-150-53.ec2.internal container=community-operators container exited with code 2 (Error): Nov 05 11:59:08.637 E ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p node/ip-10-0-139-89.ec2.internal container=openshift-state-metrics container exited with code 2 (Error): Nov 05 11:59:09.164 E ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-449t9 node/ip-10-0-150-53.ec2.internal container=prometheus-adapter container exited with code 2 (Error): I1105 11:51:47.465904 1 adapter.go:93] successfully using in-cluster auth\nI1105 11:51:48.495255 1 secure_serving.go:116] Serving securely on [::]:6443\n Nov 05 11:59:10.364 E ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 node/ip-10-0-150-53.ec2.internal container=certified-operators container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 05 11:59:11.965 E ns/openshift-ingress pod/router-default-5796647cdf-x8md7 node/ip-10-0-150-53.ec2.internal container=router container exited with code 2 (Error): 1105 11:55:53.013434 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:56:07.666667 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:56:26.032612 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:56:31.028484 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nE1105 11:57:17.695030 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=21, ErrCode=NO_ERROR, debug=""\nE1105 11:57:17.695038 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=21, ErrCode=NO_ERROR, debug=""\nE1105 11:57:17.695030 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=21, ErrCode=NO_ERROR, debug=""\nW1105 11:57:18.734279 1 reflector.go:341] github.com/openshift/router/pkg/router/template/service_lookup.go:32: watch of *v1.Service ended with: too old resource version: 12231 (13688)\nI1105 11:57:26.257407 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:57:31.247581 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:57:36.250430 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:59:06.317934 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n Nov 05 11:59:27.444 E ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-137-180.ec2.internal container=prometheus container exited with code 1 (Error): Nov 05 12:43:38.090 E ns/openshift-image-registry pod/node-ca-2pr9z node/ip-10-0-139-89.ec2.internal container=node-ca container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 05 12:48:27.140 E clusteroperator/machine-config changed Degraded to True: MachineConfigDaemonFailed: Failed to resync 4.2.0-0.nightly-2019-11-05-111920 because: Operation cannot be fulfilled on daemonsets.apps "machine-config-daemon": the object has been modified; please apply your changes to the latest version and try again Nov 05 13:09:20.157 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available Nov 05 13:09:56.882 E ns/openshift-ingress pod/router-default-5796647cdf-sr9ft node/ip-10-0-150-53.ec2.internal container=router container exited with code 2 (Error): ter reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 12:16:48.708498 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 12:17:14.974496 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nW1105 12:23:28.302364 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nI1105 12:23:56.627168 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 12:24:01.623675 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nW1105 12:31:10.415533 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nW1105 12:40:43.490744 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nW1105 12:56:23.573194 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nI1105 13:09:41.126985 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 13:09:46.124267 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 13:09:51.126087 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nfrom junit_e2e_20191105-131059.xml
Timeline: Nov 05 11:57:33.408 W clusteroperator/kube-apiserver changed Progressing to False: AsExpected: Progressing: 3 nodes are at revision 7 Nov 05 11:57:33.419 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Updated node "ip-10-0-143-111.ec2.internal" from revision 3 to 7 because static pod is ready Nov 05 11:57:33.451 I ns/openshift-kube-controller-manager pod/revision-pruner-5-ip-10-0-143-111.ec2.internal node/ip-10-0-143-111.ec2.internal created Nov 05 11:57:34.333 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Updated ConfigMap/revision-status-7 -n openshift-kube-apiserver: cause by changes in data.status Nov 05 11:57:38.133 I ns/openshift-kube-apiserver pod/revision-pruner-7-ip-10-0-143-111.ec2.internal node/ip-10-0-143-111.ec2.internal created Nov 05 11:57:38.141 I ns/openshift-kube-apiserver-operator deployment/kube-apiserver-operator Created Pod/revision-pruner-7-ip-10-0-143-111.ec2.internal -n openshift-kube-apiserver because it was missing Nov 05 11:57:41.892 I ns/openshift-kube-controller-manager pod/revision-pruner-5-ip-10-0-143-111.ec2.internal Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0cc51b84b503a0fac694c93fc2ac120edb0cc57f5abaf52d9221f52c83b3c37e" already present on machine Nov 05 11:57:42.090 I ns/openshift-kube-controller-manager pod/revision-pruner-5-ip-10-0-143-111.ec2.internal Created container pruner Nov 05 11:57:42.116 I ns/openshift-kube-controller-manager pod/revision-pruner-5-ip-10-0-143-111.ec2.internal Started container pruner Nov 05 11:57:46.345 I ns/openshift-kube-apiserver pod/revision-pruner-7-ip-10-0-143-111.ec2.internal Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:91a7a30b47883cc4d6cfaed0d481451d7a810045452d9baf6ead760f8f70fe83" already present on machine Nov 05 11:57:46.557 I ns/openshift-kube-apiserver pod/revision-pruner-7-ip-10-0-143-111.ec2.internal Created container pruner Nov 05 11:57:46.596 I ns/openshift-kube-apiserver pod/revision-pruner-7-ip-10-0-143-111.ec2.internal Started container pruner Nov 05 11:59:06.228 W ns/openshift-image-registry pod/node-ca-tjqkj node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 11:59:06.230 W ns/openshift-machine-config-operator pod/machine-config-daemon-sn4w6 node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 11:59:06.237 I ns/openshift-image-registry pod/node-ca-tjqkj Stopping container node-ca Nov 05 11:59:06.237 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-tjqkj Nov 05 11:59:06.246 I ns/openshift-machine-config-operator pod/machine-config-daemon-sn4w6 Stopping container machine-config-daemon Nov 05 11:59:06.246 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-sn4w6 Nov 05 11:59:06.249 I ns/openshift-image-registry pod/node-ca-tjqkj Marking for deletion Pod openshift-image-registry/node-ca-tjqkj Nov 05 11:59:06.254 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p Marking for deletion Pod openshift-monitoring/openshift-state-metrics-557985667-f5c9p Nov 05 11:59:06.261 I ns/openshift-monitoring pod/alertmanager-main-0 Marking for deletion Pod openshift-monitoring/alertmanager-main-0 Nov 05 11:59:06.261 W ns/openshift-monitoring pod/alertmanager-main-0 node/ip-10-0-139-89.ec2.internal graceful deletion within 120s Nov 05 11:59:06.263 W ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 11:59:06.270 W ns/openshift-marketplace pod/redhat-operators-f5fd5d65-9wm5g node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 11:59:06.280 W ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 11:59:06.281 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v Marking for deletion Pod openshift-monitoring/telemeter-client-5c4c449647-vwh7v Nov 05 11:59:06.283 I ns/openshift-monitoring pod/alertmanager-main-0 Stopping container alertmanager-proxy Nov 05 11:59:06.294 I ns/openshift-monitoring pod/alertmanager-main-0 Stopping container config-reloader Nov 05 11:59:06.305 I ns/openshift-monitoring pod/alertmanager-main-0 Stopping container alertmanager Nov 05 11:59:06.306 I ns/openshift-machine-config-operator pod/machine-config-daemon-sn4w6 Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-sn4w6 Nov 05 11:59:06.325 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v Stopping container kube-rbac-proxy Nov 05 11:59:06.325 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-9wm5g Marking for deletion Pod openshift-marketplace/redhat-operators-f5fd5d65-9wm5g Nov 05 11:59:06.338 I ns/openshift-image-registry pod/node-ca-qtjsp Marking for deletion Pod openshift-image-registry/node-ca-qtjsp Nov 05 11:59:06.344 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr Marking for deletion Pod openshift-monitoring/grafana-5d5d6cdf5-s8smr Nov 05 11:59:06.349 I ns/openshift-monitoring pod/alertmanager-main-1 Marking for deletion Pod openshift-monitoring/alertmanager-main-1 Nov 05 11:59:06.351 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v Stopping container reload Nov 05 11:59:06.351 W ns/openshift-image-registry pod/node-ca-qtjsp node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 11:59:06.362 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 11:59:06.366 W ns/openshift-marketplace pod/community-operators-7c4757bbd5-cjnrh node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 11:59:06.371 W ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-150-53.ec2.internal graceful deletion within 120s Nov 05 11:59:06.375 W ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-449t9 node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 11:59:06.376 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 Marking for deletion Pod openshift-marketplace/certified-operators-57cdf6f756-c85g6 Nov 05 11:59:06.376 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v Stopping container telemeter-client Nov 05 11:59:06.376 I ns/openshift-image-registry pod/node-ca-qtjsp Stopping container node-ca Nov 05 11:59:06.381 W ns/openshift-machine-config-operator pod/machine-config-daemon-w9xwh node/ip-10-0-150-53.ec2.internal graceful deletion within 600s Nov 05 11:59:06.382 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 11:59:06.383 W ns/openshift-ingress pod/router-default-5796647cdf-x8md7 node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 11:59:06.385 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal graceful deletion within 600s Nov 05 11:59:06.393 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr Stopping container grafana-proxy Nov 05 11:59:06.398 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-9wm5g Stopping container redhat-operators Nov 05 11:59:06.425 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-cjnrh Marking for deletion Pod openshift-marketplace/community-operators-7c4757bbd5-cjnrh Nov 05 11:59:06.426 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p Stopping container openshift-state-metrics Nov 05 11:59:06.426 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr Stopping container grafana Nov 05 11:59:06.437 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-cjnrh Stopping container community-operators Nov 05 11:59:06.438 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 node/ created Nov 05 11:59:06.441 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p Stopping container kube-rbac-proxy-self Nov 05 11:59:06.445 I ns/openshift-ingress pod/router-default-5796647cdf-x8md7 Marking for deletion Pod openshift-ingress/router-default-5796647cdf-x8md7 Nov 05 11:59:06.453 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager-proxy Nov 05 11:59:06.458 I ns/openshift-monitoring replicaset/openshift-state-metrics-557985667 Created pod: openshift-state-metrics-557985667-bvq65 Nov 05 11:59:06.461 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 node/ created Nov 05 11:59:06.465 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-449t9 Marking for deletion Pod openshift-monitoring/prometheus-adapter-6d9d776844-449t9 Nov 05 11:59:06.471 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container config-reloader Nov 05 11:59:06.475 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Successfully assigned openshift-monitoring/openshift-state-metrics-557985667-bvq65 to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.480 I ns/openshift-marketplace replicaset/redhat-operators-f5fd5d65 Created pod: redhat-operators-f5fd5d65-cz8m7 Nov 05 11:59:06.480 I ns/openshift-monitoring pod/alertmanager-main-1 Stopping container alertmanager Nov 05 11:59:06.485 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss node/ created Nov 05 11:59:06.494 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 Successfully assigned openshift-marketplace/redhat-operators-f5fd5d65-cz8m7 to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.499 I ns/openshift-machine-config-operator pod/machine-config-daemon-w9xwh Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-w9xwh Nov 05 11:59:06.503 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-449t9 Stopping container prometheus-adapter Nov 05 11:59:06.512 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp node/ created Nov 05 11:59:06.513 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn node/ created Nov 05 11:59:06.515 I ns/openshift-monitoring replicaset/grafana-5d5d6cdf5 Created pod: grafana-5d5d6cdf5-pdkvp Nov 05 11:59:06.518 I ns/openshift-monitoring pod/prometheus-k8s-0 Marking for deletion Pod openshift-monitoring/prometheus-k8s-0 Nov 05 11:59:06.524 I ns/openshift-monitoring replicaset/telemeter-client-5c4c449647 Created pod: telemeter-client-5c4c449647-qg5ss Nov 05 11:59:06.524 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Successfully assigned openshift-monitoring/telemeter-client-5c4c449647-qg5ss to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.528 I ns/openshift-machine-config-operator pod/machine-config-daemon-w9xwh Stopping container machine-config-daemon Nov 05 11:59:06.540 I ns/openshift-marketplace replicaset/community-operators-7c4757bbd5 Created pod: community-operators-7c4757bbd5-2nvqn Nov 05 11:59:06.547 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp Successfully assigned openshift-monitoring/grafana-5d5d6cdf5-pdkvp to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.554 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 Stopping container certified-operators Nov 05 11:59:06.569 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Successfully assigned openshift-marketplace/community-operators-7c4757bbd5-2nvqn to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.579 I ns/openshift-ingress pod/router-default-5796647cdf-x8md7 Stopping container router Nov 05 11:59:06.594 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-jpz58 node/ created Nov 05 11:59:06.608 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq node/ created Nov 05 11:59:06.615 I ns/openshift-monitoring replicaset/prometheus-adapter-6d9d776844 Created pod: prometheus-adapter-6d9d776844-jpz58 Nov 05 11:59:06.622 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-jpz58 Successfully assigned openshift-monitoring/prometheus-adapter-6d9d776844-jpz58 to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.630 I ns/openshift-marketplace replicaset/certified-operators-57cdf6f756 Created pod: certified-operators-57cdf6f756-vd5hq Nov 05 11:59:06.630 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Successfully assigned openshift-marketplace/certified-operators-57cdf6f756-vd5hq to ip-10-0-137-180.ec2.internal Nov 05 11:59:06.642 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p Stopping container kube-rbac-proxy-main Nov 05 11:59:06.643 I ns/openshift-ingress pod/router-default-5796647cdf-sr9ft node/ created Nov 05 11:59:06.644 I ns/openshift-ingress replicaset/router-default-5796647cdf Created pod: router-default-5796647cdf-sr9ft Nov 05 11:59:06.649 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft 0/6 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 2 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector. Nov 05 11:59:06.762 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prometheus-config-reloader Nov 05 11:59:06.962 I ns/openshift-monitoring pod/prometheus-k8s-0 Stopping container prometheus Nov 05 11:59:07.161 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 Liveness probe errored: rpc error: code = Unknown desc = container is not created or running Nov 05 11:59:07.436 E ns/openshift-marketplace pod/redhat-operators-f5fd5d65-9wm5g node/ip-10-0-139-89.ec2.internal container=redhat-operators container exited with code 2 (Error): Nov 05 11:59:07.636 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-token-k94vv" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.642 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-config" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.647 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-prometheus" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.652 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-cluster" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.656 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-datasources" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.663 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-node-rsrc-use" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.667 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-workloads-namespace" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.672 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-etcd" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.677 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-workload" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.682 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboard-prometheus-remote-write" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:07.841 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboard-node-cluster-rsrc-use" : couldn't propagate object cache: timed out waiting for the condition (2 times) Nov 05 11:59:08.039 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "secret-grafana-tls" : couldn't propagate object cache: timed out waiting for the condition (3 times) Nov 05 11:59:08.238 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-pod" : couldn't propagate object cache: timed out waiting for the condition (4 times) Nov 05 11:59:08.363 E ns/openshift-marketplace pod/community-operators-7c4757bbd5-cjnrh node/ip-10-0-150-53.ec2.internal container=community-operators container exited with code 2 (Error): Nov 05 11:59:08.438 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-namespace" : couldn't propagate object cache: timed out waiting for the condition (5 times) Nov 05 11:59:08.637 E ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p node/ip-10-0-139-89.ec2.internal container=openshift-state-metrics container exited with code 2 (Error): Nov 05 11:59:08.641 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboards" : couldn't propagate object cache: timed out waiting for the condition (6 times) Nov 05 11:59:09.162 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-workloads-namespace" : couldn't propagate object cache: timed out waiting for the condition (7 times) Nov 05 11:59:09.164 E ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-449t9 node/ip-10-0-150-53.ec2.internal container=prometheus-adapter container exited with code 2 (Error): I1105 11:51:47.465904 1 adapter.go:93] successfully using in-cluster auth\nI1105 11:51:48.495255 1 secure_serving.go:116] Serving securely on [::]:6443\n Nov 05 11:59:09.170 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-token-k94vv" : couldn't propagate object cache: timed out waiting for the condition (8 times) Nov 05 11:59:09.241 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-workload" : couldn't propagate object cache: timed out waiting for the condition (9 times) Nov 05 11:59:09.244 W ns/openshift-monitoring pod/openshift-state-metrics-557985667-f5c9p node/ip-10-0-139-89.ec2.internal deleted Nov 05 11:59:09.255 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft 0/6 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 2 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector. (2 times) Nov 05 11:59:09.438 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-datasources" : couldn't propagate object cache: timed out waiting for the condition (2 times) Nov 05 11:59:09.646 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-node-cluster-rsrc-use" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:09.839 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp (combined from similar events): MountVolume.SetUp failed for volume "secret-grafana-tls" : couldn't propagate object cache: timed out waiting for the condition (10 times) Nov 05 11:59:09.963 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 node/ip-10-0-150-53.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 11:59:09.963 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 node/ip-10-0-150-53.ec2.internal container=certified-operators container stopped being ready Nov 05 11:59:10.037 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp MountVolume.SetUp failed for volume "grafana-dashboard-k8s-resources-namespace" : couldn't propagate object cache: timed out waiting for the condition Nov 05 11:59:10.364 E ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 node/ip-10-0-150-53.ec2.internal container=certified-operators container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 05 11:59:10.766 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr node/ip-10-0-150-53.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 11:59:10.766 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr node/ip-10-0-150-53.ec2.internal container=grafana container stopped being ready Nov 05 11:59:10.766 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr node/ip-10-0-150-53.ec2.internal container=grafana-proxy container stopped being ready Nov 05 11:59:10.972 W ns/openshift-monitoring pod/grafana-5d5d6cdf5-s8smr node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:11.278 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft 0/6 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 2 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector. (3 times) Nov 05 11:59:11.364 W ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-150-53.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 11:59:11.364 W ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-150-53.ec2.internal container=alertmanager-proxy container stopped being ready Nov 05 11:59:11.364 W ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-150-53.ec2.internal container=config-reloader container stopped being ready Nov 05 11:59:11.364 W ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-150-53.ec2.internal container=alertmanager container stopped being ready Nov 05 11:59:11.572 W ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:11.612 I ns/openshift-monitoring pod/alertmanager-main-1 node/ created Nov 05 11:59:11.616 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-1 in StatefulSet alertmanager-main successful Nov 05 11:59:11.626 I ns/openshift-monitoring pod/alertmanager-main-1 Successfully assigned openshift-monitoring/alertmanager-main-1 to ip-10-0-137-180.ec2.internal Nov 05 11:59:11.638 W ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 11:59:11.638 W ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v node/ip-10-0-139-89.ec2.internal container=kube-rbac-proxy container stopped being ready Nov 05 11:59:11.638 W ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v node/ip-10-0-139-89.ec2.internal container=telemeter-client container stopped being ready Nov 05 11:59:11.638 W ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v node/ip-10-0-139-89.ec2.internal container=reload container stopped being ready Nov 05 11:59:11.847 W ns/openshift-monitoring pod/telemeter-client-5c4c449647-vwh7v node/ip-10-0-139-89.ec2.internal deleted Nov 05 11:59:11.965 E ns/openshift-ingress pod/router-default-5796647cdf-x8md7 node/ip-10-0-150-53.ec2.internal container=router container exited with code 2 (Error): 1105 11:55:53.013434 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:56:07.666667 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:56:26.032612 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:56:31.028484 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nE1105 11:57:17.695030 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=21, ErrCode=NO_ERROR, debug=""\nE1105 11:57:17.695038 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=21, ErrCode=NO_ERROR, debug=""\nE1105 11:57:17.695030 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=21, ErrCode=NO_ERROR, debug=""\nW1105 11:57:18.734279 1 reflector.go:341] github.com/openshift/router/pkg/router/template/service_lookup.go:32: watch of *v1.Service ended with: too old resource version: 12231 (13688)\nI1105 11:57:26.257407 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:57:31.247581 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:57:36.250430 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 11:59:06.317934 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n Nov 05 11:59:12.172 W ns/openshift-ingress pod/router-default-5796647cdf-x8md7 node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:12.445 W ns/openshift-machine-config-operator pod/machine-config-daemon-sn4w6 node/ip-10-0-139-89.ec2.internal deleted Nov 05 11:59:12.453 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-w9xwh Nov 05 11:59:12.841 W ns/openshift-monitoring pod/alertmanager-main-0 node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 11:59:12.841 W ns/openshift-monitoring pod/alertmanager-main-0 node/ip-10-0-139-89.ec2.internal container=config-reloader container stopped being ready Nov 05 11:59:12.841 W ns/openshift-monitoring pod/alertmanager-main-0 node/ip-10-0-139-89.ec2.internal container=alertmanager container stopped being ready Nov 05 11:59:12.841 W ns/openshift-monitoring pod/alertmanager-main-0 node/ip-10-0-139-89.ec2.internal container=alertmanager-proxy container stopped being ready Nov 05 11:59:12.969 W ns/openshift-marketplace pod/community-operators-7c4757bbd5-cjnrh node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:13.044 W ns/openshift-monitoring pod/alertmanager-main-0 node/ip-10-0-139-89.ec2.internal deleted Nov 05 11:59:13.094 I ns/openshift-monitoring pod/alertmanager-main-0 node/ created Nov 05 11:59:13.104 I ns/openshift-monitoring statefulset/alertmanager-main create Pod alertmanager-main-0 in StatefulSet alertmanager-main successful Nov 05 11:59:13.109 I ns/openshift-monitoring pod/alertmanager-main-0 Successfully assigned openshift-monitoring/alertmanager-main-0 to ip-10-0-137-180.ec2.internal Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal container=prometheus-proxy container stopped being ready Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal container=kube-rbac-proxy container stopped being ready Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal container=rules-configmap-reloader container stopped being ready Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal container=prom-label-proxy container stopped being ready Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal container=prometheus container stopped being ready Nov 05 11:59:13.366 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal container=prometheus-config-reloader container stopped being ready Nov 05 11:59:13.572 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:13.633 I ns/openshift-monitoring pod/prometheus-k8s-0 node/ created Nov 05 11:59:13.642 I ns/openshift-monitoring statefulset/prometheus-k8s create Pod prometheus-k8s-0 in StatefulSet prometheus-k8s successful Nov 05 11:59:13.654 I ns/openshift-monitoring pod/prometheus-k8s-0 Successfully assigned openshift-monitoring/prometheus-k8s-0 to ip-10-0-137-180.ec2.internal Nov 05 11:59:13.657 W ns/openshift-marketplace pod/redhat-operators-f5fd5d65-9wm5g node/ip-10-0-139-89.ec2.internal deleted Nov 05 11:59:14.210 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0524536b3a5c1ee235a4c661ef66f524471641ccdcac556443a686cd2414b380" Nov 05 11:59:14.242 W ns/openshift-image-registry pod/node-ca-tjqkj node/ip-10-0-139-89.ec2.internal deleted Nov 05 11:59:14.251 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-qtjsp Nov 05 11:59:14.333 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0524536b3a5c1ee235a4c661ef66f524471641ccdcac556443a686cd2414b380" Nov 05 11:59:14.342 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0524536b3a5c1ee235a4c661ef66f524471641ccdcac556443a686cd2414b380" Nov 05 11:59:14.570 W ns/openshift-image-registry pod/node-ca-qtjsp node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:14.609 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" already present on machine Nov 05 11:59:14.837 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Created container kube-rbac-proxy-main Nov 05 11:59:14.916 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Started container kube-rbac-proxy-main Nov 05 11:59:14.922 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" already present on machine Nov 05 11:59:15.009 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:74e42ebecb258ca0419280345c130c3cefb69bc1442ac102577300fea3200917" already present on machine Nov 05 11:59:15.100 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-jpz58 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9c5629ef379647cdcfa19ba709c061b01759e029d8918cb0595ddcb3aa13ef44" already present on machine Nov 05 11:59:15.294 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Created container kube-rbac-proxy-self Nov 05 11:59:15.404 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Created container telemeter-client Nov 05 11:59:15.409 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Started container kube-rbac-proxy-self Nov 05 11:59:15.417 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c066a37d778ec86d4383ab0b1de8f6fdc05665e5a355dca6c18dd3b2aad18c54" Nov 05 11:59:15.423 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-jpz58 Created container prometheus-adapter Nov 05 11:59:15.507 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Started container telemeter-client Nov 05 11:59:15.534 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93cf21818601f537507b0169a0badfd62b1d0d7ff7a1ac988921ee07971b3411" already present on machine Nov 05 11:59:15.610 I ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-jpz58 Started container prometheus-adapter Nov 05 11:59:16.049 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Created container reload Nov 05 11:59:16.270 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Started container reload Nov 05 11:59:16.279 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft 0/6 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 2 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector. (4 times) Nov 05 11:59:16.318 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" already present on machine Nov 05 11:59:17.171 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-c85g6 node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:17.379 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Created container kube-rbac-proxy Nov 05 11:59:17.503 I ns/openshift-monitoring pod/telemeter-client-5c4c449647-qg5ss Started container kube-rbac-proxy Nov 05 11:59:17.975 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0a768b328417ab619d5508f0bcbcb7b42061376d5e9525488bb1a2ddd662825" Nov 05 11:59:19.719 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3ae1d04427f80ba6d9a2dd421d2a245393e279954444087880835dbb05867ca4" already present on machine Nov 05 11:59:19.725 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c066a37d778ec86d4383ab0b1de8f6fdc05665e5a355dca6c18dd3b2aad18c54" Nov 05 11:59:20.032 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager Nov 05 11:59:20.038 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Created container openshift-state-metrics Nov 05 11:59:20.112 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager Nov 05 11:59:20.119 I ns/openshift-monitoring pod/openshift-state-metrics-557985667-bvq65 Started container openshift-state-metrics Nov 05 11:59:22.417 I ns/openshift-monitoring pod/alertmanager-main-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3ae1d04427f80ba6d9a2dd421d2a245393e279954444087880835dbb05867ca4" already present on machine Nov 05 11:59:22.426 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0db637d736c2f5246ff824f0b0d4e97616f81882eed178befad3c568e37aef89" already present on machine Nov 05 11:59:22.435 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93cf21818601f537507b0169a0badfd62b1d0d7ff7a1ac988921ee07971b3411" already present on machine Nov 05 11:59:22.441 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0524536b3a5c1ee235a4c661ef66f524471641ccdcac556443a686cd2414b380" Nov 05 11:59:22.447 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0524536b3a5c1ee235a4c661ef66f524471641ccdcac556443a686cd2414b380" Nov 05 11:59:22.471 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0524536b3a5c1ee235a4c661ef66f524471641ccdcac556443a686cd2414b380" Nov 05 11:59:22.709 I ns/openshift-monitoring pod/alertmanager-main-0 Created container alertmanager Nov 05 11:59:22.760 I ns/openshift-monitoring pod/alertmanager-main-0 Started container alertmanager Nov 05 11:59:22.766 I ns/openshift-monitoring pod/alertmanager-main-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93cf21818601f537507b0169a0badfd62b1d0d7ff7a1ac988921ee07971b3411" already present on machine Nov 05 11:59:23.027 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus Nov 05 11:59:23.124 I ns/openshift-monitoring pod/alertmanager-main-1 Created container config-reloader Nov 05 11:59:23.140 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Created container certified-operators Nov 05 11:59:23.175 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus Nov 05 11:59:23.222 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee1e29644837e48b725b90fa360458b7413d64bde97fc21b7558836617bd0cc4" already present on machine Nov 05 11:59:23.418 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Created container community-operators Nov 05 11:59:23.618 I ns/openshift-monitoring pod/alertmanager-main-1 Started container config-reloader Nov 05 11:59:23.818 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 Created container redhat-operators Nov 05 11:59:24.026 I ns/openshift-monitoring pod/alertmanager-main-1 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:737b9f043fb85469441af421d0549146648fbfd04b0ef990dbcc7531c23bf1c9" already present on machine Nov 05 11:59:24.218 I ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Started container certified-operators Nov 05 11:59:24.418 I ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Started container community-operators Nov 05 11:59:24.618 I ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 Started container redhat-operators Nov 05 11:59:24.818 I ns/openshift-monitoring pod/alertmanager-main-0 Created container config-reloader Nov 05 11:59:25.018 I ns/openshift-monitoring pod/alertmanager-main-0 Started container config-reloader Nov 05 11:59:25.219 I ns/openshift-monitoring pod/alertmanager-main-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:737b9f043fb85469441af421d0549146648fbfd04b0ef990dbcc7531c23bf1c9" already present on machine Nov 05 11:59:25.284 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft 0/6 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 2 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector. (5 times) Nov 05 11:59:25.418 I ns/openshift-monitoring pod/alertmanager-main-1 Created container alertmanager-proxy Nov 05 11:59:25.618 I ns/openshift-monitoring pod/alertmanager-main-1 Started container alertmanager-proxy Nov 05 11:59:25.818 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus-config-reloader Nov 05 11:59:26.018 I ns/openshift-monitoring pod/alertmanager-main-0 Created container alertmanager-proxy Nov 05 11:59:26.218 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus-config-reloader Nov 05 11:59:26.418 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:93cf21818601f537507b0169a0badfd62b1d0d7ff7a1ac988921ee07971b3411" already present on machine Nov 05 11:59:26.499 W ns/openshift-machine-config-operator pod/machine-config-daemon-w9xwh node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:26.531 W ns/openshift-monitoring pod/prometheus-adapter-6d9d776844-449t9 node/ip-10-0-150-53.ec2.internal deleted Nov 05 11:59:26.617 I ns/openshift-monitoring pod/alertmanager-main-0 Started container alertmanager-proxy Nov 05 11:59:26.818 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container rules-configmap-reloader Nov 05 11:59:27.018 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container rules-configmap-reloader Nov 05 11:59:27.218 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:737b9f043fb85469441af421d0549146648fbfd04b0ef990dbcc7531c23bf1c9" already present on machine Nov 05 11:59:27.418 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0a768b328417ab619d5508f0bcbcb7b42061376d5e9525488bb1a2ddd662825" Nov 05 11:59:27.444 E ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-137-180.ec2.internal container=prometheus container exited with code 1 (Error): Nov 05 11:59:27.617 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus-proxy Nov 05 11:59:27.821 I ns/openshift-monitoring pod/grafana-5d5d6cdf5-pdkvp Created container grafana Nov 05 11:59:28.018 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus-proxy Nov 05 11:59:28.218 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" already present on machine Nov 05 11:59:28.418 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container kube-rbac-proxy Nov 05 11:59:28.451 W ns/openshift-monitoring pod/prometheus-k8s-0 node/ip-10-0-137-180.ec2.internal container=prometheus container restarted Nov 05 11:59:28.618 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container kube-rbac-proxy Nov 05 11:59:28.820 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f57c717539e3c5015a222e9fce80cb310c822fe73bc1417b9153bac133b66c05" already present on machine Nov 05 11:59:29.018 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prom-label-proxy Nov 05 11:59:29.218 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prom-label-proxy Nov 05 11:59:29.420 I ns/openshift-monitoring pod/prometheus-k8s-0 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0db637d736c2f5246ff824f0b0d4e97616f81882eed178befad3c568e37aef89" already present on machine (2 times) Nov 05 11:59:29.621 I ns/openshift-monitoring pod/prometheus-k8s-0 Created container prometheus (2 times) Nov 05 11:59:29.819 I ns/openshift-monitoring pod/prometheus-k8s-0 Started container prometheus (2 times) Nov 05 11:59:30.763 W ns/openshift-marketplace pod/redhat-operators-f5fd5d65-cz8m7 Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 05 11:59:31.266 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Liveness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 05 11:59:36.283 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft 0/6 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 2 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector. (6 times) Nov 05 11:59:38.067 W ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 05 11:59:38.221 W ns/openshift-marketplace pod/certified-operators-57cdf6f756-vd5hq Readiness probe failed: timeout: failed to connect service "localhost:50051" within 1s\n Nov 05 11:59:38.807 W ns/openshift-marketplace pod/community-operators-7c4757bbd5-2nvqn Liveness probe errored: rpc error: code = Unknown desc = command error: command timed out, stdout: , stderr: , exit code -1 Nov 05 12:00:11.368 I ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl node/ created Nov 05 12:00:11.379 I ns/openshift-ingress pod/router-default-5796647cdf-sr9ft Successfully assigned openshift-ingress/router-default-5796647cdf-sr9ft to ip-10-0-150-53.ec2.internal Nov 05 12:00:11.384 I ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl Successfully assigned openshift-machine-config-operator/machine-config-daemon-jkfpl to ip-10-0-150-53.ec2.internal Nov 05 12:00:11.391 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-jkfpl Nov 05 12:00:11.400 I ns/openshift-image-registry pod/node-ca-gtr4s node/ created Nov 05 12:00:11.408 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-gtr4s Nov 05 12:00:11.418 I ns/openshift-image-registry pod/node-ca-gtr4s Successfully assigned openshift-image-registry/node-ca-gtr4s to ip-10-0-150-53.ec2.internal Nov 05 12:00:11.428 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr node/ created Nov 05 12:00:11.434 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-9w9nr Nov 05 12:00:11.444 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr Successfully assigned openshift-machine-config-operator/machine-config-daemon-9w9nr to ip-10-0-139-89.ec2.internal Nov 05 12:00:11.453 I ns/openshift-image-registry pod/node-ca-txpkr node/ created Nov 05 12:00:11.461 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-txpkr Nov 05 12:00:11.466 I ns/openshift-image-registry pod/node-ca-txpkr Successfully assigned openshift-image-registry/node-ca-txpkr to ip-10-0-139-89.ec2.internal Nov 05 12:00:12.005 I ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:00:12.123 I ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl Created container machine-config-daemon Nov 05 12:00:12.144 I ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl Started container machine-config-daemon Nov 05 12:00:12.179 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:00:12.312 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr Created container machine-config-daemon Nov 05 12:00:12.337 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr Started container machine-config-daemon Nov 05 12:00:12.612 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft MountVolume.SetUp failed for volume "default-certificate" : couldn't propagate object cache: timed out waiting for the condition Nov 05 12:00:12.616 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft MountVolume.SetUp failed for volume "metrics-certs" : couldn't propagate object cache: timed out waiting for the condition Nov 05 12:00:18.272 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft node/ip-10-0-150-53.ec2.internal pod has been pending longer than a minute Nov 05 12:00:19.685 I ns/openshift-image-registry pod/node-ca-gtr4s Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:00:19.746 I ns/openshift-image-registry pod/node-ca-txpkr Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:00:19.828 I ns/openshift-image-registry pod/node-ca-gtr4s Created container node-ca Nov 05 12:00:19.855 I ns/openshift-image-registry pod/node-ca-gtr4s Started container node-ca Nov 05 12:00:19.880 I ns/openshift-image-registry pod/node-ca-txpkr Created container node-ca Nov 05 12:00:19.903 I ns/openshift-image-registry pod/node-ca-txpkr Started container node-ca Nov 05 12:00:20.965 I ns/openshift-ingress pod/router-default-5796647cdf-sr9ft Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f0b7726741f00cc06e3c594e4f5d2db08da4c5cd0cd7bf14a467668ec5f07c0" already present on machine Nov 05 12:00:21.118 I ns/openshift-ingress pod/router-default-5796647cdf-sr9ft Created container router Nov 05 12:00:21.140 I ns/openshift-ingress pod/router-default-5796647cdf-sr9ft Started container router Nov 05 12:05:17.243 W persistentvolume/pvc-714d70e9-ffc4-11e9-9c9a-0a0d6c8cf44b Error deleting EBS volume "vol-0fc006376fdde0200" since volume is currently attached to "i-037fe04051141c22d" Nov 05 12:06:48.679 W ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 12:06:48.684 W ns/openshift-image-registry pod/node-ca-txpkr node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 12:06:48.692 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr Stopping container machine-config-daemon Nov 05 12:06:48.692 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-9w9nr Nov 05 12:06:48.696 I ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-9w9nr Nov 05 12:06:48.700 I ns/openshift-image-registry pod/node-ca-txpkr Stopping container node-ca Nov 05 12:06:48.702 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-txpkr Nov 05 12:06:48.704 I ns/openshift-image-registry pod/node-ca-txpkr Marking for deletion Pod openshift-image-registry/node-ca-txpkr Nov 05 12:06:50.132 W ns/openshift-image-registry pod/node-ca-txpkr node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:06:50.132 W ns/openshift-image-registry pod/node-ca-txpkr node/ip-10-0-139-89.ec2.internal container=node-ca container stopped being ready Nov 05 12:06:57.005 W ns/openshift-image-registry pod/node-ca-txpkr node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:06:57.028 W ns/openshift-machine-config-operator pod/machine-config-daemon-9w9nr node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:07:21.424 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (5 times) Nov 05 12:07:23.054 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (5 times) Nov 05 12:07:24.392 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (5 times) Nov 05 12:07:24.586 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (9 times) Nov 05 12:07:24.758 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (8 times) Nov 05 12:07:24.929 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (9 times) Nov 05 12:08:07.057 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh node/ created Nov 05 12:08:07.062 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-rjfmh Nov 05 12:08:07.075 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh Successfully assigned openshift-machine-config-operator/machine-config-daemon-rjfmh to ip-10-0-139-89.ec2.internal Nov 05 12:08:07.078 I ns/openshift-image-registry pod/node-ca-2l6jt node/ created Nov 05 12:08:07.084 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-2l6jt Nov 05 12:08:07.096 I ns/openshift-image-registry pod/node-ca-2l6jt Successfully assigned openshift-image-registry/node-ca-2l6jt to ip-10-0-139-89.ec2.internal Nov 05 12:08:07.815 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:08:07.978 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh Created container machine-config-daemon Nov 05 12:08:08.004 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh Started container machine-config-daemon Nov 05 12:08:15.368 I ns/openshift-image-registry pod/node-ca-2l6jt Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:08:15.499 I ns/openshift-image-registry pod/node-ca-2l6jt Created container node-ca Nov 05 12:08:15.532 I ns/openshift-image-registry pod/node-ca-2l6jt Started container node-ca Nov 05 12:10:44.447 W persistentvolume/pvc-3215f7b9-ffc5-11e9-9c9a-0a0d6c8cf44b Error deleting EBS volume "vol-0765f1f7f0e0abc3d" since volume is currently attached to "i-037fe04051141c22d" Nov 05 12:14:35.756 W node/node1 deleted Nov 05 12:16:30.273 I ns/kube-system pod/pod0-system-node-critical node/ created Nov 05 12:16:30.292 I ns/kube-system pod/pod1-system-cluster-critical node/ created Nov 05 12:16:30.292 I ns/kube-system pod/pod0-system-node-critical Successfully assigned kube-system/pod0-system-node-critical to ip-10-0-139-89.ec2.internal Nov 05 12:16:30.309 W ns/kube-system pod/pod1-system-cluster-critical node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 12:16:30.311 I ns/kube-system pod/pod1-system-cluster-critical Successfully assigned kube-system/pod1-system-cluster-critical to ip-10-0-150-53.ec2.internal Nov 05 12:16:30.314 W ns/kube-system pod/pod1-system-cluster-critical node/ip-10-0-150-53.ec2.internal deleted Nov 05 12:16:30.332 W ns/kube-system pod/pod0-system-node-critical node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 12:16:30.336 W ns/kube-system pod/pod0-system-node-critical node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:16:30.416 W ns/kube-system pod/pod0-system-node-critical MountVolume.SetUp failed for volume "default-token-2xnkf" : object "kube-system"/"default-token-2xnkf" not registered Nov 05 12:17:21.281 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (6 times) Nov 05 12:17:22.470 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (6 times) Nov 05 12:17:23.687 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (6 times) Nov 05 12:17:23.853 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (10 times) Nov 05 12:17:24.016 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (9 times) Nov 05 12:17:24.277 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (10 times) Nov 05 12:18:33.303 W ns/kube-system pod/pod0-system-node-critical Unable to mount volumes for pod "pod0-system-node-critical_kube-system(192e954c-ffc6-11e9-becc-129f61888dee)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"pod0-system-node-critical". list of unmounted volumes=[default-token-2xnkf]. list of unattached volumes=[default-token-2xnkf] Nov 05 12:18:33.323 W ns/kube-system pod/pod1-system-cluster-critical Unable to mount volumes for pod "pod1-system-cluster-critical_kube-system(1931a7b7-ffc6-11e9-becc-129f61888dee)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"pod1-system-cluster-critical". list of unmounted volumes=[default-token-2xnkf]. list of unattached volumes=[default-token-2xnkf] Nov 05 12:27:21.384 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (7 times) Nov 05 12:27:21.604 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (11 times) Nov 05 12:27:21.767 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (10 times) Nov 05 12:27:21.917 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (11 times) Nov 05 12:27:23.167 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (7 times) Nov 05 12:27:24.436 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (7 times) Nov 05 12:37:21.244 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (8 times) Nov 05 12:37:22.509 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (8 times) Nov 05 12:37:23.803 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (8 times) Nov 05 12:37:23.975 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (12 times) Nov 05 12:37:24.159 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (11 times) Nov 05 12:37:24.323 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (12 times) Nov 05 12:42:19.702 W ns/openshift-image-registry pod/node-ca-2l6jt node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 12:42:19.703 W ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 12:42:19.707 I ns/openshift-image-registry pod/node-ca-2l6jt Stopping container node-ca Nov 05 12:42:19.710 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-2l6jt Nov 05 12:42:19.714 I ns/openshift-image-registry pod/node-ca-2l6jt Marking for deletion Pod openshift-image-registry/node-ca-2l6jt Nov 05 12:42:19.718 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh Stopping container machine-config-daemon Nov 05 12:42:19.719 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-rjfmh Nov 05 12:42:19.720 I ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-rjfmh Nov 05 12:42:20.920 W ns/openshift-image-registry pod/node-ca-2l6jt node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:42:20.920 W ns/openshift-image-registry pod/node-ca-2l6jt node/ip-10-0-139-89.ec2.internal container=node-ca container stopped being ready Nov 05 12:42:20.931 W ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:42:20.931 W ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh node/ip-10-0-139-89.ec2.internal container=machine-config-daemon container stopped being ready Nov 05 12:42:27.004 W ns/openshift-image-registry pod/node-ca-2l6jt node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:42:27.028 W ns/openshift-machine-config-operator pod/machine-config-daemon-rjfmh node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:42:27.036 I ns/openshift-image-registry pod/node-ca-2pr9z node/ created Nov 05 12:42:27.043 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n node/ created Nov 05 12:42:27.044 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-2pr9z Nov 05 12:42:27.050 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-dsb5n Nov 05 12:42:27.051 I ns/openshift-image-registry pod/node-ca-2pr9z Successfully assigned openshift-image-registry/node-ca-2pr9z to ip-10-0-139-89.ec2.internal Nov 05 12:42:27.057 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n Successfully assigned openshift-machine-config-operator/machine-config-daemon-dsb5n to ip-10-0-139-89.ec2.internal Nov 05 12:42:28.279 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:42:28.396 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n Created container machine-config-daemon Nov 05 12:42:28.417 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n Started container machine-config-daemon Nov 05 12:42:34.532 I ns/openshift-image-registry pod/node-ca-2pr9z Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:42:34.669 I ns/openshift-image-registry pod/node-ca-2pr9z Created container node-ca Nov 05 12:42:34.690 I ns/openshift-image-registry pod/node-ca-2pr9z Started container node-ca Nov 05 12:43:36.895 W ns/openshift-image-registry pod/node-ca-2pr9z node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 12:43:36.900 W ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 12:43:36.902 I ns/openshift-image-registry pod/node-ca-2pr9z Stopping container node-ca Nov 05 12:43:36.902 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-2pr9z Nov 05 12:43:36.909 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n Stopping container machine-config-daemon Nov 05 12:43:36.910 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-dsb5n Nov 05 12:43:36.912 I ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-dsb5n Nov 05 12:43:36.916 I ns/openshift-image-registry pod/node-ca-2pr9z Marking for deletion Pod openshift-image-registry/node-ca-2pr9z Nov 05 12:43:38.070 W ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:43:38.070 W ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n node/ip-10-0-139-89.ec2.internal container=machine-config-daemon container stopped being ready Nov 05 12:43:38.080 W ns/openshift-image-registry pod/node-ca-2pr9z node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:43:38.080 W ns/openshift-image-registry pod/node-ca-2pr9z node/ip-10-0-139-89.ec2.internal container=node-ca container stopped being ready Nov 05 12:43:38.090 E ns/openshift-image-registry pod/node-ca-2pr9z node/ip-10-0-139-89.ec2.internal container=node-ca container exited with code 137 (ContainerStatusUnknown): The container could not be located when the pod was terminated Nov 05 12:43:39.081 W ns/openshift-machine-config-operator pod/machine-config-daemon-dsb5n node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:43:47.007 W ns/openshift-image-registry pod/node-ca-2pr9z node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:44:08.000 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k node/ created Nov 05 12:44:08.007 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-6qp9k Nov 05 12:44:08.020 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k Successfully assigned openshift-machine-config-operator/machine-config-daemon-6qp9k to ip-10-0-139-89.ec2.internal Nov 05 12:44:08.023 I ns/openshift-image-registry pod/node-ca-dwqmd node/ created Nov 05 12:44:08.031 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-dwqmd Nov 05 12:44:08.054 I ns/openshift-image-registry pod/node-ca-dwqmd Successfully assigned openshift-image-registry/node-ca-dwqmd to ip-10-0-139-89.ec2.internal Nov 05 12:44:08.674 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:44:08.801 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k Created container machine-config-daemon Nov 05 12:44:08.826 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k Started container machine-config-daemon Nov 05 12:44:15.813 I ns/openshift-image-registry pod/node-ca-dwqmd Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:44:15.934 I ns/openshift-image-registry pod/node-ca-dwqmd Created container node-ca Nov 05 12:44:15.961 I ns/openshift-image-registry pod/node-ca-dwqmd Started container node-ca Nov 05 12:45:42.102 W ns/openshift-image-registry pod/node-ca-dwqmd node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 12:45:42.105 W ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 12:45:42.109 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-dwqmd Nov 05 12:45:42.109 I ns/openshift-image-registry pod/node-ca-dwqmd Stopping container node-ca Nov 05 12:45:42.115 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k Stopping container machine-config-daemon Nov 05 12:45:42.115 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-6qp9k Nov 05 12:45:42.117 I ns/openshift-image-registry pod/node-ca-dwqmd Marking for deletion Pod openshift-image-registry/node-ca-dwqmd Nov 05 12:45:42.121 I ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-6qp9k Nov 05 12:45:43.305 W ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:45:43.305 W ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k node/ip-10-0-139-89.ec2.internal container=machine-config-daemon container stopped being ready Nov 05 12:45:43.324 W ns/openshift-image-registry pod/node-ca-dwqmd node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:45:43.324 W ns/openshift-image-registry pod/node-ca-dwqmd node/ip-10-0-139-89.ec2.internal container=node-ca container stopped being ready Nov 05 12:45:47.006 W ns/openshift-image-registry pod/node-ca-dwqmd node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:45:47.030 W ns/openshift-machine-config-operator pod/machine-config-daemon-6qp9k node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:46:47.157 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp node/ created Nov 05 12:46:47.167 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-q7rjp Nov 05 12:46:47.169 I ns/openshift-image-registry pod/node-ca-mdkmg node/ created Nov 05 12:46:47.179 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-mdkmg Nov 05 12:46:47.180 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp Successfully assigned openshift-machine-config-operator/machine-config-daemon-q7rjp to ip-10-0-139-89.ec2.internal Nov 05 12:46:47.187 I ns/openshift-image-registry pod/node-ca-mdkmg Successfully assigned openshift-image-registry/node-ca-mdkmg to ip-10-0-139-89.ec2.internal Nov 05 12:46:47.841 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:46:47.987 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp Created container machine-config-daemon Nov 05 12:46:48.011 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp Started container machine-config-daemon Nov 05 12:46:55.000 I ns/openshift-image-registry pod/node-ca-mdkmg Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:46:55.126 I ns/openshift-image-registry pod/node-ca-mdkmg Created container node-ca Nov 05 12:46:55.152 I ns/openshift-image-registry pod/node-ca-mdkmg Started container node-ca Nov 05 12:47:21.089 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (9 times) Nov 05 12:47:22.209 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (9 times) Nov 05 12:47:22.415 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (13 times) Nov 05 12:47:22.654 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (12 times) Nov 05 12:47:22.851 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (13 times) Nov 05 12:47:24.179 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (9 times) Nov 05 12:48:14.917 W ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 12:48:14.921 W ns/openshift-image-registry pod/node-ca-mdkmg node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 12:48:14.925 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp Stopping container machine-config-daemon Nov 05 12:48:14.926 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-q7rjp Nov 05 12:48:14.929 I ns/openshift-image-registry pod/node-ca-mdkmg Marking for deletion Pod openshift-image-registry/node-ca-mdkmg Nov 05 12:48:14.932 I ns/openshift-image-registry pod/node-ca-mdkmg Stopping container node-ca Nov 05 12:48:14.935 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-mdkmg Nov 05 12:48:14.941 I ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-q7rjp Nov 05 12:48:27.011 W ns/openshift-machine-config-operator pod/machine-config-daemon-q7rjp node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:48:27.033 W ns/openshift-image-registry pod/node-ca-mdkmg node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:48:27.103 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk node/ created Nov 05 12:48:27.110 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-qsnfk Nov 05 12:48:27.127 I ns/openshift-image-registry pod/node-ca-88nd8 node/ created Nov 05 12:48:27.128 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk Successfully assigned openshift-machine-config-operator/machine-config-daemon-qsnfk to ip-10-0-139-89.ec2.internal Nov 05 12:48:27.136 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-88nd8 Nov 05 12:48:27.140 E clusteroperator/machine-config changed Degraded to True: MachineConfigDaemonFailed: Failed to resync 4.2.0-0.nightly-2019-11-05-111920 because: Operation cannot be fulfilled on daemonsets.apps "machine-config-daemon": the object has been modified; please apply your changes to the latest version and try again Nov 05 12:48:27.144 I ns/openshift-image-registry pod/node-ca-88nd8 Successfully assigned openshift-image-registry/node-ca-88nd8 to ip-10-0-139-89.ec2.internal Nov 05 12:48:27.161 W clusteroperator/machine-config changed Available to False: Cluster not available for 4.2.0-0.nightly-2019-11-05-111920 Nov 05 12:48:27.892 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:48:28.016 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk Created container machine-config-daemon Nov 05 12:48:28.043 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk Started container machine-config-daemon Nov 05 12:48:34.171 W clusteroperator/machine-config changed Degraded to False Nov 05 12:48:34.183 W clusteroperator/machine-config changed Available to True: Cluster has deployed 4.2.0-0.nightly-2019-11-05-111920 Nov 05 12:48:36.064 I ns/openshift-image-registry pod/node-ca-88nd8 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:48:36.189 I ns/openshift-image-registry pod/node-ca-88nd8 Created container node-ca Nov 05 12:48:36.225 I ns/openshift-image-registry pod/node-ca-88nd8 Started container node-ca Nov 05 12:51:58.507 W ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk node/ip-10-0-139-89.ec2.internal graceful deletion within 600s Nov 05 12:51:58.508 W ns/openshift-image-registry pod/node-ca-88nd8 node/ip-10-0-139-89.ec2.internal graceful deletion within 30s Nov 05 12:51:58.518 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk Stopping container machine-config-daemon Nov 05 12:51:58.518 I ns/openshift-image-registry daemonset/node-ca Deleted pod: node-ca-88nd8 Nov 05 12:51:58.519 I ns/openshift-image-registry pod/node-ca-88nd8 Marking for deletion Pod openshift-image-registry/node-ca-88nd8 Nov 05 12:51:58.523 I ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk Marking for deletion Pod openshift-machine-config-operator/machine-config-daemon-qsnfk Nov 05 12:51:58.525 I ns/openshift-image-registry pod/node-ca-88nd8 Stopping container node-ca Nov 05 12:51:58.525 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Deleted pod: machine-config-daemon-qsnfk Nov 05 12:51:59.945 W ns/openshift-image-registry pod/node-ca-88nd8 node/ip-10-0-139-89.ec2.internal invariant violation (bug): pod should not transition Running->Pending even when terminated Nov 05 12:51:59.945 W ns/openshift-image-registry pod/node-ca-88nd8 node/ip-10-0-139-89.ec2.internal container=node-ca container stopped being ready Nov 05 12:52:03.272 W ns/openshift-image-registry pod/node-ca-88nd8 node/ip-10-0-139-89.ec2.internal pod has been pending longer than a minute Nov 05 12:52:07.005 W ns/openshift-image-registry pod/node-ca-88nd8 node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:52:07.028 W ns/openshift-machine-config-operator pod/machine-config-daemon-qsnfk node/ip-10-0-139-89.ec2.internal deleted Nov 05 12:52:07.038 I ns/openshift-image-registry pod/node-ca-pgwsf node/ created Nov 05 12:52:07.044 I ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 node/ created Nov 05 12:52:07.045 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-pgwsf Nov 05 12:52:07.051 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-97dv6 Nov 05 12:52:07.053 I ns/openshift-image-registry pod/node-ca-pgwsf Successfully assigned openshift-image-registry/node-ca-pgwsf to ip-10-0-139-89.ec2.internal Nov 05 12:52:07.058 I ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 Successfully assigned openshift-machine-config-operator/machine-config-daemon-97dv6 to ip-10-0-139-89.ec2.internal Nov 05 12:52:08.576 I ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" already present on machine Nov 05 12:52:08.690 I ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 Created container machine-config-daemon Nov 05 12:52:08.716 I ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 Started container machine-config-daemon Nov 05 12:52:14.903 I ns/openshift-image-registry pod/node-ca-pgwsf Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" already present on machine Nov 05 12:52:15.035 I ns/openshift-image-registry pod/node-ca-pgwsf Created container node-ca Nov 05 12:52:15.060 I ns/openshift-image-registry pod/node-ca-pgwsf Started container node-ca Nov 05 12:56:32.900 W persistentvolume/pvc-75fcfe55-ffcb-11e9-becc-129f61888dee Error deleting EBS volume "vol-0b1d9a84fdccf2d97" since volume is currently attached to "i-037fe04051141c22d" Nov 05 12:57:21.365 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (10 times) Nov 05 12:57:22.832 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (10 times) Nov 05 12:57:23.004 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (14 times) Nov 05 12:57:23.218 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (13 times) Nov 05 12:57:23.373 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (14 times) Nov 05 12:57:24.623 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (10 times) Nov 05 13:06:24.012 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Created Machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Nov 05 13:06:24.187 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Nov 05 13:06:26.063 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Created Machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Nov 05 13:06:26.316 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Nov 05 13:06:27.496 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 (2 times) Nov 05 13:06:44.197 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v (2 times) Nov 05 13:06:44.322 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v (3 times) Nov 05 13:06:45.752 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v (4 times) Nov 05 13:06:47.720 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 (3 times) Nov 05 13:06:47.882 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 (4 times) Nov 05 13:07:21.229 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-0 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-0 (11 times) Nov 05 13:07:22.582 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-1 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-1 (11 times) Nov 05 13:07:22.756 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj (15 times) Nov 05 13:07:22.983 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-94fz8 (14 times) Nov 05 13:07:23.184 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg (15 times) Nov 05 13:07:23.347 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v (5 times) Nov 05 13:07:23.548 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 (5 times) Nov 05 13:07:24.799 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-master-2 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-master-2 (11 times) Nov 05 13:09:16.802 I node/ip-10-0-154-237.ec2.internal Node ip-10-0-154-237.ec2.internal status is now: NodeHasSufficientMemory (8 times) Nov 05 13:09:16.809 I node/ip-10-0-154-237.ec2.internal Node ip-10-0-154-237.ec2.internal status is now: NodeHasSufficientPID (8 times) Nov 05 13:09:16.828 I ns/openshift-sdn pod/ovs-gk2d8 node/ created Nov 05 13:09:16.837 I ns/openshift-sdn daemonset/ovs Created pod: ovs-gk2d8 Nov 05 13:09:16.841 I ns/openshift-sdn pod/ovs-gk2d8 Successfully assigned openshift-sdn/ovs-gk2d8 to ip-10-0-154-237.ec2.internal Nov 05 13:09:16.857 W clusteroperator/network changed Progressing to True: Deploying: DaemonSet "openshift-sdn/ovs" is not available (awaiting 1 nodes) Nov 05 13:09:16.865 I ns/openshift-dns pod/dns-default-mlstv node/ created Nov 05 13:09:16.870 I ns/openshift-monitoring pod/node-exporter-6snqf node/ created Nov 05 13:09:16.879 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-mlstv Nov 05 13:09:16.885 I ns/openshift-dns pod/dns-default-mlstv Successfully assigned openshift-dns/dns-default-mlstv to ip-10-0-154-237.ec2.internal Nov 05 13:09:16.891 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-6snqf Nov 05 13:09:16.893 I ns/openshift-monitoring pod/node-exporter-6snqf Successfully assigned openshift-monitoring/node-exporter-6snqf to ip-10-0-154-237.ec2.internal Nov 05 13:09:16.894 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Nov 05 13:09:16.898 I ns/openshift-sdn pod/sdn-scq8s node/ created Nov 05 13:09:16.906 I ns/openshift-sdn daemonset/sdn Created pod: sdn-scq8s Nov 05 13:09:16.912 I ns/openshift-sdn pod/sdn-scq8s Successfully assigned openshift-sdn/sdn-scq8s to ip-10-0-154-237.ec2.internal Nov 05 13:09:16.930 I ns/openshift-multus pod/multus-cqsgk node/ created Nov 05 13:09:16.935 I ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q node/ created Nov 05 13:09:16.935 I ns/openshift-multus daemonset/multus Created pod: multus-cqsgk Nov 05 13:09:16.943 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-rlz5q Nov 05 13:09:16.951 I ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q Successfully assigned openshift-cluster-node-tuning-operator/tuned-rlz5q to ip-10-0-154-237.ec2.internal Nov 05 13:09:16.958 I ns/openshift-multus pod/multus-cqsgk Successfully assigned openshift-multus/multus-cqsgk to ip-10-0-154-237.ec2.internal Nov 05 13:09:17.011 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 (6 times) Nov 05 13:09:17.011 W clusteroperator/dns changed Progressing to True: Reconciling: Not all DNS DaemonSets available. Nov 05 13:09:18.152 W ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q MountVolume.SetUp failed for volume "etc-tuned-recommend" : couldn't propagate object cache: timed out waiting for the condition Nov 05 13:09:18.158 W ns/openshift-multus pod/multus-cqsgk MountVolume.SetUp failed for volume "multus-token-cwp7z" : couldn't propagate object cache: timed out waiting for the condition Nov 05 13:09:18.272 - 15s W node/ip-10-0-154-237.ec2.internal node is not ready Nov 05 13:09:19.071 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (2 times) Nov 05 13:09:19.884 I node/ip-10-0-132-180.ec2.internal Node ip-10-0-132-180.ec2.internal status is now: NodeHasSufficientMemory (8 times) Nov 05 13:09:19.895 I node/ip-10-0-132-180.ec2.internal Node ip-10-0-132-180.ec2.internal status is now: NodeHasNoDiskPressure (8 times) Nov 05 13:09:19.898 I ns/openshift-multus pod/multus-r5rhl node/ created Nov 05 13:09:19.907 I ns/openshift-multus daemonset/multus Created pod: multus-r5rhl Nov 05 13:09:19.919 I ns/openshift-multus pod/multus-r5rhl Successfully assigned openshift-multus/multus-r5rhl to ip-10-0-132-180.ec2.internal Nov 05 13:09:19.959 I ns/openshift-cluster-node-tuning-operator pod/tuned-4dpxv node/ created Nov 05 13:09:19.967 I ns/openshift-cluster-node-tuning-operator daemonset/tuned Created pod: tuned-4dpxv Nov 05 13:09:19.974 I ns/openshift-cluster-node-tuning-operator pod/tuned-4dpxv Successfully assigned openshift-cluster-node-tuning-operator/tuned-4dpxv to ip-10-0-132-180.ec2.internal Nov 05 13:09:19.977 I ns/openshift-monitoring pod/node-exporter-4l262 node/ created Nov 05 13:09:19.983 I ns/openshift-monitoring daemonset/node-exporter Created pod: node-exporter-4l262 Nov 05 13:09:20.012 I ns/openshift-monitoring pod/node-exporter-4l262 Successfully assigned openshift-monitoring/node-exporter-4l262 to ip-10-0-132-180.ec2.internal Nov 05 13:09:20.018 I ns/openshift-dns pod/dns-default-m847c node/ created Nov 05 13:09:20.023 I ns/openshift-dns daemonset/dns-default Created pod: dns-default-m847c Nov 05 13:09:20.025 I ns/openshift-sdn pod/ovs-6ml2j node/ created Nov 05 13:09:20.035 I ns/openshift-sdn daemonset/ovs Created pod: ovs-6ml2j Nov 05 13:09:20.048 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network Nov 05 13:09:20.048 I ns/openshift-sdn pod/ovs-6ml2j Successfully assigned openshift-sdn/ovs-6ml2j to ip-10-0-132-180.ec2.internal Nov 05 13:09:20.054 I ns/openshift-sdn pod/sdn-7gvsv node/ created Nov 05 13:09:20.063 I ns/openshift-sdn daemonset/sdn Created pod: sdn-7gvsv Nov 05 13:09:20.068 I ns/openshift-dns pod/dns-default-m847c Successfully assigned openshift-dns/dns-default-m847c to ip-10-0-132-180.ec2.internal Nov 05 13:09:20.076 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v (6 times) Nov 05 13:09:20.082 I ns/openshift-sdn pod/sdn-7gvsv Successfully assigned openshift-sdn/sdn-7gvsv to ip-10-0-132-180.ec2.internal Nov 05 13:09:20.157 E clusteroperator/dns changed Degraded to True: NotAllDNSesAvailable: Not all desired DNS DaemonSets available Nov 05 13:09:21.182 W ns/openshift-sdn pod/ovs-6ml2j MountVolume.SetUp failed for volume "sdn-token-8gf7t" : couldn't propagate object cache: timed out waiting for the condition Nov 05 13:09:21.187 W ns/openshift-dns pod/dns-default-m847c MountVolume.SetUp failed for volume "config-volume" : couldn't propagate object cache: timed out waiting for the condition Nov 05 13:09:21.284 W ns/openshift-sdn pod/sdn-7gvsv MountVolume.SetUp failed for volume "config" : couldn't propagate object cache: timed out waiting for the condition Nov 05 13:09:21.288 W ns/openshift-sdn pod/sdn-7gvsv MountVolume.SetUp failed for volume "sdn-token-8gf7t" : couldn't propagate object cache: timed out waiting for the condition Nov 05 13:09:21.307 I node/ip-10-0-154-237.ec2.internal Node ip-10-0-154-237.ec2.internal event: Registered Node ip-10-0-154-237.ec2.internal in Controller Nov 05 13:09:21.311 I node/ip-10-0-132-180.ec2.internal Node ip-10-0-132-180.ec2.internal event: Registered Node ip-10-0-132-180.ec2.internal in Controller Nov 05 13:09:22.156 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (2 times) Nov 05 13:09:23.660 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (3 times) Nov 05 13:09:25.039 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (3 times) Nov 05 13:09:25.071 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (4 times) Nov 05 13:09:26.530 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (4 times) Nov 05 13:09:27.072 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (5 times) Nov 05 13:09:28.210 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (5 times) Nov 05 13:09:29.072 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (6 times) Nov 05 13:09:30.153 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (6 times) Nov 05 13:09:31.072 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (7 times) Nov 05 13:09:31.441 I ns/openshift-sdn pod/ovs-gk2d8 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fb6bdabf98794c00d2efc5d84527b9f16e513d2b7cf701bb24d8faaf1716e9aa" Nov 05 13:09:31.487 I ns/openshift-monitoring pod/node-exporter-6snqf Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:61a07ec4339553f5b39d39d6fe216ab78cd4670233c2cf387e51cb50d51638e5" Nov 05 13:09:31.511 I ns/openshift-multus pod/multus-cqsgk Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:31.568 I ns/openshift-sdn pod/sdn-scq8s Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:31.594 I ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:37d55994d18dabf18dcc6ded9d68952b2681451ea03ea9c12ee94a5253cfae34" Nov 05 13:09:32.153 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (7 times) Nov 05 13:09:33.075 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (8 times) Nov 05 13:09:33.272 - 15s W node/ip-10-0-132-180.ec2.internal node is not ready Nov 05 13:09:34.153 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (8 times) Nov 05 13:09:34.277 I ns/openshift-monitoring pod/node-exporter-6snqf Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:61a07ec4339553f5b39d39d6fe216ab78cd4670233c2cf387e51cb50d51638e5" Nov 05 13:09:34.309 I ns/openshift-sdn pod/ovs-6ml2j Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fb6bdabf98794c00d2efc5d84527b9f16e513d2b7cf701bb24d8faaf1716e9aa" Nov 05 13:09:34.349 I ns/openshift-sdn pod/sdn-7gvsv Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:34.400 I ns/openshift-monitoring pod/node-exporter-4l262 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:61a07ec4339553f5b39d39d6fe216ab78cd4670233c2cf387e51cb50d51638e5" Nov 05 13:09:34.450 I ns/openshift-multus pod/multus-r5rhl Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:34.487 I ns/openshift-cluster-node-tuning-operator pod/tuned-4dpxv Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:37d55994d18dabf18dcc6ded9d68952b2681451ea03ea9c12ee94a5253cfae34" Nov 05 13:09:34.541 I ns/openshift-monitoring pod/node-exporter-6snqf Created container node-exporter Nov 05 13:09:34.573 I ns/openshift-monitoring pod/node-exporter-6snqf Started container node-exporter Nov 05 13:09:34.578 I ns/openshift-monitoring pod/node-exporter-6snqf Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" Nov 05 13:09:35.071 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (9 times) Nov 05 13:09:36.154 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (9 times) Nov 05 13:09:37.072 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (10 times) Nov 05 13:09:37.244 I ns/openshift-monitoring pod/node-exporter-4l262 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:61a07ec4339553f5b39d39d6fe216ab78cd4670233c2cf387e51cb50d51638e5" Nov 05 13:09:37.428 I ns/openshift-monitoring pod/node-exporter-4l262 Created container node-exporter Nov 05 13:09:37.498 I ns/openshift-monitoring pod/node-exporter-4l262 Started container node-exporter Nov 05 13:09:37.502 I ns/openshift-monitoring pod/node-exporter-4l262 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" Nov 05 13:09:38.154 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (10 times) Nov 05 13:09:38.814 I ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:37d55994d18dabf18dcc6ded9d68952b2681451ea03ea9c12ee94a5253cfae34" Nov 05 13:09:38.819 I ns/openshift-multus pod/multus-cqsgk Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:38.825 I ns/openshift-sdn pod/sdn-scq8s Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:38.831 I ns/openshift-sdn pod/ovs-gk2d8 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fb6bdabf98794c00d2efc5d84527b9f16e513d2b7cf701bb24d8faaf1716e9aa" Nov 05 13:09:38.996 I ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q Created container tuned Nov 05 13:09:39.050 I ns/openshift-cluster-node-tuning-operator pod/tuned-rlz5q Started container tuned Nov 05 13:09:39.065 I ns/openshift-multus pod/multus-cqsgk Created container cni-plugins Nov 05 13:09:39.072 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (11 times) Nov 05 13:09:39.085 I ns/openshift-sdn pod/ovs-gk2d8 Created container openvswitch Nov 05 13:09:39.097 I ns/openshift-sdn pod/sdn-scq8s Created container install-cni-plugins Nov 05 13:09:39.115 I ns/openshift-multus pod/multus-cqsgk Started container cni-plugins Nov 05 13:09:39.214 I ns/openshift-sdn pod/ovs-gk2d8 Started container openvswitch Nov 05 13:09:39.414 I ns/openshift-sdn pod/sdn-scq8s Started container install-cni-plugins Nov 05 13:09:39.960 I ns/openshift-monitoring pod/node-exporter-6snqf Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" Nov 05 13:09:40.058 I ns/openshift-multus pod/multus-cqsgk Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55697815941b9d640533256ba35cb52b3f26cb5d472a96cc7610a394eb6b48ca" Nov 05 13:09:40.065 I ns/openshift-sdn pod/sdn-scq8s Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fb6bdabf98794c00d2efc5d84527b9f16e513d2b7cf701bb24d8faaf1716e9aa" already present on machine Nov 05 13:09:40.156 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (11 times) Nov 05 13:09:40.214 I ns/openshift-monitoring pod/node-exporter-6snqf Created container kube-rbac-proxy Nov 05 13:09:40.414 I ns/openshift-monitoring pod/node-exporter-6snqf Started container kube-rbac-proxy Nov 05 13:09:40.616 I ns/openshift-sdn pod/sdn-scq8s Created container sdn Nov 05 13:09:40.814 I ns/openshift-sdn pod/sdn-scq8s Started container sdn Nov 05 13:09:41.074 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (12 times) Nov 05 13:09:41.238 I node/ip-10-0-154-237.ec2.internal openshift-sdn done initializing node networking. Nov 05 13:09:41.605 I ns/openshift-sdn pod/sdn-7gvsv Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:41.609 I ns/openshift-multus pod/multus-r5rhl Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf663f087d6f23e354fa4dc77427ba1805e12b343c78d59719489c541cd647e5" Nov 05 13:09:41.613 I ns/openshift-cluster-node-tuning-operator pod/tuned-4dpxv Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:37d55994d18dabf18dcc6ded9d68952b2681451ea03ea9c12ee94a5253cfae34" Nov 05 13:09:41.617 I ns/openshift-sdn pod/ovs-6ml2j Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fb6bdabf98794c00d2efc5d84527b9f16e513d2b7cf701bb24d8faaf1716e9aa" Nov 05 13:09:41.811 I ns/openshift-sdn pod/sdn-7gvsv Created container install-cni-plugins Nov 05 13:09:41.867 I ns/openshift-cluster-node-tuning-operator pod/tuned-4dpxv Created container tuned Nov 05 13:09:41.874 I ns/openshift-multus pod/multus-r5rhl Created container cni-plugins Nov 05 13:09:41.890 I ns/openshift-sdn pod/sdn-7gvsv Started container install-cni-plugins Nov 05 13:09:41.916 I ns/openshift-sdn pod/ovs-6ml2j Created container openvswitch Nov 05 13:09:41.921 I ns/openshift-cluster-node-tuning-operator pod/tuned-4dpxv Started container tuned Nov 05 13:09:41.931 I ns/openshift-multus pod/multus-r5rhl Started container cni-plugins Nov 05 13:09:42.005 I ns/openshift-sdn pod/sdn-7gvsv Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fb6bdabf98794c00d2efc5d84527b9f16e513d2b7cf701bb24d8faaf1716e9aa" already present on machine Nov 05 13:09:42.205 I ns/openshift-sdn pod/ovs-6ml2j Started container openvswitch Nov 05 13:09:42.320 I ns/openshift-multus pod/multus-cqsgk Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55697815941b9d640533256ba35cb52b3f26cb5d472a96cc7610a394eb6b48ca" Nov 05 13:09:42.405 I ns/openshift-sdn pod/sdn-7gvsv Created container sdn Nov 05 13:09:42.463 I ns/openshift-multus pod/multus-cqsgk Created container kube-multus Nov 05 13:09:42.484 I ns/openshift-multus pod/multus-cqsgk Started container kube-multus Nov 05 13:09:42.606 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (12 times) Nov 05 13:09:42.809 I ns/openshift-sdn pod/sdn-7gvsv Started container sdn Nov 05 13:09:43.004 I ns/openshift-monitoring pod/node-exporter-4l262 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cf44080491a81dd53d683bca8fbfd8d51639ff8ba460e56b45f3455c6e6fa0c1" Nov 05 13:09:43.072 W ns/openshift-dns pod/dns-default-mlstv network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (13 times) Nov 05 13:09:43.196 I node/ip-10-0-132-180.ec2.internal openshift-sdn done initializing node networking. Nov 05 13:09:43.205 I ns/openshift-multus pod/multus-r5rhl Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55697815941b9d640533256ba35cb52b3f26cb5d472a96cc7610a394eb6b48ca" Nov 05 13:09:43.405 I ns/openshift-monitoring pod/node-exporter-4l262 Created container kube-rbac-proxy Nov 05 13:09:43.605 I ns/openshift-monitoring pod/node-exporter-4l262 Started container kube-rbac-proxy Nov 05 13:09:44.155 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (13 times) Nov 05 13:09:44.891 I ns/openshift-multus pod/multus-r5rhl Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55697815941b9d640533256ba35cb52b3f26cb5d472a96cc7610a394eb6b48ca" Nov 05 13:09:45.053 I ns/openshift-multus pod/multus-r5rhl Created container kube-multus Nov 05 13:09:45.080 I ns/openshift-multus pod/multus-r5rhl Started container kube-multus Nov 05 13:09:46.153 W ns/openshift-dns pod/dns-default-m847c network is not ready: runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: Missing CNI default network (14 times) Nov 05 13:09:46.847 W node/ip-10-0-154-237.ec2.internal condition Ready changed Nov 05 13:09:46.873 I ns/openshift-machine-config-operator pod/machine-config-daemon-fdwr2 node/ created Nov 05 13:09:46.876 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-fdwr2 Nov 05 13:09:46.884 I ns/openshift-machine-config-operator pod/machine-config-daemon-fdwr2 Successfully assigned openshift-machine-config-operator/machine-config-daemon-fdwr2 to ip-10-0-154-237.ec2.internal Nov 05 13:09:46.894 I ns/openshift-image-registry pod/node-ca-wfq45 node/ created Nov 05 13:09:46.899 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-wfq45 Nov 05 13:09:46.908 I ns/openshift-image-registry pod/node-ca-wfq45 Successfully assigned openshift-image-registry/node-ca-wfq45 to ip-10-0-154-237.ec2.internal Nov 05 13:09:47.124 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-g4ml8 (7 times) Nov 05 13:09:47.579 I ns/openshift-machine-config-operator pod/machine-config-daemon-fdwr2 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" Nov 05 13:09:49.923 W node/ip-10-0-132-180.ec2.internal condition Ready changed Nov 05 13:09:49.953 I ns/openshift-machine-config-operator pod/machine-config-daemon-f6p2d node/ created Nov 05 13:09:49.962 I ns/openshift-machine-config-operator daemonset/machine-config-daemon Created pod: machine-config-daemon-f6p2d Nov 05 13:09:49.970 I ns/openshift-machine-config-operator pod/machine-config-daemon-f6p2d Successfully assigned openshift-machine-config-operator/machine-config-daemon-f6p2d to ip-10-0-132-180.ec2.internal Nov 05 13:09:49.984 I ns/openshift-image-registry pod/node-ca-th7vz node/ created Nov 05 13:09:49.988 I ns/openshift-image-registry daemonset/node-ca Created pod: node-ca-th7vz Nov 05 13:09:49.992 I ns/openshift-image-registry pod/node-ca-th7vz Successfully assigned openshift-image-registry/node-ca-th7vz to ip-10-0-132-180.ec2.internal Nov 05 13:09:50.120 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v Updated machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-rq25v (7 times) Nov 05 13:09:50.740 I ns/openshift-machine-config-operator pod/machine-config-daemon-f6p2d Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" Nov 05 13:09:52.535 I ns/openshift-machine-config-operator pod/machine-config-daemon-fdwr2 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" Nov 05 13:09:52.654 I ns/openshift-machine-config-operator pod/machine-config-daemon-fdwr2 Created container machine-config-daemon Nov 05 13:09:52.676 I ns/openshift-machine-config-operator pod/machine-config-daemon-fdwr2 Started container machine-config-daemon Nov 05 13:09:53.085 I ns/openshift-dns pod/dns-default-mlstv Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bb507aff0831dc2fa73b5b2ad7cd48c87c92aa63090863250e455342b5330864" Nov 05 13:09:54.712 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Node "ip-10-0-139-89.ec2.internal" drained Nov 05 13:09:54.957 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Deleted machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1a-4g5hj Nov 05 13:09:55.060 W node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:09:55.167 I ns/openshift-image-registry pod/node-ca-wfq45 Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" Nov 05 13:09:55.692 I ns/openshift-machine-config-operator pod/machine-config-daemon-f6p2d Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2718d42a03608c657c255a9019a6d2bbebf556546b745492dd249d1ef630ab72" Nov 05 13:09:55.812 I ns/openshift-machine-config-operator pod/machine-config-daemon-f6p2d Created container machine-config-daemon Nov 05 13:09:55.834 I ns/openshift-machine-config-operator pod/machine-config-daemon-f6p2d Started container machine-config-daemon Nov 05 13:09:55.866 I ns/openshift-dns pod/dns-default-mlstv Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bb507aff0831dc2fa73b5b2ad7cd48c87c92aa63090863250e455342b5330864" Nov 05 13:09:56.010 I ns/openshift-dns pod/dns-default-mlstv Created container dns Nov 05 13:09:56.037 I ns/openshift-dns pod/dns-default-mlstv Started container dns Nov 05 13:09:56.042 I ns/openshift-dns pod/dns-default-mlstv Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75534c301f7025ee45995a29b21c155308ea16020370aac3dc50dd60e15a48f5" Nov 05 13:09:56.132 I ns/openshift-dns pod/dns-default-m847c Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bb507aff0831dc2fa73b5b2ad7cd48c87c92aa63090863250e455342b5330864" Nov 05 13:09:56.150 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft node/ip-10-0-150-53.ec2.internal graceful deletion within 30s Nov 05 13:09:56.156 I ns/openshift-ingress pod/router-default-5796647cdf-sr9ft Stopping container router Nov 05 13:09:56.185 I ns/openshift-ingress pod/router-default-5796647cdf-ww9br node/ created Nov 05 13:09:56.191 I ns/openshift-ingress replicaset/router-default-5796647cdf Created pod: router-default-5796647cdf-ww9br Nov 05 13:09:56.211 I ns/openshift-ingress pod/router-default-5796647cdf-ww9br Successfully assigned openshift-ingress/router-default-5796647cdf-ww9br to ip-10-0-154-237.ec2.internal Nov 05 13:09:56.333 I node/ip-10-0-139-89.ec2.internal Node ip-10-0-139-89.ec2.internal event: Removing Node ip-10-0-139-89.ec2.internal from Controller Nov 05 13:09:56.882 E ns/openshift-ingress pod/router-default-5796647cdf-sr9ft node/ip-10-0-150-53.ec2.internal container=router container exited with code 2 (Error): ter reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 12:16:48.708498 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 12:17:14.974496 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nW1105 12:23:28.302364 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nI1105 12:23:56.627168 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 12:24:01.623675 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nW1105 12:31:10.415533 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nW1105 12:40:43.490744 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nW1105 12:56:23.573194 1 reflector.go:341] github.com/openshift/router/pkg/router/controller/factory/factory.go:112: watch of *v1.Route ended with: The resourceVersion for the provided watch is too old.\nI1105 13:09:41.126985 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 13:09:46.124267 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\nI1105 13:09:51.126087 1 router.go:561] Router reloaded:\n - Proxy protocol on, checking http://localhost:80 ...\n - Health check ok : 0 retry attempt(s).\n Nov 05 13:09:57.775 I ns/openshift-image-registry pod/node-ca-th7vz Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" Nov 05 13:09:58.772 I ns/openshift-dns pod/dns-default-m847c Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bb507aff0831dc2fa73b5b2ad7cd48c87c92aa63090863250e455342b5330864" Nov 05 13:09:58.904 W clusteroperator/network changed Progressing to False Nov 05 13:09:58.931 I ns/openshift-dns pod/dns-default-m847c Created container dns Nov 05 13:09:58.972 I ns/openshift-dns pod/dns-default-m847c Started container dns Nov 05 13:09:58.977 I ns/openshift-dns pod/dns-default-m847c Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75534c301f7025ee45995a29b21c155308ea16020370aac3dc50dd60e15a48f5" Nov 05 13:09:59.889 I ns/openshift-dns pod/dns-default-mlstv Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75534c301f7025ee45995a29b21c155308ea16020370aac3dc50dd60e15a48f5" Nov 05 13:10:00.224 I ns/openshift-dns pod/dns-default-mlstv Created container dns-node-resolver Nov 05 13:10:00.266 I ns/openshift-dns pod/dns-default-mlstv Started container dns-node-resolver Nov 05 13:10:01.832 I node/ip-10-0-150-53.ec2.internal Node ip-10-0-150-53.ec2.internal status is now: NodeNotSchedulable Nov 05 13:10:02.696 I ns/openshift-dns pod/dns-default-m847c Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75534c301f7025ee45995a29b21c155308ea16020370aac3dc50dd60e15a48f5" Nov 05 13:10:03.006 I ns/openshift-dns pod/dns-default-m847c Created container dns-node-resolver Nov 05 13:10:03.045 I ns/openshift-dns pod/dns-default-m847c Started container dns-node-resolver Nov 05 13:10:03.779 I ns/openshift-ingress pod/router-default-5796647cdf-ww9br Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f0b7726741f00cc06e3c594e4f5d2db08da4c5cd0cd7bf14a467668ec5f07c0" Nov 05 13:10:06.509 W ns/openshift-ingress pod/router-default-5796647cdf-sr9ft node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:07.137 W ns/openshift-sdn pod/ovs-bf9k4 node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.143 W ns/openshift-sdn pod/ovs-bf9k4 node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.154 W ns/openshift-multus pod/multus-9f997 node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.160 W ns/openshift-multus pod/multus-9f997 node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.174 W ns/openshift-monitoring pod/node-exporter-6csgh node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.174 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Node "ip-10-0-150-53.ec2.internal" drained Nov 05 13:10:07.181 W ns/openshift-monitoring pod/node-exporter-6csgh node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.193 W ns/openshift-cluster-node-tuning-operator pod/tuned-fh9mw node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.199 W ns/openshift-monitoring endpoints/node-exporter Failed to update endpoint openshift-monitoring/node-exporter: Operation cannot be fulfilled on endpoints "node-exporter": the object has been modified; please apply your changes to the latest version and try again Nov 05 13:10:07.204 W ns/openshift-cluster-node-tuning-operator pod/tuned-fh9mw node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.218 W ns/openshift-sdn pod/sdn-mqvgr node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.235 W ns/openshift-sdn pod/sdn-mqvgr node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.250 W ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.256 W ns/openshift-machine-config-operator pod/machine-config-daemon-97dv6 node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.268 W ns/openshift-dns pod/dns-default-dg9dw node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.273 W ns/openshift-dns pod/dns-default-dg9dw node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.294 W ns/openshift-image-registry pod/node-ca-pgwsf node/ip-10-0-139-89.ec2.internal graceful deletion within 0s Nov 05 13:10:07.303 W ns/openshift-image-registry pod/node-ca-pgwsf node/ip-10-0-139-89.ec2.internal deleted Nov 05 13:10:07.508 I ns/openshift-machine-api machine/ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Deleted machine ci-op-d9nlcxqc-0350a-4ttjd-worker-us-east-1b-8v9hg Nov 05 13:10:07.513 W node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:07.705 W clusteroperator/dns changed Degraded to False: AsExpected: All desired DNS DaemonSets available and operand Namespace exists Nov 05 13:10:11.223 I ns/openshift-ingress pod/router-default-5796647cdf-ww9br Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3f0b7726741f00cc06e3c594e4f5d2db08da4c5cd0cd7bf14a467668ec5f07c0" Nov 05 13:10:11.245 I ns/openshift-image-registry pod/node-ca-wfq45 Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" Nov 05 13:10:11.331 I node/ip-10-0-150-53.ec2.internal Node ip-10-0-150-53.ec2.internal event: Removing Node ip-10-0-150-53.ec2.internal from Controller Nov 05 13:10:11.400 I ns/openshift-ingress pod/router-default-5796647cdf-ww9br Created container router Nov 05 13:10:11.406 I ns/openshift-image-registry pod/node-ca-wfq45 Created container node-ca Nov 05 13:10:11.427 I ns/openshift-ingress pod/router-default-5796647cdf-ww9br Started container router Nov 05 13:10:11.436 I ns/openshift-image-registry pod/node-ca-wfq45 Started container node-ca Nov 05 13:10:13.052 I ns/openshift-image-registry pod/node-ca-th7vz Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f0892cc064b5ae214018b88035b9e3ae406f7ef291f8dd0dc1abe91e212c96f" Nov 05 13:10:13.173 I ns/openshift-image-registry pod/node-ca-th7vz Created container node-ca Nov 05 13:10:13.198 I ns/openshift-image-registry pod/node-ca-th7vz Started container node-ca Nov 05 13:10:17.423 W clusteroperator/dns changed Progressing to False: AsExpected: Desired and available number of DNS DaemonSets are equal Nov 05 13:10:27.319 W ns/openshift-dns pod/dns-default-t7c4g node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.332 W ns/openshift-dns pod/dns-default-t7c4g node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.339 W ns/openshift-sdn pod/sdn-64k7s node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.353 W ns/openshift-sdn pod/sdn-64k7s node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.361 W ns/openshift-image-registry pod/node-ca-gtr4s node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.374 W ns/openshift-image-registry pod/node-ca-gtr4s node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.385 W ns/openshift-multus pod/multus-cs74v node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.392 W ns/openshift-multus pod/multus-cs74v node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.399 W ns/openshift-cluster-node-tuning-operator pod/tuned-2smxx node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.408 W ns/openshift-cluster-node-tuning-operator pod/tuned-2smxx node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.426 W ns/openshift-monitoring pod/node-exporter-82mqc node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.430 W ns/openshift-monitoring pod/node-exporter-82mqc node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.443 W ns/openshift-sdn pod/ovs-vdg2d node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.452 W ns/openshift-sdn pod/ovs-vdg2d node/ip-10-0-150-53.ec2.internal deleted Nov 05 13:10:27.460 W ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl node/ip-10-0-150-53.ec2.internal graceful deletion within 0s Nov 05 13:10:27.466 W ns/openshift-machine-config-operator pod/machine-config-daemon-jkfpl node/ip-10-0-150-53.ec2.internal deleted
Find was mentions in log files
openshift-tests [Feature:Image][triggers][Serial] ImageStream API TestImageStreamMappingCreate [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream API TestImageStreamTagLifecycleHook [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream API TestImageStreamWithoutDockerImageConfig [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream admission TestImageStreamAdmitSpecUpdate [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream admission TestImageStreamAdmitStatusUpdate [Suite:openshift/conformance/serial]
openshift-tests [Feature:Image][triggers][Serial] ImageStream admission TestImageStreamTagsAdmission [Suite:openshift/conformance/serial]
openshift-tests [Feature:Machines][Serial] Managed cluster should grow and decrease when scaling different machineSets simultaneously [Suite:openshift/conformance/serial]
openshift-tests [Feature:OpenShiftAuthorization][Serial] authorization TestAuthorizationResourceAccessReview should succeed [Suite:openshift/conformance/serial]
openshift-tests [Feature:ProjectAPI][Serial] TestUnprivilegedNewProjectDenied [Suite:openshift/conformance/serial]
openshift-tests [Serial] API data in etcd should be stored at the correct location and version for all resources [Suite:openshift/conformance/serial]
openshift-tests [Suite:openshift/oauth][Serial] ldap group sync can sync groups from ldap [Suite:openshift/conformance/serial]
openshift-tests [k8s.io] [sig-node] kubelet [k8s.io] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. [Suite:openshift/conformance/serial] [Suite:k8s] [Serial]
openshift-tests [sig-api-machinery] Namespaces [Serial] should always delete fast (ALL of 100 namespaces in 150 seconds) [Feature:ComprehensiveNamespaceDraining] [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-api-machinery] Namespaces [Serial] should delete fast enough (90 percent of 100 namespaces in 150 seconds) [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-api-machinery] Namespaces [Serial] should ensure that all pods are removed when a namespace is deleted [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-api-machinery] Namespaces [Serial] should ensure that all services are removed when a namespace is deleted [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should not update pod when spec was updated and update strategy is OnDelete [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should retry creating failed daemon pods [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should rollback without unnecessary restarts [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should run and stop complex daemon [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should run and stop complex daemon with node affinity [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should run and stop simple daemon [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-apps] Daemon set [Serial] should update pod when spec was updated and update strategy is RollingUpdate [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-cli] Kubectl client [k8s.io] Kubectl taint [Serial] should remove all the taints with the same key off a node [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-cli] Kubectl client [k8s.io] Kubectl taint [Serial] should update the taint on a node [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-network] Service endpoints latency should not be very high [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s] [Serial]
openshift-tests [sig-scheduling] NoExecuteTaintManager Multiple Pods [Serial] evicts pods with minTolerationSeconds [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Multiple Pods [Serial] only evicts pods without tolerations from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] doesn't evict pod with tolerations from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] eventually evict pod with finite tolerations from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] evicts pods from tainted nodes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] NoExecuteTaintManager Single Pod [Serial] removing taint cancels eviction [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] PodPriorityResolution [Serial] validates critical system priorities are created and resolved [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that NodeAffinity is respected if not matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that NodeSelector is respected if matching [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that NodeSelector is respected if not matching [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that required NodeAffinity setting is respected if matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that taints-tolerations is respected if matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPriorities [Serial] Pod should avoid nodes that have avoidPod annotation [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPriorities [Serial] Pod should be preferably scheduled to nodes pod can tolerate [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-scheduling] SchedulerPriorities [Serial] Pod should be scheduled to node that don't match the PodAntiAffinity terms [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] EmptyDir wrapper volumes should not cause race condition when used for configmaps [Serial] [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]
openshift-tests [sig-storage] PersistentVolumes-local Pods sharing a single local PV [Serial] all pods should be running [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] PersistentVolumes-local Stress with local volumes [Serial] should be able to process many pods and reuse local volumes [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create volume metrics in Volume Manager [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create volume metrics with the correct PVC ref [Suite:openshift/conformance/serial] [Suite:k8s]
operator All images are built and tagged into stable
operator Create the release image containing all images built by this job
operator Find all of the input images from ocp/4.2:${component} and tag them into the output image stream
operator Run template e2e-aws-serial - e2e-aws-serial container lease
operator Run template e2e-aws-serial - e2e-aws-serial container setup
operator Run template e2e-aws-serial - e2e-aws-serial container teardown
operator Run template e2e-aws-serial - e2e-aws-serial container test
openshift-tests [registry][Serial][Suite:openshift/registry/serial] Image signature workflow can push a signed image to openshift registry and verify it [Suite:openshift/conformance/serial]
openshift-tests [sig-storage] CSI Volumes CSI Topology test using GCE PD driver [Serial] should fail to schedule a pod with a zone missing from AllowedTopologies; PD is provisioned with delayed volume binding [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes CSI Topology test using GCE PD driver [Serial] should fail to schedule a pod with a zone missing from AllowedTopologies; PD is provisioned with immediate volume binding [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes CSI Topology test using GCE PD driver [Serial] should provision zonal PD with delayed volume binding and AllowedTopologies set and mount the volume to a pod [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes CSI Topology test using GCE PD driver [Serial] should provision zonal PD with delayed volume binding and mount the volume to a pod [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes CSI Topology test using GCE PD driver [Serial] should provision zonal PD with immediate volume binding and AllowedTopologies set and mount the volume to a pod [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] volumeMode should fail in binding dynamic provisioned PV to PVC [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should access volume from different nodes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should allow concurrent writes on the same node [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should create and delete block persistent volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with defaults [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with mount options [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should verify container cannot write to subpath readonly volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ext3)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ext3)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ext4)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ext4)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should create sc, pod, pv, and pvc, read/write to the pv, and delete all created resources [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should access volume from different nodes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should allow concurrent writes on the same node [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should create and delete block persistent volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should provision storage with defaults [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should provision storage with mount options [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (xfs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (xfs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic Snapshot] snapshottable should create snapshot with defaults [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should be able to unmount after the subpath directory is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing directories when readOnly specified in the volumeSource [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing directory [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing single file [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support file as subpath [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support non-existent path [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support readOnly directory specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support readOnly file specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should verify container cannot write to subpath readonly volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (ext3)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (ext3)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (ext4)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (ext4)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (ntfs)][sig-windows] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (ntfs)][sig-windows] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (xfs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (xfs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode should fail to create pod by failing to mount volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing directory [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing single file [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support file as subpath [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support non-existent path [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support readOnly directory specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support readOnly file specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should verify container cannot write to subpath readonly volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ext3)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ext3)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ext4)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ext4)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode should create sc, pod, pv, and pvc, read/write to the pv, and delete all created resources [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ntfs)][sig-windows] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ntfs)][sig-windows] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (xfs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (xfs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (block volmode)] volumeMode should fail in binding dynamic provisioned PV to PVC [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] provisioning should access volume from different nodes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] provisioning should allow concurrent writes on the same node [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] provisioning should create and delete block persistent volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with defaults [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with mount options [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] subPath should verify container cannot write to subpath readonly volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (default fs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ext3)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ext3)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ext4)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ext4)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should create sc, pod, pv, and pvc, read/write to the pv, and delete all created resources [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should access volume from different nodes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should allow concurrent writes on the same node [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should create and delete block persistent volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should provision storage with defaults [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should provision storage with mount options [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (ntfs)][sig-windows] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (xfs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Dynamic PV (xfs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should be able to unmount after the subpath directory is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing directories when readOnly specified in the volumeSource [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing directory [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing single file [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support file as subpath [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support non-existent path [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support readOnly directory specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should support readOnly file specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] subPath should verify container cannot write to subpath readonly volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (default fs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (ext3)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (ext3)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (ext4)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (ext4)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (ntfs)][sig-windows] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (ntfs)][sig-windows] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (xfs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Inline-volume (xfs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode should fail to create pod by failing to mount volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing directory [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing single file [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support file as subpath [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support non-existent path [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support readOnly directory specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support readOnly file specified in the volumeMount [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should verify container cannot write to subpath readonly volumes [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (default fs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (ext3)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (ext3)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (ext4)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (ext4)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode should create sc, pod, pv, and pvc, read/write to the pv, and delete all created resources [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (ntfs)][sig-windows] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (ntfs)][sig-windows] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (xfs)] volumes should allow exec of files on the volume [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] In-tree Volumes [Driver: local][LocalVolumeType: gce-localssd-scsi-fs] [Serial] [Testpattern: Pre-provisioned PV (xfs)] volumes should be mountable [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] One pod requesting one prebound PVC should be able to mount volume and read from pod1 [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] One pod requesting one prebound PVC should be able to mount volume and write from pod1 [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] Set fsGroup for local volume should set different fsGroup for second pod if first pod is deleted [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] Set fsGroup for local volume should set fsGroup for one pod [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] Set fsGroup for local volume should set same fsGroup for two pods simultaneously [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] Two pods mounting a local volume at the same time should be able to write from pod1 and read from pod2 [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] PersistentVolumes-local [Volume type: gce-localssd-scsi-fs] [Serial] Two pods mounting a local volume one after the other should be able to write from pod1 and read from pod2 [Suite:openshift/conformance/serial] [Suite:k8s] [Skipped:gce]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create bound pv/pvc count metrics for pvc controller after creating both pv and pvc [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create none metrics for pvc controller before creating any PV or PVC [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create unbound pv count metrics for pvc controller after creating pv only [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics PVController should create unbound pvc count metrics for pvc controller after creating pvc only [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create metrics for total number of volumes in A/D Controller [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create metrics for total time taken in volume operations in P/V Controller [Suite:openshift/conformance/serial] [Suite:k8s]
openshift-tests [sig-storage] [Serial] Volume metrics should create prometheus metrics for volume provisioning and attach/detach [Suite:openshift/conformance/serial] [Suite:k8s]