[Feature:Builds][Conformance] s2i build with a root user image should create a root build and pass with a privileged SCC #17883

sosiouxme · 2017-12-19T16:33:45Z

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/17773/test_pull_request_origin_extended_conformance_gce/13246/

/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:75
Expected error:
    <*errors.errorString | 0xc42115c500>: {
        s: "The build \"nodejspass-1\" status is \"Failed\"",
    }
    The build "nodejspass-1" status is "Failed"
not to have occurred
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:91

The text was updated successfully, but these errors were encountered:

bparees · 2017-12-22T05:58:39Z

failure to get pod logs

Dec 19 12:36:26.607: INFO: Running 'oc logs --config=/tmp/extended-test-s2i-build-root-xqdq2-dnlfm-user.kubeconfig --namespace=extended-test-s2i-build-root-xqdq2-dnlfm pod/nodejspass-1-build'
Dec 19 12:36:26.980: INFO: Error running &{/data/src/github.com/openshift/origin/_output/local/bin/linux/amd64/oc [oc logs --config=/tmp/extended-test-s2i-build-root-xqdq2-dnlfm-user.kubeconfig --namespace=extended-test-s2i-build-root-xqdq2-dnlfm pod/nodejspass-1-build] []   Error from server (NotFound): the server could not find the requested resource ( pods/log nodejspass-1-build)
 Error from server (NotFound): the server could not find the requested resource ( pods/log nodejspass-1-build)
 [] <nil> 0xc421c9ee40 exit status 1 <nil> <nil> true [0xc4218881f8 0xc421888270 0xc421888270] [0xc4218881f8 0xc421888270] [0xc421888208 0xc421888260] [0x9895f0 0x9896f0] 0xc421aba540 <nil>}:
Error from server (NotFound): the server could not find the requested resource ( pods/log nodejspass-1-build)
Dec 19 12:36:26.980: INFO: Error retrieving logs for pod "nodejspass-1-build": exit status 1

but describing the pod worked:

Dec 19 12:36:26.298: INFO: Running 'oc describe --config=/tmp/extended-test-s2i-build-root-xqdq2-dnlfm-user.kubeconfig --namespace=extended-test-s2i-build-root-xqdq2-dnlfm pod/nodejspass-1-build'
Dec 19 12:36:26.607: INFO: Describing pod "nodejspass-1-build"
Name:           nodejspass-1-build
Namespace:      extended-test-s2i-build-root-xqdq2-dnlfm

sjenning · 2018-01-22T19:36:01Z

@frobware PTAL

0xmichalis · 2018-01-28T21:10:30Z

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/18244/test_pull_request_origin_extended_conformance_gce/15179/

frobware · 2018-02-01T10:22:13Z

I have been trying to reproduce this error for quite a few days but without success.

I did run into the following yesterday:

Expected error:
    <*errors.errorString | 0xc420299f70>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
not to have occurred

but that took some effort:

$ ./ginkgo -p -nodes=4 -focus=".*s2i build with a root user image" -untilItFails=true extended.test
Tests failed on attempt #214
Ginkgo ran 1 suite in 9h38m12.151562892s

frobware · 2018-02-05T10:05:13Z

First time I have seen this error now. Running via:

./ginkgo -p -nodes=4 -focus=".*Builds" -untilItFails=true extended.test

• Failure in Spec Setup (JustBeforeEach) [693.919 seconds]
[Feature:Builds][Conformance] s2i build with a root user image
/home/aim/go-projects/origin/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:16
  
  /home/aim/go-projects/origin/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:23
    should create a root build and fail without a privileged SCC [Suite:openshift/conformance/parallel] [JustBeforeEach]
    /home/aim/go-projects/origin/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:44

    Expected error:
        <*errors.errorString | 0xc4201490f0>: {
            s: "The build \"nodejsroot-1\" status is \"Error\"",
        }
        The build "nodejsroot-1" status is "Error"
    not to have occurred

    /home/aim/go-projects/origin/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:34

Feb  2 08:12:20.589: INFO: Running AfterSuite actions on all node
Feb  2 08:12:20.590: INFO: Running AfterSuite actions on node 1

I have been trying to reproduce this bug for a while now but maybe my
invocations have been too short. This (and the only time I have seen
this) failed after 25 hours.

[Fail] [Feature:Builds][Conformance] s2i build with a root user image  [JustBeforeEach] should create a root build and fail without a privileged SCC [Suite:openshift/conformance/parallel] 
/home/aim/go-projects/origin/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:34

Ran 2 of 440 Specs in 694.479 seconds
FAIL! -- 0 Passed | 2 Failed | 0 Pending | 438 Skipped 
Tests failed on attempt #545

Ginkgo ran 1 suite in 25h20m35.279131559s

sjenning · 2018-02-05T15:58:42Z

@frobware this looks yet another "quick container GC" where the container and its logs are GCed before the test can read them

sjenning · 2018-02-05T20:03:23Z

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/18417/test_pull_request_origin_extended_conformance_gce/15596

frobware · 2018-02-06T12:44:10Z

I switched my testing to use overlay2 (similar to CI) and was able to reproduce a little quicker. The quickest has been 36 minutes in 4 iterations, the other occurrence happened after 50 minutes.

frobware · 2018-02-15T09:54:51Z

I'm still investigating this but it is very hard to reproduce. This week, and on a whim, I switched from docker 1.12.6 to docker 1.13 and got 1 error in 25 hours. Last night/today I have had one occurrence in 13 hours. Looking through the logs I believe this "context cancelled" error happens each time:

Feb 15 08:26:13 rhel-74-vm-1 etcd[1287]: failed to receive watch request from gRPC stream ("rpc error: code = Unavailable desc = stream error: str
Feb 15 08:26:13 rhel-74-vm-1 etcd[1287]: failed to receive watch request from gRPC stream ("rpc error: code = Canceled desc = context canceled")

which appears to be similar to grpc/grpc-go#1134

sosiouxme · 2018-03-06T19:02:48Z

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/18641/test_pull_request_origin_extended_conformance_install/8549/ today

openshift-bot · 2018-06-05T00:41:28Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot · 2018-07-05T00:42:47Z

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

openshift-bot · 2018-08-04T02:02:47Z

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

sosiouxme added the kind/test-flake Categorizes issue or PR as related to test flakes. label Dec 19, 2017

sosiouxme mentioned this issue Dec 19, 2017

diagnostics: individual parameters #17773

Merged

bparees assigned sjenning Dec 22, 2017

bparees added the sig/pod label Dec 22, 2017

pweil- added the priority/P1 label Jan 3, 2018

mfojtik mentioned this issue Jan 8, 2018

Fix passing container to pod logs from dc #18017

Merged

sjenning assigned frobware and unassigned sjenning Jan 22, 2018

0xmichalis mentioned this issue Jan 28, 2018

node selector annotation for any label. #18244

Merged

sjenning mentioned this issue Feb 5, 2018

UPSTREAM: 59279: nodelifecycle: set OutOfDisk unknown on node timeout #18417

Merged

sosiouxme mentioned this issue Mar 6, 2018

diagnostics: AggregatedLogging ClusterRoleBindings false negative fix #18641

Merged

knobunc mentioned this issue Mar 6, 2018

Multiple auto-egress-IP fixes #18808

Merged

gabemontero mentioned this issue Mar 20, 2018

allow for bypass of not proceeding deployment check jenkins extended tests given update center sync delay is now on startup #19008

Merged

openshift-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 5, 2018

openshift-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jul 5, 2018

openshift-ci-robot closed this as completed Aug 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature:Builds][Conformance] s2i build with a root user image should create a root build and pass with a privileged SCC #17883

[Feature:Builds][Conformance] s2i build with a root user image should create a root build and pass with a privileged SCC #17883

sosiouxme commented Dec 19, 2017

bparees commented Dec 22, 2017

sjenning commented Jan 22, 2018

0xmichalis commented Jan 28, 2018

frobware commented Feb 1, 2018

frobware commented Feb 5, 2018

sjenning commented Feb 5, 2018

sjenning commented Feb 5, 2018

frobware commented Feb 6, 2018

frobware commented Feb 15, 2018

sosiouxme commented Mar 6, 2018

openshift-bot commented Jun 5, 2018

openshift-bot commented Jul 5, 2018

openshift-bot commented Aug 4, 2018

[Feature:Builds][Conformance] s2i build with a root user image should create a root build and pass with a privileged SCC #17883

[Feature:Builds][Conformance] s2i build with a root user image should create a root build and pass with a privileged SCC #17883

Comments

sosiouxme commented Dec 19, 2017

bparees commented Dec 22, 2017

sjenning commented Jan 22, 2018

0xmichalis commented Jan 28, 2018

frobware commented Feb 1, 2018

frobware commented Feb 5, 2018

sjenning commented Feb 5, 2018

sjenning commented Feb 5, 2018

frobware commented Feb 6, 2018

frobware commented Feb 15, 2018

sosiouxme commented Mar 6, 2018

openshift-bot commented Jun 5, 2018

openshift-bot commented Jul 5, 2018

openshift-bot commented Aug 4, 2018