Gitlab build fails with docker image cypress/included:3.8.3

[bahrmichael]

I'm trying to run a gitlab cicd job with cypress/included:3.8.3 as the base image.

e2e:
    image: cypress/included:3.8.3
    script:
        - cypress run --record --project ./e2e --config-file ./e2e/cypress-ci.json

Current behavior:

The build stalls with the following message. Tests don't seem to run at all.

Uncaught Exception:
 TypeError: Cannot read property 'split' of undefined
     at pipesToCommas (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/lib/util/args.js:77:14)
     at JSONOrCoerce (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/lib/util/args.js:97:9)
     at /root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:13401:38
     at /root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:4905:15
     at baseForOwn (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:2990:24)
     at Function.mapValues (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:13400:7)
     at /root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:4368:28
     at arrayReduce (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:683:21)
     at baseWrapperValue (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:4367:14)
     at LodashWrapper.wrapperValue (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/node_modules/lodash/lodash.js:9050:14)
     at sanitizeAndConvertNestedArgs (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/lib/util/args.js:135:4)
     at Object.toObject (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/lib/util/args.js:243:24)
     at Object.start (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/lib/cypress.js:154:31)
     at Object.<anonymous> (/root/.cache/Cypress/3.8.3/Cypress/resources/app/packages/server/index.js:27:43)
     at Module._compile (internal/modules/cjs/loader.js:880:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:892:10)
     at Module.load (internal/modules/cjs/loader.js:735:32)
     at Module._load (internal/modules/cjs/loader.js:648:12)
     at Module._load (electron/js2c/asar.js:717:26)
     at Function.Module._load (electron/js2c/asar.js:717:26)
     at Module.require (internal/modules/cjs/loader.js:775:19)
     at require (internal/modules/cjs/helpers.js:68:18)
     at Object.<anonymous> (/root/.cache/Cypress/3.8.3/Cypress/resources/app/index.js:2:1)
     at Module._compile (internal/modules/cjs/loader.js:880:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:892:10)
     at Module.load (internal/modules/cjs/loader.js:735:32)
     at Module._load (internal/modules/cjs/loader.js:648:12)
     at Module._load (electron/js2c/asar.js:717:26)
     at Function.Module._load (electron/js2c/asar.js:717:26)
     at Object.<anonymous> (electron/js2c/browser_init.js:5718:12)
     at Object../lib/browser/init.ts (electron/js2c/browser_init.js:5725:30)
     at __webpack_require__ (electron/js2c/browser_init.js:20:30)
     at electron/js2c/browser_init.js:84:18
     at electron/js2c/browser_init.js:87:10
     at NativeModule.compile (internal/bootstrap/loaders.js:302:5)
     at NativeModule.compileForPublicLoader (internal/bootstrap/loaders.js:219:8)
     at loadNativeModule (internal/modules/cjs/helpers.js:23:9)
     at Module._load (internal/modules/cjs/loader.js:630:15)
     at Module._load (electron/js2c/asar.js:717:26)
     at Function.Module._load (electron/js2c/asar.js:717:26)
     at Function.Module.runMain (internal/modules/cjs/loader.js:944:10)
     at internal/main/run_main_module.js:17:11
 Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"

Desired behavior:

I would like the tests to start and complete.

Test code to reproduce

Code for reproduction: https://gitlab.com/mptickets/gitlab-reproducer
Build that shows the failure: https://gitlab.com/mptickets/gitlab-reproducer/-/jobs/421010601

Versions

Cypress docker image: cypress/included:3.8.3

[bahmutov]

I can recreate this problem in https://gitlab.com/cypress-io/cypress-example-included. When looking at debug logs with DEBUG=cypress:cli I see weird config parsed

2020-01-30T18:03:31.544Z cypress:cli NODE_OPTIONS is not set
 2020-01-30T18:03:31.586Z cypress:cli parsed cli options 
{ config: 'if [ -x /usr/local/bin/bash ]; then\n' + 
'\texec /usr/local/bin/bash \n' + 'elif [ -x /usr/bin/bash ]; then\n' + 
'\texec /usr/bin/bash \n' + 'elif [ -x /bin/bash ]; then\n' + 
'\texec /bin/bash \n' + 'elif [ -x /usr/local/bin/sh ]; then\n' + 
'\texec /usr/local/bin/sh \n' + 'elif [ -x /usr/bin/sh ]; then\n' + 
'\texec /usr/bin/sh \n' + 'elif [ -x /bin/sh ]; then\n' + 
'\texec /bin/sh \n' + 'elif [ -x /busybox/sh ]; then\n' + 
'\texec /busybox/sh \n' + 'else\n' + '\techo shell not found\n' + 
'\texit 1\n' + 'fi\n' + '\n' }

Hmm, how is this running there

[bahmutov]

Second problem: after the crash, the process does NOT exit, so we are not handling the error correctly.

[bahmutov]

Running with DEBUG=cypress:cli:cli shows there are arguments coming from GitLab CI script?

These are not part of our Docker image

$ docker run -it --entrypoint=/bin/bash cypress/included:3.8.3
root@b01a4742c330:/# DEBUG=* cypress run
  cypress:cli:cli cli starts with arguments ["/usr/local/bin/node","/usr/local/bin/cypress","run"] +0ms
  cypress:cli NODE_OPTIONS is not set

[bahmutov]

For reference, here is Cypress CLI entry script

$ docker run -it --entrypoint=/bin/bash cypress/included:3.8.3
root@0ef9cbfda54b:/# which cypress
/usr/local/bin/cypress
root@0ef9cbfda54b:/# cat /usr/local/bin/cypress
#!/usr/bin/env node

require('../lib/cli').init()

[bahmutov]

@bahrmichael I have solved the immediate issue, see https://gitlab.com/cypress-io/cypress-example-included Turns out GitLab wraps entrypoint in its own script, breaking it. So you need to clear the entrypoint like this

e2e:
  image:
    name: cypress/included:3.8.3
    # cypress/included images have entrypoint set to "cypress"
    # which conflicts with GitLab CI wrapper shell script
    entrypoint: [""]
  script:
    - cypress run --project ./e2e

I will keep this issue open for myself - we still need to catch the config parsing exception and exit with status code and not keep the test runner hanging.

[bahmutov]

Trying this locally, the hanging might be GitLab thing, local npm run dev with invalid config does exit fine (but without explaining what is going on)

[bahmutov]

Simplest case that crashes:

DEBUG=cypress:server:args npm run dev -- --project ~/git/cypress-test-tiny --config 'a'
  cypress:server:args parsing config: a +0ms

/Users/gleb/git/cypress/packages/server/lib/util/args.js:77
  return str.split('|').join(',')
             ^
TypeError: Cannot read property 'split' of undefined

[bahrmichael]

@bahrmichael I have solved the immediate issue, see https://gitlab.com/cypress-io/cypress-example-included Turns out GitLab wraps entrypoint in its own script, breaking it. So you need to clear the entrypoint like this

e2e:
  image:
    name: cypress/included:3.8.3
    # cypress/included images have entrypoint set to "cypress"
    # which conflicts with GitLab CI wrapper shell script
    entrypoint: [""]
  script:
    - cypress run --project ./e2e

That fixed it for me. Thank you very much for the quick response!

[cypress-bot]

The code for this is done in cypress-io/cypress#6280, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[flotwig]

Released in 4.0.0.