ERR_LOADER_CHAIN_INCOMPLETE in CI, works locally in a Dev Container

[brianjenkins94]

Current behavior

Stacktrace:

Error [ERR_LOADER_CHAIN_INCOMPLETE]: "file:///root/.cache/Cypress/10.3.0/Cypress/resources/app/node_modules/ts-node/esm/transpile-only.mjs 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return.
    at new NodeError (node:internal/errors:387:5)
    at ESMLoader.resolve (node:internal/modules/esm/loader:860:13)
    at async ESMLoader.getModuleJob (node:internal/modules/esm/loader:439:7)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:541:24)
    at async loadESM (node:internal/process/esm_loader:83:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12) {
  code: 'ERR_LOADER_CHAIN_INCOMPLETE'

I noticed this too:

libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
[1395:0714/145055.804094:ERROR:sandbox_linux.cc(377)] InitializeSandbox() called with multiple threads in process gpu-process.
[1395:0714/145055.807765:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

But otherwise didn't find any useful search results when searching for "ERR_LOADER_CHAIN_INCOMPLETE" so I'm going to file this so I can answer it when I figure it out :)

Desired behavior

Don't short circuit on resolve.

Test code to reproduce

Not sure yet... I'm going to start by trying v10.2.* because it only recently stopped working in CI, so maybe it broke with v10.2.*.

I am using this .gitlab-ci.yml:

{
  "image": "mcr.microsoft.com/playwright:latest",
  "before_script": [
    "# Node.js v18",
    "apt-get remove nodejs -y",
    "apt-get update",
    "apt-get install -y curl",
    "curl -fsSL https://deb.nodesource.com/setup_18.x | bash -",
    "apt-get install -y nodejs",
    "npm install",
    "# SSH",
    "eval $(ssh-agent -s)",
    "echo \"$SSH_PRIVATE_KEY\" | ssh-add -",
    "mkdir -p ~/.ssh",
    "chmod 700 ~/.ssh",
    "ssh-keyscan host.docker.internal >> ~/.ssh/known_hosts",
    "chmod 644 ~/.ssh/known_hosts"
  ],
  "stages": [
    "build",
    "confidence-check"
  ],
  "pages": {
    "stage": "build",
    "script": [
      "npm run cypress",
      "npm run cypress-static",
      "rm -rf public",
      "mv out public"
    ],
    "artifacts": {
      "paths": [
        "public"
      ]
    },
    "only": [
      "main"
    ]
  },
  "e2e": {
    "stage": "confidence-check",
    "script": [
      "npx cypress run --config baseUrl=$CI_PAGES_URL"
    ],
    "only": [
      "main"
    ]
  }
}

and here are the relevant scripts from my package.json, which are a bit gnarly:

{
  "scripts": {
    "cypress": "CODE=0 ; (npm start & { npx cypress run ; CODE=$? ; kill -INT $(jobs -p) ; }) ; (exit $CODE)",
    "cypress-static": "CODE=0 ; npm run build && (npm run serve & { npx cypress run --config baseUrl=http://localhost:8080/ ; CODE=$? ; kill -INT $(jobs -p) ; }) ; (exit $CODE)",
    "build": "next build && npm run postbuild && next export && npm run postexport",
    "postbuild": "node --experimental-specifier-resolution=node --loader=ts-node/esm scripts/postbuild.ts",
    "postexport": "node --experimental-specifier-resolution=node --loader=ts-node/esm scripts/postexport.ts",
    "serve": "npx http-server . -c-1",
    "start": "node --experimental-specifier-resolution=node --loader=ts-node/esm server.ts"
  }
}

and my cypress.config.ts, even though it's pretty plain:

import { defineConfig } from "cypress";

export default defineConfig({
	"e2e": {
		"baseUrl": "http://localhost:3000/",
		"setupNodeEvents": function(on, config) {
			// Implement node event listeners here
		}
	},
	"video": false,
	"viewportHeight": 1080,
	"viewportWidth": 1920
});

Cypress Version

v10.3.0

Other

No response

[neuronetio]

I think it may be related to TypeStrong/ts-node#1714

I have the same error after updating nodejs to 18.6.0

[cspotcode]

Can someone please create a ts-node issue to discuss? It's easier for me to keep track of those.

The short-circuit thing is potentially misleading, because often loaders should short-circuit, but node's loaders API demands that we signal the intent via a flag in the return value. This error happens when that intention is not signaled. It's complicated.

[philiparvidsson]

No idea on the issue internals or specifics but this just occurred in our system too. No changes (code or else) except Node.js upgrade.

[brianjenkins94]

Using DEBUG=cypress:*:

  cypress:scaffold-config:detect Detected cypress.config.ts - using TS +0ms
  cypress:lifecycle:ProjectConfigIpc fork child process { CHILD_PROCESS_FILE_PATH: '/root/.cache/Cypress/10.3.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/require_async_child.js', configProcessArgs: [ '--projectRoot', '/builds/project', '--file', '/builds/project/cypress.config.ts' ], childOptions: { stdio: 'pipe', cwd: '/builds/project', execPath: '/usr/bin/node' } } +0ms
  cypress:lifecycle:ProjectConfigIpc trigger the load of the file +10ms
  cypress:server:video using ffmpeg from /root/.cache/Cypress/10.3.0/Cypress/resources/app/node_modules/@ffmpeg-installer/linux-x64/ffmpeg +0ms

(node:1466) ExperimentalWarning: Custom ESM Loaders is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1466) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
Error [ERR_LOADER_CHAIN_INCOMPLETE]: "file:///root/.cache/Cypress/10.3.0/Cypress/resources/app/node_modules/ts-node/esm/transpile-only.mjs 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return.
    at new NodeError (node:internal/errors:387:5)
    at ESMLoader.resolve (node:internal/modules/esm/loader:860:13)
    at async ESMLoader.getModuleJob (node:internal/modules/esm/loader:439:7)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:541:24)
    at async loadESM (node:internal/process/esm_loader:83:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12) {
  code: 'ERR_LOADER_CHAIN_INCOMPLETE'
}

  cypress:server:browsers:utils found browsers { browsers: [] } +362ms
  cypress:server:browsers:utils adding Electron browser { name: 'electron', channel: 'stable', family: 'chromium', displayName: 'Electron', version: '100.0.4896.160', path: '', majorVersion: 100, info: 'Electron is the default browser that comes with Cypress. This is the default browser that runs in headless mode. Selecting this browser is useful when debugging. The version number indicates the underlying Chromium version that Electron uses.' } +0ms

[brianjenkins94]

Sounds like Cypress needs to version-bump ts-node:

TypeStrong/ts-node#1839 (comment)

[chrisbreiding]

I was able to get the ERR_LOADER_CHAIN_INCOMPLETE error to show up with the following: https://github.com/chrisbreiding/cypress-test-tiny/tree/issue-22795

However, it only logs the error and continues with the test. Is that behavior you're seeing or is your test run failing?

I also found that having ts-node in the project's own package.json was necessary to get the error to show up. Otherwise, it would be a [ERR_MODULE_NOT_FOUND]: Cannot find package 'ts-node' error. Upgrading the project's ts-node to 10.8.0 stopped the error from displaying. This leads me to believe this might not be an issue with the ts-node bundled with Cypress.

What displays if you run npm ls ts-node?

[brianjenkins94]

Mine comes to a halt and doesn't run any further tests but doesn't exit.

I'll try your branch.

[nevenduranec]

Clean install, ^10.3.0
node 18.6.0
no local ts-node

> cypress open

objc[22811]: Class WebSwapCGLLayer is implemented in both /System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebCore.framework/Versions/A/Frameworks/libANGLE-shared.dylib (0x23c063808) and /Users/user/Library/Caches/Cypress/10.3.0/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libGLESv2.dylib (0x10999d3c8). One of the two will be used. Which one is undefined.
GraphQL server is running at http://localhost:59662/__launchpad/graphql
(node:22813) ExperimentalWarning: Custom ESM Loaders is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:22813) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
Error [ERR_LOADER_CHAIN_INCOMPLETE]: "file:///Users/user/Library/Caches/Cypress/10.3.0/Cypress.app/Contents/Resources/app/node_modules/ts-node/esm/transpile-only.mjs 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return.
    at new NodeError (node:internal/errors:387:5)
    at ESMLoader.resolve (node:internal/modules/esm/loader:860:13)
    at async ESMLoader.getModuleJob (node:internal/modules/esm/loader:439:7)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:541:24)
    at async loadESM (node:internal/process/esm_loader:83:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12) {
  code: 'ERR_LOADER_CHAIN_INCOMPLETE'
}

[mjhenkes]

@brianjenkins94 or @nevenduranec, Would one of you be able to provide a reproduction repo? @chrisbreiding 's attempt might be a good place to start.

[brianjenkins94]

Here is a minimal reproducible sample using Cypress: ts-node-1839.zip

Using VSCode to open the above as a dev container, reproduce with:

$ npm install
$ npx cypress run

[mjhenkes]

@brianjenkins94, when running the provided steps i get this result

➜  ts-node-1839 npx cypress run
(node:5742) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
Cypress could not verify that this server is running:

  > http://localhost:3000

We are verifying this server because it has been configured as your baseUrl.

Cypress automatically waits until your server is accessible before running tests.

We will try connecting to it 3 more times...
We will try connecting to it 2 more times...
We will try connecting to it 1 more time...

Cypress failed to verify that your server is running.

Please start this server and then run Cypress again.

So i tried the 'start' script but that errors with this:

➜  ts-node-1839 npm run start

> ts-node-1839@0.1.0 start
> node --experimental-specifier-resolution=node --loader=ts-node/esm server.ts

(node:8823) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:352
    throw new ERR_MODULE_NOT_FOUND(
          ^
CustomError: Cannot find module '/Users/matthenkes/Source/ts-node-1839/server.ts' imported from /Users/matthenkes/Source/ts-node-1839/
    at finalizeResolution (/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:352:11)
    at moduleResolve (/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:801:10)
    at Object.defaultResolve (/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:912:11)
    at /Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/src/esm.ts:218:35
    at entrypointFallback (/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/src/esm.ts:168:34)
    at /Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/src/esm.ts:217:14
    at addShortCircuitFlag (/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/src/esm.ts:409:21)
    at resolve (/Users/matthenkes/Source/ts-node-1839/node_modules/ts-node/src/esm.ts:197:12)
    at ESMLoader.resolve (node:internal/modules/esm/loader:422:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:222:40)

I think the test project may be missing the server.ts file.

[brianjenkins94]

I'm seeing the ERR_LOADER_CHAIN_INCOMPLETE error with npx cypress run so I don't even get to the part where it tries to load server.ts.

I think you're probably not on a new enough Node:

$ node -v
v18.6.0
$ npm -v
8.14.0

[mjhenkes]

@brianjenkins94, I get the same results with the never version of node. The test project seems to want to run a server.ts file that isn't included in the project provided.

[brianjenkins94]

Windows:

Linux (Dev Container):

[mjhenkes]

I took another look and was able to recreate the error. @brianjenkins94 thanks for bearing with me. This appears to work on node v18.2.0 but does not work on node 18.6.0

[cspotcode]

18.6.0 introduced loader chaining, that error code, and a new requirement which loaders must implement in some situations to avoid it. So if any loaders are being used, and if they're out-of-date, they may trigger that error.

[isgj]

I'm getting the same error stack with:
Cypress: v10.6.0
Node: 16.17.0
no local ts-node.

To note: I've "type": "module" in my package.json
Removing "type": "module" will make cypress work (but I'd prefer to keep type to module).

[AriPerkkio]

@isgj Same issue here. Cypress v10 is broken on current Node LTS.

Node 16.17.0 also introduced the same Experimental ESM Loader Hooks API as 18.6.0.
Reverting back to previous LTS 16.15.1 works for now. 🎉

[lmiller1990]

I will look into this, most minimal repo is

import { defineConfig } from "cypress";

export default defineConfig({
  e2e: {
    supportFile: false,
  },
});

{
  "type": "module",
  "dependencies": {
    "cypress": "^10.7.0",
    "typescript": "^4.8.2"
  }
}

Basically any combination of

1. type: module
2. cypress.config.ts (must be .ts)
3. node 16.17, node 18.6, etc.

[lmiller1990]

I updated ts-node. Node.js 16.17 is working fine now.

Edit: Node.js 18.6 also seems okay. I'll leave the below debugging info for future reference, we will need to figure out why our tests are doing something different to production for Node.js 18.6.

## Node 18.6 - new error

[786297:0831/162827.757817:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.
(node:786314) ExperimentalWarning: Custom ESM Loaders is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:786314) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
asdfsaf v18.6.0

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    10.7.0                                                                             │
  │ Browser:    Chrome 104 (headless)                                                              │
  │ Specs:      1 found (app.cy.js)                                                                │
  │ Searched:   cypress/e2e/app.cy.js                                                              │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  app.cy.js                                                                       (1 of 1)

Your configFile threw an error from: cypress.config.js

We stopped running your tests because your config file crashed.

Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:71:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (/home/lachlan/code/work/dev/node_modules/webpack/lib/util/createHash.js:135:53)
    at NormalModule._initBuildHash (/home/lachlan/code/work/dev/node_modules/webpack/lib/NormalModule.js:417:16)
    at handleParseError (/home/lachlan/code/work/dev/node_modules/webpack/lib/NormalModule.js:471:10)
    at /home/lachlan/code/work/dev/node_modules/webpack/lib/NormalModule.js:503:5
    at /home/lachlan/code/work/dev/node_modules/webpack/lib/NormalModule.js:358:12
    at /home/lachlan/code/work/dev/node_modules/loader-runner/lib/LoaderRunner.js:373:3
    at iterateNormalLoaders (/home/lachlan/code/work/dev/node_modules/loader-runner/lib/LoaderRunner.js:214:10)
    at iterateNormalLoaders (/home/lachlan/code/work/dev/node_modules/loader-runner/lib/LoaderRunner.js:221:10)
    at /home/lachlan/code/work/dev/node_modules/loader-runner/lib/LoaderRunner.js:236:3
    at context.callback (/home/lachlan/code/work/dev/node_modules/loader-runner/lib/LoaderRunner.js:111:13)
    at /home/lachlan/code/work/dev/npm/webpack-batteries-included-preprocessor/node_modules/babel-loader/lib/index.js:59:71

  (Results)

The above problem is actually related to webpack 4 - see webpack/webpack#14532 - which we use for our default preprocessor.

The only work-around is to use a newer webpack - NODE_OPTIONS=--openssl-legacy-provider does not work with Node.js 18.6, it says "--openssl-legacy-provided is not allowed", as described here. It's possible to work around this:

1. use webpack 5 (requires user to figure out the problem is webpack 4, and install webpack 4)
2. do not use a pre-preprocessor at all (not ideal, also requires user intervention)

We should handle this error specifically, but I think we need to revisit #19555 and try to move to webpack 5.

Hmm this seems to only happen in our internal system tests - could be a red herring. Perhaps the ts-node upgrade is enough.

[jsolly]

Looking forward to this being fixed. I am running into the same error message on my local dev machine. I've never used Cypress before, so I am just trying to get started as a new user.

Repro steps

pnpm install cypress --save-dev
pnpx cypress open

The app begins to load

Error in console. App hangs and never loads.

🍁johnsolly:22-09-03:GeoAsteroids $ pnpx cypress open
.../Library/pnpm/store/v3/tmp/dlx-71863  | +166 +++++++++++++++++
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /Users/johnsolly/Library/pnpm/store/v3
  Virtual store is at:             ../../../Library/pnpm/store/v3/tmp/dlx-71863/node_modules/.pnpm
.../Library/pnpm/store/v3/tmp/dlx-71863  | Progress: resolved 166, reused 166, downloaded 0, added 166, done
(node:71915) ExperimentalWarning: Custom ESM Loaders is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:71915) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
Error [ERR_LOADER_CHAIN_INCOMPLETE]: "file:///Users/johnsolly/Library/Caches/Cypress/10.7.0/Cypress.app/Contents/Resources/app/node_modules/ts-node/esm/transpile-only.mjs 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return.
    at new NodeError (node:internal/errors:387:5)
    at ESMLoader.resolve (node:internal/modules/esm/loader:852:13)
    at async ESMLoader.getModuleJob (node:internal/modules/esm/loader:431:7)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:533:24)
    at async loadESM (node:internal/process/esm_loader:91:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12) {
  code: 'ERR_LOADER_CHAIN_INCOMPLETE'
}

Environment

• Using pnpm package manager
• MacOS Monterey 12.5.1 (Intel Mac Mini)
• Node v16.17.0

[lmiller1990]

@jsolly #23637 should fix this. In the meantime you could install Node.js 16.16, and that should work, too. Node.js 16.17 had a breaking change (surprise minor version breaking change).

Edit: see below - worth nothing that my above comment isn't accurate - while this did catch us off guard, es modules in Node.js is still experimental, so there is not commitment to server.

[cspotcode]

minor version breaking change

It was a breaking change in an experimental feature which does not follow semver, right?
https://nodejs.org/dist/latest-v18.x/docs/api/documentation.html#stability-index

[lmiller1990]

Right, yes, my bad -- I've been using this feature for so long I'd forgotten it's still experimental. Thanks for pointing that out. I've updated my comment to clarify (but left original one there, too, for context).

Either way, @cspotcode's great ts-node library fixed the issue almost immediately and now we've updated to use the latest version, everything should work just fine here, too.

Thanks for making ts-node - I (personally, and at Cypress) use it every day and it works great 💯

[YoLoADR]

npm -v

I confirm it's a node version issue. I'm using now nvm with a node@16.15.1 an it works well now

[SchulteMarkus]

@YoLoADR This is solved, update to https://github.com/cypress-io/cypress/releases/tag/v10.9.0

[isgj]

@SchulteMarkus I get this error with 10.9

[brianjenkins94]

That's not the error this issue is about.