fix(jest-runtime): fix node:-prefixed modules loading #11802

antongolub · 2021-08-31T12:36:19Z

Got trouble on loading node:-prefixed modules with Node.js 14 and jest v27.1.0:

https://github.com/antongolub/tsc-esm-fix/runs/3472541312?check_suite_focus=true  

 $ /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/.bin/jest -v
27.1.0

 FAIL src/test/ts/index.ts

  ● Test suite failed to run

    Cannot find module 'node:fs'
    Require stack:
    - /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/jest-runtime/build/index.js
    - /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/@jest/core/build/cli/index.js
    - /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/@jest/core/build/jest.js
    - /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/jest/node_modules/jest-cli/build/cli/index.js
    - /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/jest/node_modules/jest-cli/bin/jest.js
    - /home/runner/work/tsc-esm-fix/tsc-esm-fix/node_modules/jest/bin/jest.js

    Require stack:
      node_modules/jest-runtime/build/index.js
      node_modules/@jest/core/build/cli/index.js
      node_modules/@jest/core/build/jest.js
      node_modules/jest/node_modules/jest-cli/build/cli/index.js
      node_modules/jest/node_modules/jest-cli/bin/jest.js
      node_modules/jest/bin/jest.js
      
at Runtime._requireCoreModule (node_modules/jest-runtime/build/index.js:1561:12)
 at Object.<anonymous> (node_modules/globby/index.js:20:38)

The current impl trims node: if the runtime already supports it.

   private _requireCoreModule(moduleName: string, supportPrefix: boolean) {
  const moduleWithoutNodePrefix =
    supportPrefix && moduleName.startsWith('node:')
      ? moduleName.slice('node:'.length)
      : moduleName;

  if (moduleWithoutNodePrefix === 'process') {
    return this._environment.global.process;
  }

  if (moduleWithoutNodePrefix === 'module') {
    return this._getMockedNativeModule();
  }

  return require(moduleWithoutNodePrefix);
}

I suggest to invert this logic:

  private _requireCoreModule(moduleName: string, supportPrefix: boolean) {
    const moduleWithoutNodePrefix = moduleName.startsWith('node:')
      ? moduleName.slice('node:'.length)
      : moduleName;

    if (moduleWithoutNodePrefix === 'process') {
      return this._environment.global.process;
    }

    if (moduleWithoutNodePrefix === 'module') {
      return this._getMockedNativeModule();
    }

    return require(supportPrefix ? moduleName : moduleWithoutNodePrefix);
  }

codecov-commenter · 2021-08-31T12:55:09Z

Codecov Report

Merging #11802 (335f0a4) into master (98f10e6) will not change coverage.
The diff coverage is 75.00%.

@@           Coverage Diff           @@
##           master   #11802   +/-   ##
=======================================
  Coverage   69.04%   69.04%           
=======================================
  Files         312      312           
  Lines       16366    16366           
  Branches     4746     4747    +1     
=======================================
  Hits        11300    11300           
  Misses       5039     5039           
  Partials       27       27

Impacted Files	Coverage Δ
packages/jest-runtime/src/index.ts	`59.14% <75.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 98f10e6...335f0a4. Read the comment docs.

SimenB · 2021-08-31T13:11:31Z

Our current behavior matches Node itself, though.

$ node -e 'console.log(process.versions.node); require("node:fs")'
14.17.5
internal/modules/cjs/loader.js:892
  throw err;
  ^

Error: Cannot find module 'node:fs'
Require stack:
- /Users/simen/[eval]
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:889:15)
    at Function.Module._load (internal/modules/cjs/loader.js:745:27)
    at Module.require (internal/modules/cjs/loader.js:961:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at [eval]:1:37
    at Script.runInThisContext (vm.js:134:12)
    at Object.runInThisContext (vm.js:310:38)
    at internal/process/execution.js:81:19
    at [eval]-wrapper:6:22
    at evalScript (internal/process/execution.js:80:60) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/Users/simen/[eval]' ]
}

I don't think Jest should work when it would fail at runtime when you deploy it

antongolub · 2021-08-31T13:40:43Z

Agree. But the code that I'm trying to test should be loaded with import syntax (esm module), not require.

node -e 'console.log(process.versions.node); (async() => await import("node:fs"))()' 
14.17.1

SimenB · 2021-08-31T14:21:51Z

If you use ESM in your tests (https://jestjs.io/docs/ecmascript-modules) it should work already. Does it not?

antongolub · 2021-08-31T19:25:44Z

Unfortunately, I have TS which is transpiled with ts-jest. As I can see, It somehow produces require for all kinds of module loading.

SimenB · 2021-09-01T07:16:54Z

https://kulshekhar.github.io/ts-jest/docs/guides/esm-support

antongolub · 2021-09-01T07:48:47Z

No luck. "useESM" is applied to the project's own codebase, not its deps. Anyway, I'll do some more experiments.

SimenB · 2021-09-01T09:20:49Z

That's an issue with ts-jest, then, not Jest (more likely a configuration issue than a bug). Regardless of where, your code uses require at runtime and I do think it's correct behavior by Jest to respect how require works in Node as much as possible, so I'll close this.

antongolub · 2021-09-01T19:42:09Z

@SimenB,

Hold on. I've removed TS, ts-jest and transforms that could be confusing. Just a pure ESM stand. Take a look: https://github.com/antongolub/jest-issue-11802

antongolub · 2021-09-03T12:19:50Z

@SimenB,

I seem to have found the source of the problem. Here's the point where ESM flow breaks against the legacyrequire API.

// #289 node_modules/jest-resolve/build/resolver.js
        const resolvedModule =
          resolveNodeModule(module) || require.resolve(module);

Digging notes: https://github.com/antongolub/jest-issue-11802

tusbar · 2021-09-07T13:10:13Z

I have the same issue, without typescript as well, it seems to only fail when the node: import is in node_modules.

antongolub · 2021-09-07T13:38:56Z

Oh, thanks for the reply. I really thought that I have the most crooked hands in the world)
I checked again: the issue occurs in Node.js <=16 when you invoke require API for node:-prefixed module at any level of the project's dep tree.

So even

// index.js

import fs from 'node:fs'
console.log('success')

is working with a direct node index.js invocation, but is failing with the latest v27.1.0 Jest.

@SimenB,
Have you been able to reproduce the bug?

SimenB · 2021-09-07T15:00:40Z

@antongolub thanks for the minimal reproduction 👍 I've opened up a PR fixing it: #11817

SimenB · 2021-09-08T10:14:37Z

Published in https://github.com/facebook/jest/releases/tag/v27.1.1

github-actions · 2021-10-09T00:04:08Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

fix(jest-runtime): fix node:-prefixed modules loading

335f0a4

facebook-github-bot added the cla signed label Aug 31, 2021

SimenB closed this Sep 1, 2021

This comment has been minimized.

Sign in to view

SimenB mentioned this pull request Sep 7, 2021

fix(resolver): support node prefix from ESM #11817

Merged

github-actions bot locked as resolved and limited conversation to collaborators Oct 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(jest-runtime): fix node:-prefixed modules loading #11802

fix(jest-runtime): fix node:-prefixed modules loading #11802

antongolub commented Aug 31, 2021 •

edited

codecov-commenter commented Aug 31, 2021

SimenB commented Aug 31, 2021 •

edited

antongolub commented Aug 31, 2021 •

edited

SimenB commented Aug 31, 2021

antongolub commented Aug 31, 2021 •

edited

SimenB commented Sep 1, 2021

antongolub commented Sep 1, 2021 •

edited

SimenB commented Sep 1, 2021

antongolub commented Sep 1, 2021 •

edited

antongolub commented Sep 3, 2021 •

edited

tusbar commented Sep 7, 2021 •

edited

antongolub commented Sep 7, 2021

This comment has been minimized.

SimenB commented Sep 7, 2021

SimenB commented Sep 8, 2021

github-actions bot commented Oct 9, 2021

fix(jest-runtime): fix node:-prefixed modules loading #11802

fix(jest-runtime): fix node:-prefixed modules loading #11802

Conversation

antongolub commented Aug 31, 2021 • edited

codecov-commenter commented Aug 31, 2021

Codecov Report

SimenB commented Aug 31, 2021 • edited

antongolub commented Aug 31, 2021 • edited

SimenB commented Aug 31, 2021

antongolub commented Aug 31, 2021 • edited

SimenB commented Sep 1, 2021

antongolub commented Sep 1, 2021 • edited

SimenB commented Sep 1, 2021

antongolub commented Sep 1, 2021 • edited

antongolub commented Sep 3, 2021 • edited

tusbar commented Sep 7, 2021 • edited

antongolub commented Sep 7, 2021

This comment has been minimized.

SimenB commented Sep 7, 2021

SimenB commented Sep 8, 2021

github-actions bot commented Oct 9, 2021

antongolub commented Aug 31, 2021 •

edited

SimenB commented Aug 31, 2021 •

edited

antongolub commented Aug 31, 2021 •

edited

antongolub commented Aug 31, 2021 •

edited

antongolub commented Sep 1, 2021 •

edited

antongolub commented Sep 1, 2021 •

edited

antongolub commented Sep 3, 2021 •

edited

tusbar commented Sep 7, 2021 •

edited