Segmentation fault with import() instead of calling importModuleDynamically
#35889
Comments
Yes, Jest doesn't deal with file URLs, that's a bug that should be simple to fix. I can fix that right away. EDIT: jestjs/jest#10744 @nicolo-ribaudo "Subsystem" is EDIT2: For anyone looking into this, the |
|
@devsnek sorry to ping you, but do you have any idea of the top of your head for what we could be doing wrong in Jest? It's probably something that should be tweaked in Node regardless since it segfaults, but I'm also quite certain it's Jest doing something unexpected 🙂 |
|
If it helps, I created a minimal (almost, it still uses Jest) reproduction: https://github.com/nicolo-ribaudo/node-segfault-jest-dynamic-import Also @bmeck kindly told me that this is probably a known v8 issue, but I couldn't find it in the v8 bug tracker 😅 EDIT: I created a minimal reproduction without any dependency, but it's the first time I use the |
|
It is from the VM module dealing with https://bugs.chromium.org/p/v8/issues/detail?id=10284 , which means that these fake modules from source text alone don't get properly resolved and it segfaults |
targos
added
the
esm
|
I can reproduce this bug on Node |
|
v8 has landed a partial fix for this in that they check for host_defined_options in the cache now, but this causes a large amount of cache misses and extra memory usage in both chrome and node, so they're still working on it. |
|
To be honest, I am very upset with this problem, in fact it blocks the transition very much on ESM modules, for large projects it is very difficult to just take and translate the entire codebase to ESM, but small projects started to transition, and in order to keep versions up to day we need to use It's not just about updating dependencies. This actually blocks the ecosystem from transitioning smoothly. Also, old package versions stop receiving updates, which we can potentially lead to security problems. I really hope that the problem will be fixed in the near future. |
|
Wouldn't describe myself as "very upset" per se, but it is stopping us from transitioning too atm |
|
this is a critical issue for our organization as well. |
If I'm reading the issue correctly then it sounds like that change was reverted and things are still broken. And that bug can't be actively worked on at the moment because there's a blocker. |
|
Maybe be need to ping somebody from V8 here... |
|
Worth mentioning that it isn't only Jest that is affected too - our test framework which is built on top of mocha cannot run and furthermore am being unable to (practicably) upgrade to latest version of dependencies and get a completely rosy npm audit output when groovy people like e.g. @sindresorhus are taking everything to ESM... |
|
@joyeecheung this can be closed as well 🙂 |
|
Closing as #48510 has landed and should fix this. We can re-open if that turns out to be incorrect (hopefully not). |
Consistently running into a segfault coming from Node as a result of cosmiconfig attempting to run a dynamic import on the listed .js config file caused by: nodejs/node#35889 jestjs/jest#11438 Using cosmiconfigSync to workaround this which will fall back to using a synchronous require() call.
Fixes #118 Introduces `cosmiconfig` as requested in #118 in order to support loading the config from alternative file formats like: ``` .unimportedrc.js .unimportedrc.yml package.json > "unimported" key ``` I'm using `cosmiconfigSync` utilities instead of the async version, despite them being available since the `getConfig` is an async method. When we use the async equivalent, we hit a segfault in Node as a result of `cosmiconfig` trying to call a dynamic import on the file within a Jest context that doesn't allow it to. Patched in a recent version of Node, and once the test infra can require the latest version it should be fine to switch to the async version. See nodejs/node#35889 and jestjs/jest#11438 I haven't made any efforts to change the `update` function to write updates to the loaded files, as this would be difficult/impossible to update something like a .js or .yml (if using features like anchors) in a meaningful way. A few notes on the other changes included: 1. `cosmiconfig` pulls in a newer version of TypeScript which was incompatible with the version of `@types/node` we used, updated it 2. One of the tests produced invalid JSON to a config file, which failed silently before and passed the test, but now fails loudly when `cosmiconfig` tries to read and parse the JSON. Updated it to be valid to fulfill the spirit of the test. --------- Co-authored-by: Stephan Meijer <stephan.meijer@gmail.com>
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
Dynamic imports cause segmentation faults with Jest: nodejs/node#35889. We work around this by handling imports in IdentityProviderFactory differently when Jest is running. For unit tests we use a different tsconfig that transpiles dynamic imports differently, as those are also used in AppRunner.
and others
Linux nicolo-XPS-15-9570 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/LinuxvmWhat steps will reproduce the bug?
https://github.com/nicolo-ribaudo/babel/tree/node-segfault
I'm sorry but I can't create a smaller reproduction example(EDIT: #35889 (comment)): I managed to create a small test that reproduces the crash almost always, but it still far from being "self contained".make bootstrapto install dependencies and compile everythingnode --experimental-vm-modules ./node_modules/.bin/jest -i babel-core/test/segfaultto see the segfault (try 2-3 times, sometimes the first run doesn't fail).That test will run this file: I have placed a
debugger;statement so that you can--inspect-brkit. After debugger, if you move into theimport()call, it will crash.How often does it reproduce? Is there a required condition?
Almost 100%
What is the expected behavior?
import()should call Jest'simportModuleDynamicallyfunction.What do you see instead?
@SimenB tried debugging this segfault and extracted this stacktrace (babel/babel#12288 (comment)):
Additional information
file:///home/nicolo/Documenti/dev/babel/babel/packages/babel-core/test/fixtures/example.mjsdoesn't exist even if it does. However, this might be caused by Jest?node --experimental-vm-modules ./node_modules/.bin/jest -i babel-core/test/config-chainwhich is how I originally discovered this issue. You can stop right before crashing by adding adebugger;right before the compiled version (which will be generated inpackages/babel-core/lib/config/files/import.js) of thisimport()call.Similar bugs
import(), but I don't think that it is the same bug because that one only happens in the REPL.importModuleDynamicallybefore crashing, while in my case I don't think it is.The text was updated successfully, but these errors were encountered: