Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support WebAssembly (Wasm) imports in ESM modules #13505

Merged
merged 27 commits into from Nov 6, 2022
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
95ef976
Add failing test
kachkaev Oct 24, 2022
645db88
Update `jest-runtime` to fix tests
kachkaev Oct 24, 2022
9985179
Support data:application/wasm imports
kachkaev Oct 27, 2022
5d73f9a
Run tests twice (with and without --experimental-wasm-modules)
kachkaev Oct 27, 2022
bc6d38c
Merge remote-tracking branch 'u/main' into native-esm-wasm
kachkaev Oct 27, 2022
734bbc5
Cover more rows
kachkaev Oct 27, 2022
612d480
Add CHANGELOG entry
kachkaev Oct 27, 2022
dcff11e
Simplify tests
kachkaev Oct 27, 2022
e524e40
Improve `path.endsWith`
kachkaev Oct 27, 2022
08c8906
Delete unused file
kachkaev Oct 27, 2022
66c8312
Make changelog phrase more explicit
kachkaev Oct 27, 2022
6586a24
Fix Wasm casing
kachkaev Oct 27, 2022
01f8d78
Update CHANGELOG.md
kachkaev Oct 30, 2022
27d73b6
Mention Wasm in docs
kachkaev Oct 30, 2022
2aea674
Merge remote-tracking branch 'origin/main' into native-esm-wasm
kachkaev Oct 30, 2022
79e0f40
Use Wasm file from mdn examples
kachkaev Oct 30, 2022
0d4945e
Implement `readFileBuffer`
kachkaev Oct 30, 2022
26db3fd
Call loadEsmModule instead of linkAndEvaluateModule in _importWasmModule
kachkaev Oct 30, 2022
1e6d75b
Update e2e/native-esm/__tests__/native-esm-wasm.test.js
kachkaev Oct 31, 2022
d4e9b73
Merge remote-tracking branch 'origin/main' into native-esm-wasm
kachkaev Nov 1, 2022
ad32edd
Add another test case for dynamic import
kachkaev Nov 1, 2022
397bcd1
Extract `isWasm` function
kachkaev Nov 1, 2022
976ef61
Merge remote-tracking branch 'origin/main' into native-esm-wasm
kachkaev Nov 6, 2022
66a2312
Revert "Implement `readFileBuffer`"
SimenB Nov 6, 2022
7188086
separate buffer cache
SimenB Nov 6, 2022
26e7022
tweak
SimenB Nov 6, 2022
607fc0f
clear new cache as well
SimenB Nov 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Expand Up @@ -2,6 +2,8 @@

### Features

- `[jest-runtime]` Support WebAssembly (Wasm) imports in ESM modules ([#13505](https://github.com/facebook/jest/pull/13505))

### Fixes

- `[jest-mock]` Treat cjs modules as objects so they can be mocked ([#13513](https://github.com/facebook/jest/pull/13513))
Expand Down
2 changes: 1 addition & 1 deletion docs/CodeTransformation.md
Expand Up @@ -40,7 +40,7 @@ interface TransformOptions<TransformerConfig = unknown> {
supportsTopLevelAwait: boolean;
instrument: boolean;
/** Cached file system which is used by `jest-runtime` to improve performance. */
cacheFS: Map<string, string>;
cacheFS: Map<string, Buffer | string>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should change this type.

I can tinker with this 👍

/** Jest configuration of currently running project. */
config: ProjectConfig;
/** Stringified version of the `config` - useful in cache busting. */
Expand Down
2 changes: 2 additions & 0 deletions docs/ECMAScriptModules.md
Expand Up @@ -22,6 +22,8 @@ With the warnings out of the way, this is how you activate ESM support in your t

If you use Yarn, you can use `yarn node --experimental-vm-modules $(yarn bin jest)`. This command will also work if you use [Yarn Plug'n'Play](https://yarnpkg.com/features/pnp).

If your codebase includes ESM imports from `*.wasm` files, you do _not_ need to pass `--experimental-wasm-modules` to `node`. Current implementation of WebAssembly imports in Jest relies on experimental VM modules, however, this may change in the future.

1. Beyond that, we attempt to follow `node`'s logic for activating "ESM mode" (such as looking at `type` in `package.json` or `.mjs` files), see [their docs](https://nodejs.org/api/esm.html#esm_enabling) for details.
1. If you want to treat other file extensions (such as `.jsx` or `.ts`) as ESM, please use the [`extensionsToTreatAsEsm` option](Configuration.md#extensionstotreatasesm-arraystring).

Expand Down
10 changes: 9 additions & 1 deletion e2e/__tests__/__snapshots__/nativeEsm.test.ts.snap
Expand Up @@ -8,9 +8,17 @@ Time: <<REPLACED>>
Ran all test suites matching /native-esm-deep-cjs-reexport.test.js/i."
`;

exports[`runs WebAssembly (Wasm) test with native ESM 1`] = `
"Test Suites: 1 passed, 1 total
Tests: 5 passed, 5 total
Snapshots: 0 total
Time: <<REPLACED>>
Ran all test suites matching /native-esm-wasm.test.js/i."
`;

exports[`runs test with native ESM 1`] = `
"Test Suites: 1 passed, 1 total
Tests: 34 passed, 34 total
Tests: 33 passed, 33 total
Snapshots: 0 total
Time: <<REPLACED>>
Ran all test suites matching /native-esm.test.js/i."
Expand Down
12 changes: 12 additions & 0 deletions e2e/__tests__/nativeEsm.test.ts
Expand Up @@ -67,3 +67,15 @@ onNodeVersions('>=16.9.0', () => {
expect(exitCode).toBe(0);
});
});

test('runs WebAssembly (Wasm) test with native ESM', () => {
const {exitCode, stderr, stdout} = runJest(DIR, ['native-esm-wasm.test.js'], {
nodeOptions: '--experimental-vm-modules --no-warnings',
});

const {summary} = extractSummary(stderr);

expect(summary).toMatchSnapshot();
expect(stdout).toBe('');
expect(exitCode).toBe(0);
});
58 changes: 58 additions & 0 deletions e2e/native-esm/__tests__/native-esm-wasm.test.js
@@ -0,0 +1,58 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

// the point here is that it's the node core module
// eslint-disable-next-line no-restricted-imports
import {readFileSync} from 'fs';
// file origin: https://github.com/mdn/webassembly-examples/blob/2f2163287f86fe29deb162335bccca7d5d95ca4f/understanding-text-format/add.wasm
// source code: https://github.com/mdn/webassembly-examples/blob/2f2163287f86fe29deb162335bccca7d5d95ca4f/understanding-text-format/add.was
import {add} from '../add.wasm';

const wasmFileBuffer = readFileSync('add.wasm');

test('supports native wasm imports', () => {
expect(add(1, 2)).toBe(3);

// because arguments are i32 (signed), fractional part is truncated
expect(add(0.99, 1.01)).toBe(1);

// because return value is i32 (signed), (2^31 - 1) + 1 overflows and becomes -2^31
expect(add(Math.pow(2, 31) - 1, 1)).toBe(-Math.pow(2, 31));

// invalid or missing arguments are treated as 0
expect(add('hello', 'world')).toBe(0);
expect(add()).toBe(0);
expect(add(null)).toBe(0);
expect(add({}, [])).toBe(0);

// redundant arguments are silently ignored
expect(add(1, 2, 3)).toBe(3);
});

test('supports dynamic wasm imports', async () => {
const {add: dynamicAdd} = await import('../add.wasm');
expect(dynamicAdd(1, 2)).toBe(3);
});

test('supports imports from "data:application/wasm" URI with base64 encoding', async () => {
const importedWasmModule = await import(
`data:application/wasm;base64,${wasmFileBuffer.toString('base64')}`
);
expect(importedWasmModule.add(0, 42)).toBe(42);
});

test('imports from "data:application/wasm" URI without explicit encoding fail', async () => {
await expect(() =>
import(`data:application/wasm,${wasmFileBuffer.toString('base64')}`),
).rejects.toThrow('Missing data URI encoding');
});

test('imports from "data:application/wasm" URI with invalid encoding fail', async () => {
await expect(() =>
import('data:application/wasm;charset=utf-8,oops'),
).rejects.toThrow('Invalid data URI encoding: charset=utf-8');
});
6 changes: 0 additions & 6 deletions e2e/native-esm/__tests__/native-esm.test.js
Expand Up @@ -255,12 +255,6 @@ test('imports from "data:text/javascript" URI with invalid data fail', async ()
).rejects.toThrow("Unexpected token '.'");
});

test('imports from "data:application/wasm" URI not supported', async () => {
await expect(() =>
import('data:application/wasm,96cafe00babe'),
).rejects.toThrow('WASM is currently not supported');
});

test('supports imports from "data:application/json" URI', async () => {
const data = await import('data:application/json,{"foo": "bar"}');
expect(data.default).toEqual({foo: 'bar'});
Expand Down
Binary file added e2e/native-esm/add.wasm
Binary file not shown.
8 changes: 4 additions & 4 deletions packages/babel-jest/src/__tests__/index.ts
Expand Up @@ -52,7 +52,7 @@ test('Returns source string with inline maps when no transformOptions is passed'
sourceString,
'dummy_path.js',
{
cacheFS: new Map<string, string>(),
cacheFS: new Map<string, Buffer | string>(),
config: makeProjectConfig(),
configString: JSON.stringify(makeProjectConfig()),
instrument: false,
Expand All @@ -76,7 +76,7 @@ test('Returns source string with inline maps when no transformOptions is passed
sourceString,
'dummy_path.js',
{
cacheFS: new Map<string, string>(),
cacheFS: new Map<string, Buffer | string>(),
config: makeProjectConfig(),
configString: JSON.stringify(makeProjectConfig()),
instrument: false,
Expand Down Expand Up @@ -141,7 +141,7 @@ describe('caller option correctly merges from defaults and options', () => {
],
])('%j -> %j', (input, output) => {
defaultBabelJestTransformer.process(sourceString, 'dummy_path.js', {
cacheFS: new Map<string, string>(),
cacheFS: new Map<string, Buffer | string>(),
config: makeProjectConfig(),
configString: JSON.stringify(makeProjectConfig()),
instrument: false,
Expand All @@ -166,7 +166,7 @@ describe('caller option correctly merges from defaults and options', () => {
test('can pass null to createTransformer', () => {
const transformer = createTransformer();
transformer.process(sourceString, 'dummy_path.js', {
cacheFS: new Map<string, string>(),
cacheFS: new Map<string, Buffer | string>(),
config: makeProjectConfig(),
configString: JSON.stringify(makeProjectConfig()),
instrument: false,
Expand Down
2 changes: 1 addition & 1 deletion packages/jest-repl/src/cli/repl.ts
Expand Up @@ -33,7 +33,7 @@ const evalCommand: repl.REPLEval = (
cmd,
jestGlobalConfig.replname ?? 'jest.js',
{
cacheFS: new Map<string, string>(),
cacheFS: new Map<string, Buffer | string>(),
config: jestProjectConfig,
configString: JSON.stringify(jestProjectConfig),
instrument: false,
Expand Down