[Bug]: esbuild-bundled script crashes with ENOENT (related to `vm2`?) #10352

karlhorky · 2023-06-08T11:58:48Z

Bug expectation

I expected to be able to bundle my Puppeteer script with esbuild

My esbuild-bundled Puppeteer script failed (see full error output below) because of the transitive dependency on vm2 (via puppeteer-core -> proxy-agent -> pac-proxy-agent -> pac-resolver -> degenerator -> vm2)

$ node abc.mjs
node:fs:601
  handleErrorFromBinding(ctx);
  ^

Error: ENOENT: no such file or directory, open '/home/runner/work/project/project/bridge.js'
    at Object.openSync (node:fs:601:3)
    at Object.readFileSync (node:fs:469:35)
    at ../../node_modules/@puppeteer/browsers/node_modules/vm2/lib/vm.js (file:///home/runner/work/project/project/abc.mjs:52297:66)
    at __require2 (file:///home/runner/work/project/project/abc.mjs:20:50)
    at ../../node_modules/@puppeteer/browsers/node_modules/vm2/lib/main.js (file:///home/runner/work/project/project/abc.mjs:54169:9)
    at __require2 (file:///home/runner/work/project/project/abc.mjs:20:50)
    at ../../node_modules/@puppeteer/browsers/node_modules/vm2/index.js (file:///home/runner/work/project/project/abc.mjs:54197:22)
    at __require2 (file:///home/runner/work/project/project/abc.mjs:20:50)
    at ../../node_modules/@puppeteer/browsers/node_modules/degenerator/dist/index.js (file:///home/runner/work/project/project/abc.mjs:54211:17)
    at __require2 (file:///home/runner/work/project/project/abc.mjs:20:50) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/home/runner/work/project/project/bridge.js'
}

Node.js v18.16.0

It appears to be related to this current problem with bundling vm2:

Work in a bundle patriksimek/vm2#528

This has also been reported in the repos for proxy-agent:

Seems like the first version that this happens with is puppeteer-core@20.4.0, which upgraded @puppeteer/browsers from 1.3.0 to 1.4.0, which added proxy-agent.

Bug behavior

Flaky
PDF

Minimal, reproducible example

Bundle any Puppeteer script such as the simple script below via esbuild like this:

$ esbuild index.ts --bundle --outfile=abc.mjs --platform=node --target=node18 --format=esm --banner:js=\"import { createRequire as createRequire99 } from 'module'; import { dirname as dirname99 } from 'node:path'; import { fileURLToPath as fileURLToPath99 } from 'node:url'; const __filename = fileURLToPath99(import.meta.url); const __dirname = dirname99(__filename); const require = createRequire99(import.meta.url);\"

index.ts

import puppeteer from 'puppeteer';

function wait(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

export async function puppeteerWorkflow(urlToTest: string) {
  const browser = await puppeteer.launch({
    ...(process.platform === 'linux'
      ? {
          executablePath: '/usr/bin/google-chrome-stable',
        }
      : {
          // If we're not on Linux, then maybe we're in development,
          // where we don't want a headless browser (we want to see what
          // is going on)
          headless: false,
        }),
  });
  const page = await browser.newPage();

  await page.goto(urlToTest);
  await wait(5000);
  await page.close();
  await browser.close();
}

puppeteerWorkflow('https://example.com');

Error string

ENOENT: no such file or directory, open '/home/runner/work/project/project/bridge.js'

Puppeteer configuration

No response

Puppeteer version

20.5.0

Node version

18.16.0

Package manager

pnpm

Package manager version

8.6.1

Operating system

macOS

The text was updated successfully, but these errors were encountered:

github-actions · 2023-06-08T12:01:46Z

This issue was not reproducible. Please check that your example runs locally and the following:

Ensure the script does not rely on dependencies outside of puppeteer and puppeteer-core.

Ensure the error string is just the error message.

Bad:

Error: something went wrong
  at Object.<anonymous> (/Users/username/repository/script.js:2:1)
  at Module._compile (node:internal/modules/cjs/loader:1159:14)
  at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
  at Module.load (node:internal/modules/cjs/loader:1037:32)
  at Module._load (node:internal/modules/cjs/loader:878:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
  at node:internal/main/run_main_module:23:47

Good: Error: something went wrong.

Ensure your configuration file (if applicable) is valid.
If the issue is flaky (does not reproduce all the time), make sure 'Flaky' is checked.
If the issue is not expected to error, make sure to write 'no error'.

Once the above checks are satisfied, please edit your issue with the changes and we will
try to reproduce the bug again.

Analyzer run

cnlewis3 · 2023-06-08T21:58:37Z

Having this same issue when trying to deploy to our AWS Lambda. Worked fine with previous versions, but getting a ENOENT: no such file or directory, open '/var/task/bridge.js which we have also narrowed down to vm2, which is only in the dependency tree through puppeteer-core for us.

Lightning00Blade · 2023-06-09T08:41:16Z

Hey I tried to reproduce but I keep getting this error:

Error: Dynamic require of "stream" is not supported

Would it be possible to provide a repo with the issue you are seeing?

karlhorky · 2023-06-09T09:14:12Z

To be clear, you're seeing the following error?

$ node abc.mjs
file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:13
  throw new Error('Dynamic require of "' + x + '" is not supported');
        ^

Error: Dynamic require of "stream" is not supported
    at file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:13:9
    at node_modules/.pnpm/ws@8.13.0/node_modules/ws/lib/stream.js (file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:11585:22)
    at __require2 (file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:19:50)
    at node_modules/.pnpm/ws@8.13.0/node_modules/ws/wrapper.mjs (file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:15071:29)
    at __init (file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:16:56)
    at node_modules/.pnpm/puppeteer-core@20.4.0_typescript@5.1.3/node_modules/puppeteer-core/lib/esm/puppeteer/common/NodeWebSocketTransport.js (file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:15096:5)
    at __init (file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:16:56)
    at file:///Users/k/p/puppeteer-esbuild-vm2/abc.mjs:79315:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

I'm working on making a reproduction repo that shows the no such file or directory error.

karlhorky · 2023-06-09T09:25:50Z

Sorry, this error was actually related to globals not available in esbuild by default, I forgot to copy in the banner that is required.

Here's a new reproduction (run pnpm build and node abc.mjs) - I've also updated the description above:

https://github.com/karlhorky/puppeteer-esbuild-vm2-reproduction

OrKoN · 2023-06-12T07:50:27Z

As you pointed out the problem is in the vm2 module as it requires __dirname. For example, in Webpack it means __dirname: true needs to be set. I am not sure if the corresponding feature is available in esbuild but you can probably safely exclude the module from the bundler as it is only required by the installation.

karlhorky · 2023-06-12T08:46:51Z

__dirname is not the issue here, that is already fixed using the --banner option:

https://github.com/karlhorky/puppeteer-esbuild-vm2-reproduction/blob/99d9478af4896c735eba19e1faad3fbba397927c/package.json#L6

karlhorky · 2023-06-12T08:49:20Z

The issue is that vm2 expects that there are extra files which it can fs.readFileSync at runtime:

Error: ENOENT: no such file or directory, open '/home/runner/work/project/project/bridge.js'
    at Object.openSync (node:fs:601:3)
    at Object.readFileSync (node:fs:469:35)

OrKoN · 2023-06-12T08:49:47Z

@karlhorky I believe it is the issue https://github.com/patriksimek/vm2/blob/1663f231ec02db473ed5b743e3b91b8a2ffc7982/lib/vm.js#L145 the __dirname in the esbuild-produced bundle appears to be incorrect

OrKoN · 2023-06-12T08:51:50Z

The issue is that vm2 expects that there are extra files which it can fs.readFileSync at runtime:

yeah it expects the node_modules to be around and the __dirname to point to the vm2's lib folder.

karlhorky · 2023-06-12T08:52:10Z

The way that we're bundling + deploying, we do not have a node_modules folder anymore which contains bridge.js (similar approach to a Single Executable Application, except we still use Node.js)

So anyone taking a similar approach with their bundling will also have this issue

karlhorky · 2023-06-12T08:55:05Z

Workaround 1

A possible (ugly) workaround would be to require all of your users who want to bundle to copy the necessary files out from Puppeteer's transitive dependency vm2, similar to the approach described here:

1. add post build in NPM script
"build": "webpack --config ./webpack.prod.config.js && node prepare-package.js postBuild",

2. copy files from node_modules to output directory after build in prepare-package.js
const postBuild = async () => {
  await cpvm2("bridge.js");
  await cpvm2("events.js");
  await cpvm2("setup-sandbox.js");
  await cpvm2("setup-node-sandbox.js");
};

3. add them in package.json files property to bundle them

karlhorky · 2023-06-12T08:55:37Z

But maybe Puppeteer could do something about this so that bundling is zero-config 🤞

OrKoN · 2023-06-12T08:55:49Z

@karlhorky you can probably work around it by specifying --external:vm2 as you are unlikely to need those files at runtime.

karlhorky · 2023-06-12T09:11:50Z

Workaround 2

Confirmed, using the esbuild --external flag with --external:vm2 also works for us:

"build": "esbuild index.ts --bundle --external:vm2 --outfile=abc.mjs --platform=node --target=node18 --format=esm --banner:js=\"import { createRequire as createRequire99 } from 'module'; import { dirname as dirname99 } from 'node:path'; import { fileURLToPath as fileURLToPath99 } from 'node:url'; const __filename = fileURLToPath99(import.meta.url); const __dirname = dirname99(__filename); const require = createRequire99(import.meta.url);\""

Description of the --external flag:

You can mark a file or a package as external to exclude it from your build. Instead of being bundled, the import will be preserved (using require for the iife and cjs formats and using import for the esm format) and will be evaluated at run time instead.

I guess this may not work for everyone, but represents a simpler workaround for the ENOENT: no such file or directory ... bridge.js errors with bundling Puppeteer with esbuild

tranlehaiquan · 2023-07-03T04:55:44Z

I add external and got this error, I'm using puppeteer-core 20.7.4

gabrielo91 · 2023-07-04T04:35:06Z

Having the exact same issue, we are using node v16.17.0 and puppeteer 20.7.4, we run and API which calls the function that runs const browser = await puppeteer.launch();, however we get the exact same error: Error: ENOENT: foo/dist/apps/api/bridge.js. We run the project using nx, just like this:

npx nx run api:serve

gabrielo91 · 2023-07-05T21:20:16Z

How we solve was changing build section in project.json, just removed the "externalDependencies": "none" option

PluviaCon · 2023-07-12T10:16:07Z

Workaround 2

Confirmed, using the esbuild --external flag with --external:vm2 also works for us:
"build": "esbuild index.ts --bundle --external:vm2 --outfile=abc.mjs --platform=node --target=node18 --format=esm --banner:js=\"import { createRequire as createRequire99 } from 'module'; import { dirname as dirname99 } from 'node:path'; import { fileURLToPath as fileURLToPath99 } from 'node:url'; const __filename = fileURLToPath99(import.meta.url); const __dirname = dirname99(__filename); const require = createRequire99(import.meta.url);\""
Description of the --external flag:

You can mark a file or a package as external to exclude it from your build. Instead of being bundled, the import will be preserved (using require for the iife and cjs formats and using import for the esm format) and will be evaluated at run time instead.

I guess this may not work for everyone, but represents a simpler workaround for the ENOENT: no such file or directory ... bridge.js errors with bundling Puppeteer with esbuild

I've also encountered the same issue and have tried both methods. It seems that this problem occurs in versions above 20.0. Temporarily switching to version 19.10.0 seems to work for now.

edw1882 · 2023-07-12T10:23:50Z

Workaround 1

A possible (ugly) workaround would be to require all of your users who want to bundle to copy the necessary files out from Puppeteer's transitive dependency vm2, similar to the approach described here:
1. add post build in NPM script
"build": "webpack --config ./webpack.prod.config.js && node prepare-package.js postBuild",

2. copy files from node_modules to output directory after build in prepare-package.js
const postBuild = async () => {
  await cpvm2("bridge.js");
  await cpvm2("events.js");
  await cpvm2("setup-sandbox.js");
  await cpvm2("setup-node-sandbox.js");
};

3. add them in package.json files property to bundle them

A more straightforward approach if you use CopyPlugin in your webpack is

new CopyPlugin({
		patterns: [
			{ from: '../node_modules/vm2/lib/bridge.js', to: 'server' },
			{ from: '../node_modules/vm2/lib/events.js', to: 'server' },
			{ from: '../node_modules/vm2/lib/setup-node-sandbox.js', to: 'server'	},
			{ from: '../node_modules/vm2/lib/setup-sandbox.js', to: 'server' },
		],
}),

OrKoN · 2023-07-13T07:15:43Z

fixed via #10548

karlhorky · 2023-07-13T10:15:12Z

Nice, thanks for circling back around here @OrKoN! 🙌 (and for making the PR)

I can confirm that building with esbuild works after:

upgrading to puppeteer-core@20.8.2 (which uses @puppeteer/browsers@1.4.5 internally)
removing Workaround 2 (the --external:vm2 flag)

gabrielo91 · 2023-07-14T19:06:41Z

thanks so much!

karlhorky added the bug label Jun 8, 2023

karlhorky changed the title ~~[Bug]:~~ [Bug]: esbuild-bundled script crashes with ENOENT (related to vm2?) Jun 8, 2023

karlhorky mentioned this issue Jun 8, 2023

Work in a bundle patriksimek/vm2#528

Open

github-actions bot added needs-feedback not-reproducible labels Jun 8, 2023

github-actions bot added needs-feedback not-reproducible and removed needs-feedback not-reproducible labels Jun 8, 2023

karlhorky mentioned this issue Jun 8, 2023

Running esbuild-bundled script crashes with ENOENT (related to vm2) TooTallNate/proxy-agents#197

Closed

github-actions bot added needs-feedback not-reproducible and removed not-reproducible needs-feedback labels Jun 8, 2023

github-actions bot added needs-feedback not-reproducible and removed needs-feedback not-reproducible labels Jun 9, 2023

Lightning00Blade added confirmed and removed needs-feedback not-reproducible labels Jun 9, 2023

Lightning00Blade mentioned this issue Jun 29, 2023

Bug: pupeteer-core ^20.4.0 can't be bundled with pkg #10477

Closed

2 tasks

OrKoN closed this as completed Jul 13, 2023

ikoamu mentioned this issue Jul 14, 2023

No such file or directory error using @markdown-confluence/cli - dist/bridge.js markdown-confluence/markdown-confluence#413

Open

mdesousa mentioned this issue Nov 13, 2023

esbuild-bundled script crashes with ENOENT docusign/docusign-esign-node-client#340

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: esbuild-bundled script crashes with ENOENT (related to `vm2`?) #10352

[Bug]: esbuild-bundled script crashes with ENOENT (related to `vm2`?) #10352

karlhorky commented Jun 8, 2023 •

edited

github-actions bot commented Jun 8, 2023 •

edited

cnlewis3 commented Jun 8, 2023

Lightning00Blade commented Jun 9, 2023

karlhorky commented Jun 9, 2023 •

edited

karlhorky commented Jun 9, 2023 •

edited

OrKoN commented Jun 12, 2023

karlhorky commented Jun 12, 2023

karlhorky commented Jun 12, 2023

OrKoN commented Jun 12, 2023

OrKoN commented Jun 12, 2023

karlhorky commented Jun 12, 2023 •

edited

karlhorky commented Jun 12, 2023 •

edited

karlhorky commented Jun 12, 2023

OrKoN commented Jun 12, 2023 •

edited

karlhorky commented Jun 12, 2023 •

edited

tranlehaiquan commented Jul 3, 2023

gabrielo91 commented Jul 4, 2023

gabrielo91 commented Jul 5, 2023

PluviaCon commented Jul 12, 2023

Workaround 2

edw1882 commented Jul 12, 2023 •

edited

Workaround 1

OrKoN commented Jul 13, 2023

karlhorky commented Jul 13, 2023 •

edited

gabrielo91 commented Jul 14, 2023

[Bug]: esbuild-bundled script crashes with ENOENT (related to vm2?) #10352

[Bug]: esbuild-bundled script crashes with ENOENT (related to vm2?) #10352

Comments

karlhorky commented Jun 8, 2023 • edited

Bug expectation

Bug behavior

Minimal, reproducible example

Error string

Puppeteer configuration

Puppeteer version

Node version

Package manager

Package manager version

Operating system

github-actions bot commented Jun 8, 2023 • edited

cnlewis3 commented Jun 8, 2023

Lightning00Blade commented Jun 9, 2023

karlhorky commented Jun 9, 2023 • edited

karlhorky commented Jun 9, 2023 • edited

OrKoN commented Jun 12, 2023

karlhorky commented Jun 12, 2023

karlhorky commented Jun 12, 2023

OrKoN commented Jun 12, 2023

OrKoN commented Jun 12, 2023

karlhorky commented Jun 12, 2023 • edited

karlhorky commented Jun 12, 2023 • edited

Workaround 1

karlhorky commented Jun 12, 2023

OrKoN commented Jun 12, 2023 • edited

karlhorky commented Jun 12, 2023 • edited

Workaround 2

tranlehaiquan commented Jul 3, 2023

gabrielo91 commented Jul 4, 2023

gabrielo91 commented Jul 5, 2023

PluviaCon commented Jul 12, 2023

Workaround 2

edw1882 commented Jul 12, 2023 • edited

Workaround 1

OrKoN commented Jul 13, 2023

karlhorky commented Jul 13, 2023 • edited

gabrielo91 commented Jul 14, 2023

[Bug]: esbuild-bundled script crashes with ENOENT (related to `vm2`?) #10352

[Bug]: esbuild-bundled script crashes with ENOENT (related to `vm2`?) #10352

karlhorky commented Jun 8, 2023 •

edited

github-actions bot commented Jun 8, 2023 •

edited

karlhorky commented Jun 9, 2023 •

edited

karlhorky commented Jun 9, 2023 •

edited

karlhorky commented Jun 12, 2023 •

edited

karlhorky commented Jun 12, 2023 •

edited

OrKoN commented Jun 12, 2023 •

edited

karlhorky commented Jun 12, 2023 •

edited

edw1882 commented Jul 12, 2023 •

edited

karlhorky commented Jul 13, 2023 •

edited