[Bug]: Dramatic performance drop with ESM since v14.0.0

[feesler]

Bug description

Steps to reproduce the problem:

1. Set package type as module, install puppeteer v13.7.0 dependency
2. import puppeteer from 'puppeteer';
3. Run some script and measure performance
4. Update puppeteer to v14.0.0 and repeat step 3

For convenience, I created a https://github.com/feesler/puppeteer-perf package to test this issue.

git clone https://github.com/feesler/puppeteer-perf.git
cd puppeteer-perf
npm install
npm test
npm install puppeteer@14.0.0
npm test

Script will automatically download Chromium revision 982053 to discard possible difference in browser performance.

Temporary solution to fix performance for ESM packages:

- import puppeteer from 'puppeteer';
+ import puppeteer from 'puppeteer/lib/cjs/puppeteer/puppeteer.js';

Puppeteer version

14.0.0 - 15.3.1

Node.js version

16.15.1 - 18.2.0

npm version

8.1.4

What operating system are you seeing the problem on?

Windows

Relevant log output

Node: v16.15.1
Puppeteer: 13.7.0
Browser: HeadlessChrome/101.0.4950.0
Duration: 77.40569996833801

Node: v16.15.1
Puppeteer: 14.0.0
Browser: HeadlessChrome/101.0.4950.0
Duration: 730.2374999523163

Node: v16.15.1
Puppeteer: 15.3.1
Browser: HeadlessChrome/101.0.4950.0
Duration: 742.4602999687195

Node: v18.2.0
Puppeteer: 13.7.0
Browser: HeadlessChrome/101.0.4950.0
Duration: 72.06540012359619

Node: v18.2.0
Puppeteer: 14.0.0
Browser: HeadlessChrome/101.0.4950.0
Duration: 711.3672997951508

Node: v18.2.0
Puppeteer: 15.3.1
Browser: HeadlessChrome/101.0.4950.0
Duration: 770.1029999256134

[OrKoN]

I am able to reproduce. It looks like dynamic imports are much worse performing with the ESM loader compared to the CJS loader. I suggest we cache the dynamic imports on our side.

[OrKoN]

Caching seems to help and the numbers go back down for me: #8652 Could anyone confirm?

[feesler]

Checked patch on my environment. Still 30% slower than v13.7.0, but it way more better than last results.
Thanks!

Node: v18.2.0
Puppeteer: 15.3.1
Browser: HeadlessChrome/101.0.4950.0
Duration: 93.32049989700317

[OrKoN]

On my machine, it's almost identical with Node 18:

❯ npm test

> puppeteer-perf@1.0.0 test
> node index.js

Node: v18.4.0
Puppeteer: 13.7.0
Browser: HeadlessChrome/101.0.4950.0
Duration: 47.12399911880493

~/pptr-test/puppeteer-perf main*
❯ npm test

> puppeteer-perf@1.0.0 test
> node index.js

Node: v18.4.0
Puppeteer: 13.7.0
Browser: HeadlessChrome/101.0.4950.0
Duration: 45.374149322509766

~/pptr-test/puppeteer-perf main*
❯ npm test

> puppeteer-perf@1.0.0 test
> node index.js

Node: v18.4.0
Puppeteer: 15.3.2
Browser: HeadlessChrome/101.0.4950.0
Duration: 47.71191596984863

~/pptr-test/puppeteer-perf main* 21s
❯ npm test

> puppeteer-perf@1.0.0 test
> node index.js

Node: v18.4.0
Puppeteer: 15.3.2
Browser: HeadlessChrome/101.0.4950.0
Duration: 42.071983337402344

~/pptr-test/puppeteer-perf main*
❯ npm test

> puppeteer-perf@1.0.0 test
> node index.js

Node: v18.4.0
Puppeteer: 15.3.2
Browser: HeadlessChrome/101.0.4950.0
Duration: 49.50673818588257