Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use require.resolve to detect "framework" packages (fix pnpm) #21048

Merged
merged 27 commits into from Nov 9, 2021
Merged

Use require.resolve to detect "framework" packages (fix pnpm) #21048

merged 27 commits into from Nov 9, 2021

Conversation

elliottsj
Copy link
Contributor

@elliottsj elliottsj commented Jan 13, 2021 •
edited

Fix #20884

The previous regex was not compatible with pnpm's module paths:

(?<!node_modules.*)[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]

e.g.

yarn

> /(?<!node_modules.*)[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/.test(
    "/Users/spencerelliott/dev/elliottsj/nextjs-pnpm-issue-2021-01-07-yarn/node_modules/react-dom/index.js"
  )
true

pnpm

> /(?<!node_modules.*)[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/.test(
    "/Users/spencerelliott/dev/elliottsj/nextjs-pnpm-issue-2021-01-07/node_modules/.pnpm/react-dom@17.0.1_react@17.0.1/node_modules/react-dom/index.js"
  )
false

This meant that the static/chunks/framework.[hash].js chunk was not added to the build manifest, and never rendered as a <link href="..." as="script">, so "react" / "react-dom" would never be loaded and we would never be able to execute the bundle's entry point.

By replacing the regex with require.resolve() to check whether modules should be moved to the "framework" chunk, we're using Node.js's module resolution instead of relying on a regex which implicitly assumes the folder structure produced by npm/yarn.

Tests pending. I'm thinking of modifying the existing pnpm integration test to verify that pages execute JS under a particular module configuration (like nextjs-pnpm-issue-2021-01-07), but ideally using something even more minimal than adding a dependency on react-modal. I'll look into this more soon, feedback appreciated.

I've added integration tests for the bug and verified (locally) that the tests fail without 3740547.

@ijjk

This comment has been minimized.

Sign in to view
@elliottsj elliottsj force-pushed the split-chunks-framework-pnpm-fix branch 2 times, most recently from f4a47b4 to eb0742d Compare January 16, 2021 23:00
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj elliottsj force-pushed the split-chunks-framework-pnpm-fix branch from eb0742d to 3740547 Compare January 17, 2021 00:57
@elliottsj elliottsj marked this pull request as ready for review January 17, 2021 00:57
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj elliottsj mentioned this pull request Jan 17, 2021
Closed
@elliottsj
Copy link
Contributor Author

I've added integration tests for the bug and verified (locally) that the tests fail without 3740547.

Once CI passes, this is ready for review from my end. Feedback appreciated.

@ijjk

This comment has been minimized.

Sign in to view
@ijjk

This comment has been minimized.

Sign in to view
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj
Merge remote-tracking branch 'upstream/canary' into split-chunks-fram…
a9c2ec8 
…ework-pnpm-fix
@elliottsj
Wait 5 seconds after killing app
c67d0aa 
Try to avoid 'EBUSY: resource busy or locked' on Windows when removing
temp directory
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj
Use tree-kill to kill process
0a347ed 
Try to avoid 'EBUSY: resource busy or locked' on Windows when removing
temp directory
@elliottsj
Merge remote-tracking branch 'upstream/canary' into split-chunks-fram…
edbf543 
…ework-pnpm-fix
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj elliottsj requested a review from divmain as a code owner April 11, 2021 17:23
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj
Check for .nameForCondition method before calling
6dd774b
@elliottsj
Remove prop-types from 'framework' chunk paths
4450a12 
It's no longer needed because 'next' no longer declares a direct
dep on 'prop-types' since c7e2a1d
@elliottsj elliottsj force-pushed the split-chunks-framework-pnpm-fix branch from 47ede67 to 4450a12 Compare August 28, 2021 18:41
@ijjk

This comment has been minimized.

Sign in to view
@elliottsj
Copy link
Contributor Author

@sokra I've fixed the test failures. The error in the next-swc job Error: Unexpected response. Unable to upload chunk seems unrelated.

@ijjk

This comment has been minimized.

Sign in to view
@elliottsj
Fix pnpm-support tests
16b2443 
- Upgrade to react@17 for next@11 compat
- Add .babelrc to disable SWC
- Add @next/polyfill-nomodule dep
@elliottsj
Merge remote-tracking branch 'upstream/canary' into split-chunks-fram…
5f96288 
…ework-pnpm-fix
@ijjk

This comment has been minimized.

Sign in to view
@PabloSzx
Copy link
Contributor

PabloSzx commented Nov 5, 2021 •
edited

Bump 👍

As I mentioned it here, the --shamefully-hoist workaround stopped working for Next.js 12 Serverless Functions in Vercel #16471 (comment)

EDIT: It seems like the --shamefully-hoist workaround was fixed for Next.js 12.0.3, maybe related to #30786 🎉

sokra
sokra requested changes Nov 8, 2021
View reviewed changes
packages/next/build/webpack-config.ts
// 'node_modules/react-scroll-parallax/node_modules/prop-types' are not included.
const topLevelFrameworkPaths = [
getPackagePath('react', dir),
getPackagePath('react-dom', dir),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prop-types seem to be missing here?

It was part of the original regexp

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also react and react-dom depend on object-assign so that might also need to be added here, even if it wasn't part of the regexp before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prop-types should be no longer needed since next no longer declares a direct dep on prop-types. See 4450a12 and c7e2a1d

I'll add object-assign

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

Copy link
Contributor Author

@elliottsj elliottsj Nov 9, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For what it's worth, this error is raised when getPackagePath('prop-types', dir) is added to the array:

> pnpm-app@1.0.0 build /private/var/folders/zj/25m6hvw14q5fgtm87smrxsbc0000gn/T/hobwb9bsxk9/app
> next build

info  - Checking validity of types
warn  - No ESLint configuration detected. Run next lint to begin setup
info  - Disabled SWC as replacement for Babel because of custom Babel configuration ".babelrc" https://nextjs.org/docs/messages/swc-disabled

> Build error occurred
Error: Cannot find module 'prop-types/package.json'
Require stack:
- /private/var/folders/zj/25m6hvw14q5fgtm87smrxsbc0000gn/T/hobwb9bsxk9/app/node_modules/.pnpm/local++var+folders+zj+25m6hvw14q5fgtm87smrxsbc0000_05da36a7aef9195a59c6f2b92d013fed/node_modules/next/dist/build/webpack-config.js
- /private/var/folders/zj/25m6hvw14q5fgtm87smrxsbc0000gn/T/hobwb9bsxk9/app/node_modules/.pnpm/local++var+folders+zj+25m6hvw14q5fgtm87smrxsbc0000_05da36a7aef9195a59c6f2b92d013fed/node_modules/next/dist/build/index.js

@ijjk

This comment has been minimized.

Sign in to view
@ijjk

This comment has been minimized.

Sign in to view
@ijjk

This comment has been minimized.

Sign in to view
@ijjk
Copy link
Member

ijjk commented Nov 8, 2021

Failing test suites

Commit: c097bb2

test/integration/pnpm-support/test/index.test.js

  • pnpm support > should execute client-side JS on each page
Expand output

● pnpm support › should execute client-side JS on each page

FetchError: request to http://localhost:43135/ failed, reason: connect ECONNREFUSED 127.0.0.1:43135

  at ClientRequest.<anonymous> (../node_modules/node-fetch/lib/index.js:1461:11)

@ijjk
Copy link
Member

ijjk commented Nov 9, 2021

Stats from current PR

Default Build (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
buildDuration 21.9s 21.6s -289ms
buildDurationCached 4.5s 4.4s -133ms
nodeModulesSize 332 MB 332 MB ⚠️ +2.82 kB
Page Load Tests Overall increase ✓
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
/ failed reqs 0 0 ✓
/ total time (seconds) 3.578 3.577 0
/ avg req/sec 698.79 698.93 +0.14
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.739 1.678 -0.06
/error-in-render avg req/sec 1437.93 1489.9 +51.97
Client Bundles (main, webpack, commons)
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
450.HASH.js gzip 179 B 179 B ✓
framework-HASH.js gzip 42.2 kB 42.2 kB ✓
main-HASH.js gzip 28.1 kB 28.1 kB ✓
webpack-HASH.js gzip 1.45 kB 1.45 kB ✓
Overall change 71.9 kB 71.9 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
polyfills-HASH.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
_app-HASH.js gzip 1.23 kB 1.23 kB ✓
_error-HASH.js gzip 194 B 194 B ✓
amp-HASH.js gzip 312 B 312 B ✓
css-HASH.js gzip 327 B 327 B ✓
dynamic-HASH.js gzip 2.38 kB 2.38 kB ✓
head-HASH.js gzip 350 B 350 B ✓
hooks-HASH.js gzip 635 B 635 B ✓
image-HASH.js gzip 4.44 kB 4.44 kB ✓
index-HASH.js gzip 263 B 263 B ✓
link-HASH.js gzip 1.87 kB 1.87 kB ✓
routerDirect..HASH.js gzip 321 B 321 B ✓
script-HASH.js gzip 383 B 383 B ✓
withRouter-HASH.js gzip 318 B 318 B ✓
85e02e95b279..7e3.css gzip 107 B 107 B ✓
Overall change 13.1 kB 13.1 kB ✓
Client Build Manifests
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
_buildManifest.js gzip 460 B 460 B ✓
Overall change 460 B 460 B ✓
Rendered Page Sizes
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
index.html gzip 523 B 523 B ✓
link.html gzip 536 B 536 B ✓
withRouter.html gzip 518 B 518 B ✓
Overall change 1.58 kB 1.58 kB ✓
Default Build with SWC
General Overall increase ⚠️
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
buildDuration 23.5s 22.9s -687ms
buildDurationCached 4.4s 4.1s -295ms
nodeModulesSize 332 MB 332 MB ⚠️ +2.82 kB
Page Load Tests Overall increase ✓
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
/ failed reqs 0 0 ✓
/ total time (seconds) 3.575 3.43 -0.15
/ avg req/sec 699.33 728.8 +29.47
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.65 1.677 ⚠️ +0.03
/error-in-render avg req/sec 1515.43 1490.49 ⚠️ -24.94
Client Bundles (main, webpack, commons)
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
450.HASH.js gzip 179 B 179 B ✓
framework-HASH.js gzip 42.3 kB 42.3 kB ✓
main-HASH.js gzip 28.2 kB 28.2 kB ✓
webpack-HASH.js gzip 1.43 kB 1.43 kB ✓
Overall change 72.1 kB 72.1 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
polyfills-HASH.js gzip 31 kB 31 kB ✓
Overall change 31 kB 31 kB ✓
Client Pages
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
_app-HASH.js gzip 1.22 kB 1.22 kB ✓
_error-HASH.js gzip 180 B 180 B ✓
amp-HASH.js gzip 305 B 305 B ✓
css-HASH.js gzip 321 B 321 B ✓
dynamic-HASH.js gzip 2.38 kB 2.38 kB ✓
head-HASH.js gzip 342 B 342 B ✓
hooks-HASH.js gzip 622 B 622 B ✓
image-HASH.js gzip 4.46 kB 4.46 kB ✓
index-HASH.js gzip 256 B 256 B ✓
link-HASH.js gzip 1.91 kB 1.91 kB ✓
routerDirect..HASH.js gzip 314 B 314 B ✓
script-HASH.js gzip 375 B 375 B ✓
withRouter-HASH.js gzip 309 B 309 B ✓
85e02e95b279..7e3.css gzip 107 B 107 B ✓
Overall change 13.1 kB 13.1 kB ✓
Client Build Manifests
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
_buildManifest.js gzip 459 B 459 B ✓
Overall change 459 B 459 B ✓
Rendered Page Sizes
vercel/next.js canary elliottsj/next.js split-chunks-framework-pnpm-fix Change
index.html gzip 523 B 523 B ✓
link.html gzip 535 B 535 B ✓
withRouter.html gzip 517 B 517 B ✓
Overall change 1.57 kB 1.57 kB ✓
Commit: c25339b

@elliottsj
Copy link
Contributor Author

@sokra @timneutkens tests now fixed

@timneutkens timneutkens merged commit a1e76fb into vercel:canary Nov 9, 2021
@elliottsj elliottsj deleted the split-chunks-framework-pnpm-fix branch November 9, 2021 16:03
@neeraj3029 neeraj3029 mentioned this pull request Nov 10, 2021
Closed
@vercel vercel locked as resolved and limited conversation to collaborators Jan 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@merceyz merceyz merceyz left review comments

@sokra sokra sokra approved these changes

@ijjk ijjk Awaiting requested review from ijjk ijjk is a code owner

@lfades lfades Awaiting requested review from lfades

@Timer Timer Awaiting requested review from Timer

@timneutkens timneutkens Awaiting requested review from timneutkens timneutkens is a code owner

@shuding shuding Awaiting requested review from shuding shuding is a code owner

@huozhi huozhi Awaiting requested review from huozhi huozhi is a code owner

@padmaia padmaia Awaiting requested review from padmaia

@styfle styfle Awaiting requested review from styfle

Assignees
No one assigned
Labels
type: next
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Client-side JS fails to execute when using pnpm and importing react-dom on a page
6 participants
@elliottsj @ijjk @PabloSzx @sokra @merceyz @timneutkens