Fix broken yarn pnp #32867

kaykdm · 2021-12-28T07:14:12Z

x-ref #31552
x-ref #32115
x-ref #32546
x-ref #32721

Since this PR #31455 is merged, enhanced-resolve dependency's resolved field is changed which caused broken yarn pnp.
I am not sure how this field has been changed or this is intentional or not
When I install webpack locally, enhanced-resolve's resolved field in lock file is always registry.yarnpkg.com not codeload.github.com

Bug

Related issues linked using fixes #number
Integration tests added
Errors have helpful link attached, see contributing.md

Feature

Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
Related issues linked using fixes #number
Integration tests added
Documentation added
Telemetry added. In case of a feature if it's used or not.
Errors have helpful link attached, see contributing.md

Documentation / Examples

Make sure the linting passes by running yarn lint

grefrit · 2021-12-28T13:25:32Z

Can't wait for this, because of broken PnP we have to stay on Next 11. Thanks a lot)

rtritto · 2021-12-28T14:56:31Z

If this PR fixes #31812, should #31995 be reverted/removed?

FYI @sokra

kaykdm · 2021-12-28T15:17:08Z

If this PR fixes #31812, should #31995 be reverted/removed?

FYI @sokra

@rtritto
oh, I am sorry for not being specific .
This PR does not fix #31812. You still need to set swcFileReading: false to be able to run in pnp.
I am gonna remove the issue link.
Thanks

sokra

This is definitely invalid in our lockfile, but afaik our lockfile doesn't affect next.js users at all. So I don't think this will fix any problem.

ijjk · 2022-01-11T19:39:44Z

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall decrease ✓

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
buildDuration	14s	14s	⚠️ +36ms
buildDurationCached	3.1s	3.1s	⚠️ +20ms
nodeModulesSize	355 MB	355 MB	-8.75 kB

Page Load Tests Overall increase ✓

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
/ failed reqs	0	0	✓
/ total time (seconds)	2.827	2.861	⚠️ +0.03
/ avg req/sec	884.24	873.85	⚠️ -10.39
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.293	1.277	-0.02
/error-in-render avg req/sec	1933.16	1957.98	+24.82

Client Bundles (main, webpack, commons)

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42.2 kB	42.2 kB	✓
main-HASH.js gzip	27.1 kB	27.1 kB	✓
webpack-HASH.js gzip	1.45 kB	1.45 kB	✓
Overall change	70.9 kB	70.9 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
_app-HASH.js gzip	1.37 kB	1.37 kB	✓
_error-HASH.js gzip	194 B	194 B	✓
amp-HASH.js gzip	312 B	312 B	✓
css-HASH.js gzip	326 B	326 B	✓
dynamic-HASH.js gzip	2.37 kB	2.37 kB	✓
head-HASH.js gzip	350 B	350 B	✓
hooks-HASH.js gzip	919 B	919 B	✓
image-HASH.js gzip	4.74 kB	4.74 kB	✓
index-HASH.js gzip	263 B	263 B	✓
link-HASH.js gzip	2.13 kB	2.13 kB	✓
routerDirect..HASH.js gzip	321 B	321 B	✓
script-HASH.js gzip	383 B	383 B	✓
withRouter-HASH.js gzip	318 B	318 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.1 kB	14.1 kB	✓

Client Build Manifests

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
_buildManifest.js gzip	459 B	459 B	✓
Overall change	459 B	459 B	✓

Rendered Page Sizes

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
index.html gzip	533 B	533 B	✓
link.html gzip	547 B	547 B	✓
withRouter.html gzip	528 B	528 B	✓
Overall change	1.61 kB	1.61 kB	✓

Default Build with SWC (Decrease detected ✓)

General Overall decrease ✓

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
buildDuration	15.3s	15.5s	⚠️ +220ms
buildDurationCached	3.1s	3.1s	⚠️ +39ms
nodeModulesSize	355 MB	355 MB	-8.75 kB

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
/ failed reqs	0	0	✓
/ total time (seconds)	2.836	2.869	⚠️ +0.03
/ avg req/sec	881.56	871.27	⚠️ -10.29
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.28	1.283	0
/error-in-render avg req/sec	1953.19	1949.04	⚠️ -4.15

Client Bundles (main, webpack, commons)

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42.3 kB	42.3 kB	✓
main-HASH.js gzip	27.2 kB	27.2 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	71.1 kB	71.1 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
_app-HASH.js gzip	1.35 kB	1.35 kB	✓
_error-HASH.js gzip	180 B	180 B	✓
amp-HASH.js gzip	305 B	305 B	✓
css-HASH.js gzip	321 B	321 B	✓
dynamic-HASH.js gzip	2.36 kB	2.36 kB	✓
head-HASH.js gzip	342 B	342 B	✓
hooks-HASH.js gzip	906 B	906 B	✓
image-HASH.js gzip	4.76 kB	4.76 kB	✓
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	2.19 kB	2.19 kB	✓
routerDirect..HASH.js gzip	314 B	314 B	✓
script-HASH.js gzip	375 B	375 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.1 kB	14.1 kB	✓

Client Build Manifests

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
_buildManifest.js gzip	458 B	458 B	✓
Overall change	458 B	458 B	✓

Rendered Page Sizes

	vercel/next.js canary	kaykdm/next.js fix/yarn-pnp-error	Change
index.html gzip	532 B	532 B	✓
link.html gzip	545 B	545 B	✓
withRouter.html gzip	526 B	526 B	✓
Overall change	1.6 kB	1.6 kB	✓

Commit: 0739887

kachkaev · 2022-01-11T20:44:33Z

Keen to test this with Yarn 3 once a new canary is out!

balazsorban44 · 2022-01-11T22:02:13Z

There is @kachkaev, please check out 12.0.8-canary.21 and report back in a new issue if something is wrong! 🙏

kachkaev · 2022-01-11T22:06:16Z

I just bumped Next to 12.0.8-canary.21 in kachkaev/njt#161. Happy to confirm that yarn dev, yarn build and yarn start work again for me! 🎉 Huge thanks to @kaykdm for this PR!

rauchg · 2022-01-11T23:31:09Z

How can we set up automated testing to avoid this kind of regression?

kachkaev · 2022-01-12T09:46:03Z

Yeah having some tests would be nice. I just upgraded from "next": "^12.0.8-canary.21" to "next": "^12.0.8" and cryptic Yarn PnP errors went back 😅

yarn dev

ready - started server on 0.0.0.0:3000, url: http://localhost:3000
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

error - ./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/performance-relayer.js
Error: Failed to read source code from /path/to/project/.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/performance-relayer.js

Caused by:
    No such file or directory (os error 2)
wait  - compiling...
error - ./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/performance-relayer.js
Error: Failed to read source code from /path/to/project/.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/performance-relayer.js

Caused by:
    No such file or directory (os error 2)
wait  - compiling...
error - ./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/performance-relayer.js
Error: Failed to read source code from /path/to/project/.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/performance-relayer.js

yarn build

warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

info  - Skipping validation of types  
info  - Creating an optimized production build  
Failed to compile.

./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/portal/index.js
Error: Failed to read source code from /path/to/project/.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/portal/index.js

Caused by:
    No such file or directory (os error 2)

Import trace for requested module:
./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/index.js
./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/next.js

./.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/script.js
Error: Failed to read source code from /path/to/project/.yarn/__virtual__/next-virtual-1575485aa2/0/cache/next-npm-12.0.8-aa55acca00-947f0295aa.zip/node_modules/next/dist/client/script.js

Let’s create a new issue if anyone else can confirm. You can try it in kachkaev/njt#161 by upgrading next and @next/eslint-plugin-next in package.json.

What’s interesting is that yarn dev is able to work after a couple of attempts, so there might be something with timing. yarn build fails 100% of times in 12.0.8 though.

kaykdm · 2022-01-12T09:55:43Z

@kachkaev
I just tried "next": "^12.0.8", and it is working for me.
Did you set swcFileReading: false in your next.config.js ?

kachkaev · 2022-01-12T10:03:31Z

Good shout @kaykdm! I had swcFileReading: false when testing 12.0.8-canary.21. Then I removed the option, ran the commands again and they were still OK (probably because of caching).

I can confirm that 12.0.8 works when swcFileReading: false is present in next.config.mjs 🎉

Not sure how difficult it would be to support swcFileReading but I wonder if Next could detect Yarn PnP and show a nice error instead of the cryptic ones. I wish I could submit a PR, but I currently don’t have enough time to implement this little DX improvement.

brc-dd · 2022-01-12T12:21:42Z

How can we set up automated testing to avoid this kind of regression?

@rauchg You can just check if there is any match for /:\/\/(?!registry.yarnpkg.com)/g in your lockfile. 😄

There are two such matches right now BTW. Also, I believe that setting registry "https://registry.yarnpkg.com/" in .yarnrc at project root can reduce such issues in the future.

…o-example * 'canary' of github.com:vercel/next.js: Added links to data fetching api refs, fixed title (vercel#33221) Removed backticks on data fetching api titles (vercel#33216) middlewares: limit `process.env` to inferred usage (vercel#33186) Fixed broken link (vercel#33209) v12.0.8 v12.0.8-canary.22 Refactor data fetching API docs (vercel#30615) Docs: correct ignorance pattern for .env.local (vercel#32647) Fixes vercel#33153: Updating cross-references from master to main + canary (vercel#33198) v12.0.8-canary.21 Add util for normalizing errors (vercel#33159) Fix broken yarn pnp (vercel#32867)

ijjk · 2022-01-12T22:09:21Z

Regression tests for this are being added in #33236

kaykdm requested review from huozhi, ijjk, shuding and timneutkens as code owners December 28, 2021 07:14

ijjk added the type: next label Dec 28, 2021

kaykdm mentioned this pull request Dec 28, 2021

next dev and next build broken on Yarn PnP in the latest release #32115

Closed

kaykdm force-pushed the fix/yarn-pnp-error branch 2 times, most recently from 1258ffa to 087cf9b Compare December 28, 2021 07:43

This comment has been minimized.

Sign in to view

kaykdm changed the title ~~fix broken yarn pnp~~ Fix broken yarn pnp Dec 29, 2021

kaykdm added 2 commits January 6, 2022 11:32

Fix broken yarn pnp

d457fbf

Update precompiled

1330e6c

kaykdm force-pushed the fix/yarn-pnp-error branch from abcbafd to 1330e6c Compare January 6, 2022 02:37

Remove precompiled

3b0ba34

This comment has been minimized.

Sign in to view

sokra added 2 commits January 11, 2022 14:48

Merge branch 'canary' into fix/yarn-pnp-error

ff3d32c

revert unrelated change

b0a84b6

sokra previously approved these changes Jan 11, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

update compiled

c3c68a4

ijjk dismissed sokra’s stale review via c3c68a4 January 11, 2022 18:33

ijjk previously approved these changes Jan 11, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

update font test

0739887

ijjk dismissed their stale review via 0739887 January 11, 2022 19:18

ijjk approved these changes Jan 11, 2022

View reviewed changes

kodiakhq bot merged commit 2d0fd34 into vercel:canary Jan 11, 2022

kachkaev mentioned this pull request Jan 11, 2022

The "path" argument must be of type string or an instance of Buffer or URL (react-dom) #31552

Closed

kaykdm deleted the fix/yarn-pnp-error branch January 12, 2022 02:00

kachkaev mentioned this pull request Jan 12, 2022

Error: Failed to read source code from amp-dev.js #31812

Closed

ijjk mentioned this pull request Jan 12, 2022

Update yarn PnP tests and disable swc file reading for PnP #33236

Merged

3 tasks

vercel locked as resolved and limited conversation to collaborators Feb 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix broken yarn pnp #32867

Fix broken yarn pnp #32867

kaykdm commented Dec 28, 2021 •

edited by ijjk

This comment has been minimized.

This comment has been minimized.

grefrit commented Dec 28, 2021

rtritto commented Dec 28, 2021 •

edited

kaykdm commented Dec 28, 2021

This comment has been minimized.

sokra left a comment

This comment has been minimized.

This comment has been minimized.

ijjk commented Jan 11, 2022

kachkaev commented Jan 11, 2022

balazsorban44 commented Jan 11, 2022

kachkaev commented Jan 11, 2022 •

edited

rauchg commented Jan 11, 2022

kachkaev commented Jan 12, 2022 •

edited

kaykdm commented Jan 12, 2022 •

edited

kachkaev commented Jan 12, 2022 •

edited

brc-dd commented Jan 12, 2022

ijjk commented Jan 12, 2022

Fix broken yarn pnp #32867

Fix broken yarn pnp #32867

Conversation

kaykdm commented Dec 28, 2021 • edited by ijjk

Bug

Feature

Documentation / Examples

This comment has been minimized.

This comment has been minimized.

grefrit commented Dec 28, 2021

rtritto commented Dec 28, 2021 • edited

kaykdm commented Dec 28, 2021

This comment has been minimized.

sokra left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

ijjk commented Jan 11, 2022

Stats from current PR

kachkaev commented Jan 11, 2022

balazsorban44 commented Jan 11, 2022

kachkaev commented Jan 11, 2022 • edited

rauchg commented Jan 11, 2022

kachkaev commented Jan 12, 2022 • edited

kaykdm commented Jan 12, 2022 • edited

kachkaev commented Jan 12, 2022 • edited

brc-dd commented Jan 12, 2022

ijjk commented Jan 12, 2022

kaykdm commented Dec 28, 2021 •

edited by ijjk

rtritto commented Dec 28, 2021 •

edited

kachkaev commented Jan 11, 2022 •

edited

kachkaev commented Jan 12, 2022 •

edited

kaykdm commented Jan 12, 2022 •

edited

kachkaev commented Jan 12, 2022 •

edited