[Bug?]: yarn install link step very slow

[rxliuli]

Self-service

• I'd be willing to implement a fix

Describe the bug

It still takes a long time to use yarn to reinstall dependencies without any changes. I'm not sure what happened. We have 30 modules and nested workspaces.

Below is the output

➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed in 0s 480ms
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 2s 765ms
➤ YN0000: ┌ Link step
➤ YN0062: │ fsevents@patch:fsevents@npm%3A2.3.2#~builtin<compat/fsevents>::version=2.3.2&hash=1cc4b2 The platform win32 is incompatible with this module, link skipped.
➤ YN0062: │ fsevents@patch:fsevents@npm%3A1.2.13#~builtin<compat/fsevents>::version=1.2.13&hash=1cc4b2 The platform win32 is incompatible with this module, link skipped.
➤ YN0000: └ Completed in 38s 348ms
➤ YN0000: Done with warnings in 41s 807ms

To reproduce

Unfortunately, only the production environment has this problem. I have not encountered this problem in my two personal projects. If there is any additional information I can provide, please let me know.

Maybe yarn has a debug mode switch or detailed log?

Environment

System:
    OS: Windows 10 10.0.22000
    CPU: (16) x64 Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz
  Binaries:
    Node: 16.9.1 - ~\AppData\Local\Temp\xfs-48c94d8d\node.CMD
    Yarn: 3.0.1 - ~\AppData\Local\Temp\xfs-48c94d8d\yarn.CMD
    npm: 7.21.1 - C:\Program Files\nodejs\npm.CMD
  npmPackages:
    jest: ^27.2.1 => 27.2.1

Additional context

No response

[rxliuli]

I guess it may be related to the deeply nested workspace. After I deleted the three-level nested workspace, it now takes 19s, but there may be other reasons

➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed in 0s 587ms
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 3s 106ms
➤ YN0000: ┌ Link step
➤ YN0062: │ fsevents@patch:fsevents@npm%3A2.3.2#~builtin<compat/fsevents>::version=2.3.2&hash=1cc4b2 The platform win32 is incompatible with this module, link skipped.
➤ YN0062: │ fsevents@patch:fsevents@npm%3A1.2.13#~builtin<compat/fsevents>::version=1.2.13&hash=1cc4b2 The platform win32 is incompatible with this module, link skipped.
➤ YN0000: └ Completed in 19s 213ms
➤ YN0000: Done with warnings in 23s 133ms

[larixer]

@rxliuli What linker do you use?

[rxliuli]

@rxliuli What linker do you use?

compressionLevel: 0

enableGlobalCache: true

nodeLinker: node-modules

npmRegistryServer: "https://registry.yarnpkg.com"

plugins:
  - path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs
    spec: "@yarnpkg/plugin-workspace-tools"
  - path: .yarn/plugins/@yarnpkg/plugin-interactive-tools.cjs
    spec: "@yarnpkg/plugin-interactive-tools"
  - path: .yarn/plugins/@yarnpkg/plugin-typescript.cjs
    spec: "@yarnpkg/plugin-typescript"

yarnPath: .yarn/releases/yarn-berry.cjs

[larixer]

@rxliuli Try to see the output of NM_DEBUG_LEVEL=1 yarn on the project with slow link time, how much time does hoisting take?

[rxliuli]

I redirected all the output of the above command to the log file

yarn.log

[larixer]

@rxliuli Yes, there is a problem with hoisting, could you try to use Yarn from this PR:
#3438
via
yarn set version from source --branch 3438 and then run
NM_DEBUG_LEVEL=1 yarn again and check the hoisting time?

[rxliuli]

@rxliuli Yes, there is a problem with hoisting, could you try to use Yarn from this PR:
#3438
via
yarn set version from source --branch 3438 and then run
NM_DEBUG_LEVEL=1 yarn again and check the hoisting time?

I cannot run yarn set version from source --branch 3438, please let me know if I did something wrong

$ yarn set version from source --branch 3438
Unknown Syntax Error: Command not found; did you mean:

$ yarn set version from sources [--path #0] [--repository #0] [--branch #0] [--plugin #0] [--no-minify] [-f,--force] [--skip-plugins]
While running set version from source --branch 3438

---

Got it, there is a word error in your command

[rxliuli]

It's much faster, but it still takes 7s. In addition, running yarn directly from the command line is inconsistent with NM_DEBUG_LEVEL=1 yarn.

➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed in 0s 596ms
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 2s 658ms
➤ YN0000: ┌ Link step
➤ YN0062: │ fsevents@patch:fsevents@npm%3A2.3.2#~builtin<compat/fsevents>::version=2.3.2&hash=18f3a7 The platform win32 is incompatible with this module, link skipped.
➤ YN0062: │ fsevents@patch:fsevents@npm%3A1.2.13#~builtin<compat/fsevents>::version=1.2.13&hash=18f3a7 The platform win32 is incompatible with this module, link skipped.
➤ YN0000: └ Completed in 3s 895ms
➤ YN0000: Done with warnings in 7s 356ms

yarn.log

---

In addition, I have to fetch 2s+ every time. What is yarn doing during this time?

[merceyz]

In addition, I have to fetch 2s+ every time. What is yarn doing during this time?

Checking that the checksum of all the packages you use matches the lockfile to protect against a corrupted/modified cache

[yarnbot]

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟