React Native Project keeps loading infinitely & heats up CPU

[deadcoder0904]

pnpm version: 2.10.3

Code to reproduce the issue:

https://github.com/deadcoder0904/react-native-darkmode-list

Expected behavior:

Code should run properly

Actual behavior:

Code runs infinitely. I first tried simple pnpm install but it raised an error about some babel plugin not installed. So I set .npmrc file to shamefully-flatten=true & ran pnpm install & it went well.

Then I typed pnpm start & it keeps on running infinitely while heating my CPU. Same thing with yarn runs in few seconds like 10-15 & with pnpm I waited 2 minutes until it got very much heated that I closed the running process.

Additional information:

• node -v prints: 10.4.1
• Windows, OS X, or Linux?: OS X

[zkochan]

TBH I have no experience with React Native but I remember @vjpr mentioning that it has issues with symlinks (I think @pgonzal also mentioned it).

Can we start communication with the React Native team about these issues? I am almost sure that we can't do anything on our (pnpm) end.

[deadcoder0904]

I think I might've used it for a couple of projects but not sure. Since I often jump between pnpm & yarn when something doesn't work. But, yeah if its a known issue I'll use something else with React Native for now :)

[rodrigobdz]

In my opinion, this should be in the README. I just migrated to pnpm and first thing I discover while trying to run my React Native project is that it doesn't work.

[octogonz]

A while ago someone explained to me that there might be a cultural difference behind this issue: Typical NodeJS developers expect their code to run on any OS, with any package manager, and without installing any other software. Whereas native app developers often must install huge SDKs with multiple prerequisites and special device drivers, for each device that they target. For iOS you even need a specific development machine. In that context, the idea that you must use a certain certain package manager doesn't seem like a big concern. It's just one more "hoop" you have to jump through to make an app. The theory was that this is why there is relatively little effort invested in fixing the React Native symlink handling bugs, whereas other NodeJS packages get fixed quickly.

Not sure if it's true. (I haven't used React Native myself beyond trying the demos. But we hear about this issue occasionally from people who are wanting to enable PNPM for their Rush repositories.)

[zkochan]

Are there any updates with this issue? I think they will want to support Yarn Plug'n'Play, so they might make React Native more flexible

[likern]

@zkochan Are there any updates with this issue? I think they will want to support Yarn Plug'n'Play, so they might make React Native more flexible

To my knowledge the problem is not in yarn per se or package manager at all.
It's the problem of package bundler (like webpack).
React Native uses Facebook's Metro bundler https://github.com/facebook/metro which converts all javascript files into one file. This bundler doesn't support symlinks.

This issue is the root of the problems: facebook/metro#1

I see the following approaches to overcome this limitation:

1. If we could use only hard links that should solve the problem
2. utilize https://github.com/wix/wml which runs watcher and everytime files are changed copy over content directly (doesn't create symlinks)

[octogonz]

Why not just fix Metro?

[likern]

Why not just fix Metro?

It should, but... but will not be fixed anytime soon, at least in foreseeable future.
I've made some research and want to bring in some additional context about this.

Start of the story

Original issue facebook/metro#1 was created 1252 days or 3 years 5 months 6 days ago.
1 year 10 months 15 days ago @aleclarson provided pull request facebook/metro#257 fixing this issue.
Metro team first required to merge symlinks support in Jest (not sure how they are related, might be this breaks Jest tests).

Another pull request jestjs/jest#6993 to fix issue in Jest was provided by @aleclarson 1 year 10 months 15 days ago (and later, 1 year 7 month 7 days ago, was provided updated pull request jestjs/jest#7549 superseding original).

It turned out that Jest relies on Watchman (which watches file changes) which also doesn't have symlinks support.
jestjs/jest#7549 (comment)

Basically, for performance, we rely on Watchman telling us what files are changed. Watchman has no support for symlinks (facebook/watchman#105). That means if files are updated from a symlink, the haste map won't update it.

Jest developers refused to merge PR requesting "transparent symlink" implementation (don't know what that means) instead of resolving (cloning content) symlinks.
jestjs/jest#7549 (comment)

The reason I'm hesitant about this is because ...
... if it's possible to implement this in a way that hides symlinks from the rest of the system

Watchman also has issue about symlinks support facebook/watchman#105. It was created 3 years 8 months 26 days ago.
facebook/watchman#105 (comment)

Watchman developers said in comments that adding symlinks support

... would be tremendously complex and still be error prone. As a result, it is unlikely that we'll ever add support for resolving and tracking symlink targets

Final thoughts

1. For Facebook developers adding symlinks is a rocket science.
2. Reviewing pull requests (to look at first glance on PR) might take more than two months
3. All Pull Requests are stale and not moving forward at all (there is even no more comments from devs)
4. Metro waits Jest pull request merge (+735, −233 sloc), Jest waits on new implementation, Watchman refused to add support at all by themselves
5. No symlinks support anytime soon. In my estimation in period of 3-5 years at least.

[likern]

So if pnpm would want to support React Native and could solve that problem anyhow by themselves (by hacks / workarounds / etc) - I beg to do that.

I think the most straightforward and bulletproof solution would be to add ability to use original npm resolution -

1. Resolve all symlinks
2. Don't create symlinks at all and have deep nested original structure (yes, with package content duplication)

But this is the most simple solution I think. It might be turned on in some config (package.json) only on React Native projects
at the last resort. Thus no symlinks will be created at all.

But this will bring in all the benefits of pnpm to React Native world:

1. Promised strictness
2. Works with workspaces
3. Merely substitutues lerna
4. Unified experience
5. Can fallback to symlinks whenever they will be supported by Metro

[likern]

Eventually on my machine all node_modules take up only

find . -type d -name node_modules |  grep -v node_modules.*node_modules | tr '\n' '\0' |  xargs -0 du -sch
30G total

we can deal with it.

[zkochan]

What you suggest would require months of development. Just to support a community that doesn't support us.

The only solution for this would be to use Yarn's Plug'n'Play. At some point, we will probably support it. And with that, symlinks are not needed.

[aleclarson]

@likern You could try using Haul, an alternative to Metro. They claim to support symlinks. I've not tried it myself, as I'm using the PRs that are stalled. Another option, I could generate some git patches from my PRs to both jest and metro, which you can apply with patch-package. But then you would be stuck on the version of Metro that I'm using (the Jest changes are up-to-date, I think).

[likern]

@likern You could try using Haul, an alternative to Metro. They claim to support symlinks. I've not tried it myself, as I'm using the PRs that are stalled. Another option, I could generate some git patches from my PRs to both jest and metro, which you can apply with patch-package. But then you would be stuck on the version of Metro that I'm using (the Jest changes are up-to-date, I think).

Yes, but they do not support Fast Refresh. Which for me is more important feature.

[likern]

... I'm using the PRs that are stalled.

Might be you could publish your fork of Metro with patches applied?
So other devs could just swap original Metro to patched version?

[sammarks]

@aleclarson do you have the time to generate those patches for patch-package? That would be tremendously useful while we wait for the PRs to be merged!

[zkochan]

Does React Native work with Yarn Plug'n'Play?
If yes, you can now create a .pnp.js file from a pnpm lockfile. Also, we plan to integrate PnP more to pnpm: #2902

[merceyz]

It does not, they rely on the node_modules layout so much Yarn v1 had to implement nohoist for it to even work. v2 will have something similar, but better, in the next release yarnpkg/berry#1819

[vjpr]

I spent a lot of time investigating this issue over the years. Pnpm works fine with React Native. There is a small patch to Métro needed and then you just use a custom module resolver - webpack's 'enhanced-resolve'.

There were some small patches in jest waiting to merge last time I checked a couple months ago.

When I have some free time I will create a robust example project.

[billouboq]

@vjpr Do you have a snippet of your code to make it work with pnpm ?
That would be awesome ! 👏

[ShivamJoker]

@vjpr it's been 2 months please post something

[vjpr]

Will post tomorrow!

…

On Thu 3. Dec 2020 at 16:31, Shivam ***@***.***> wrote: @vjpr <https://github.com/vjpr> it's been 2 months please post something — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1252 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACEWRKCGL7TMPORSG7XKKTSS6VLZANCNFSM4FH7QWIQ> .

[marudy]

@vjpr if you can post that buddy, you will save our lives :) I'm super excited to try out yarn berry or pnpm in order to improve npm install performance but it's unfortunate that none does that yet :/

[marudy]

@vjpr sorry for spamming you man but it seems you have done the work already and i'm trying to gather the pieces to make pnpm to work for my React Native project as well. This is the metro patch you are referring to I suppose: #1051 correct ?

[vjpr]

Diving back into this now. Want to try and solve it for good because it takes a lot of mental effort to wrap my head around all the pieces each time I return to it. Stay tuned.

[vjpr]

@marudy @ShivamJoker @adelarsq Please see #3010

Working example: https://github.com/vjpr/pnpm-expo-example

[airtonix]

Eventually on my machine all node_modules take up only

find . -type d -name node_modules |  grep -v node_modules.*node_modules | tr '\n' '\0' |  xargs -0 du -sch
30G total

we can deal with it.

not in CICD we can't.

[zkochan]

For anyone finding this issue through search. The easiest way to make React Native work with pnpm is to use the node-linker=hoisted setting. With this setting pnpm will create a flat node_modules without using symlinks. Related docs: https://pnpm.io/npmrc#node-linker

[byCedric]

In addition to the node-linker=hoisted, I want to mention the Metro config outlined in the Expo docs "Working with monorepos".

I tried to explain most of the issues you probably will run into with monorepos, React Native, and/or Metro. With that context, you should be able to solve it in your preferred way (e.g. rnx-kit, or just adopt the same Metro config properties). There is also this repository that uses Expo, pnpm, and turborepo.