Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Yarn PnP, Module not found: Error: Can't resolve 'x' in 'y' packageExtensions binaries [arm, android win32-x64, darwin, gnu, linux] #93

Open
1 task done
gruckion opened this issue Sep 26, 2022 · 0 comments
Open
1 task done

Yarn PnP, Module not found: Error: Can't resolve 'x' in 'y' packageExtensions binaries [arm, android win32-x64, darwin, gnu, linux] #93

gruckion opened this issue Sep 26, 2022 · 0 comments

Comments

@gruckion
Copy link

Self-service

  • I'd be willing to implement a fix

Describe the bug

Mongodb has dependencies that are not listed as peer dependencies. I tried to use packageExtensions to get around this but I am now getting dozens more errors from two libraries Snappy and @mongodb-js/zstd.

➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:622-662
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-android-arm64' in '/Users/sigex/workdir/backend/lambda/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'

➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 89:28-70
➤ YN0000:   Module not found: Error: Can't resolve './zstd.linux-arm-gnueabihf.node' in '/Users/sigex/workdir/backend/lambda/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'

It looks like it's trying to require all these different "binaries"? for different machines.

My packageExtensions is below and it's managed to get most of the warnings to shut up.

packageExtensions:
  "mongodb@*":
    dependencies:
      bson-ext: "*"
      kerberos: "*"
      "@mongodb-js/zstd": "*"
      aws4: "*"
      mongodb-core: "*"
      bson: "*"
      node-gyp: "*"
      "mongodb-client-encryption": "*"
    peerDependenciesMeta:
      "@mongodb-js/zstd":
        optional: true
      snappy:
        optional: true
      bson-ext:
        optional: true
      kerberos:
        optional: true

To reproduce

I created a repo here, see branch mongodb-broken-pnp.

https://github.com/gruckion/yarn-berry-serverless-workspaces/tree/mongodb-broken-pnp

Run yarn; yarn package to reproduce the errors.

❯ yarn package
➤ YN0000: 
➤ YN0000: Packaging lambdas-hello for stage testing (us-east-1)
➤ YN0000: 
➤ YN0000: Warning: in ../../.yarn/__virtual__/mongodb-virtual-904dd890e4/0/cache/mongodb-npm-4.10.0-99903c2c4a-4847fe69b6.zip/node_modules/mongodb/lib/utils.js 993:38-93
➤ YN0000:   Critical dependency: the request of a dependency is an expression
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 17:28-65
➤ YN0000:   Module not found: Error: Can't resolve './zstd.win32-x64-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 19:28-70
➤ YN0000:   Module not found: Error: Can't resolve '@mongodb-js/zstd-win32-x64-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 35:28-61
➤ YN0000:   Module not found: Error: Can't resolve './zstd.darwin-x64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 37:28-66
➤ YN0000:   Module not found: Error: Can't resolve '@mongodb-js/zstd-darwin-x64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 47:28-63
➤ YN0000:   Module not found: Error: Can't resolve './zstd.darwin-arm64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 65:28-64
➤ YN0000:   Module not found: Error: Can't resolve './zstd.linux-x64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 67:28-69
➤ YN0000:   Module not found: Error: Can't resolve '@mongodb-js/zstd-linux-x64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 77:28-66
➤ YN0000:   Module not found: Error: Can't resolve './zstd.linux-arm64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 79:28-71
➤ YN0000:   Module not found: Error: Can't resolve '@mongodb-js/zstd-linux-arm64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 89:28-70
➤ YN0000:   Module not found: Error: Can't resolve './zstd.linux-arm-gnueabihf.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 91:28-75
➤ YN0000:   Module not found: Error: Can't resolve '@mongodb-js/zstd-linux-arm-gnueabihf' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:569-607
➤ YN0000:   Module not found: Error: Can't resolve './snappy.android-arm64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:622-662
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-android-arm64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:811-852
➤ YN0000:   Module not found: Error: Can't resolve './snappy.android-arm-eabi.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:867-910
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-android-arm-eabi' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1159-1198
➤ YN0000:   Module not found: Error: Can't resolve './snappy.win32-x64-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1213-1254
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-win32-x64-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1403-1443
➤ YN0000:   Module not found: Error: Can't resolve './snappy.win32-ia32-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1458-1500
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-win32-ia32-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1651-1692
➤ YN0000:   Module not found: Error: Can't resolve './snappy.win32-arm64-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1707-1750
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-win32-arm64-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1997-2032
➤ YN0000:   Module not found: Error: Can't resolve './snappy.darwin-x64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2047-2084
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-darwin-x64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2231-2268
➤ YN0000:   Module not found: Error: Can't resolve './snappy.darwin-arm64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2626-2662
➤ YN0000:   Module not found: Error: Can't resolve './snappy.freebsd-x64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2677-2715
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-freebsd-x64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2900-2939
➤ YN0000:   Module not found: Error: Can't resolve './snappy.linux-x64-musl.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2954-2995
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-linux-x64-musl' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3131-3169
➤ YN0000:   Module not found: Error: Can't resolve './snappy.linux-x64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3184-3224
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-linux-x64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3389-3430
➤ YN0000:   Module not found: Error: Can't resolve './snappy.linux-arm64-musl.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3445-3488
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-linux-arm64-musl' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3626-3666
➤ YN0000:   Module not found: Error: Can't resolve './snappy.linux-arm64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3681-3723
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-linux-arm64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3876-3920
➤ YN0000:   Module not found: Error: Can't resolve './snappy.linux-arm-gnueabihf.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:   
➤ YN0000:   in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3935-3981
➤ YN0000:   Module not found: Error: Can't resolve '@napi-rs/snappy-linux-arm-gnueabihf' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000: Environment: darwin, node 16.13.2, framework 3.22.0 (local), plugin 6.2.2, SDK 4.3.2
➤ YN0000: Docs:        docs.serverless.com
➤ YN0000: Support:     forum.serverless.com
➤ YN0000: Bugs:        github.com/serverless/serverless/issues
➤ YN0000: 
➤ YN0000: Error:
➤ YN0000: Webpack compilation failed:
➤ YN0000: 
➤ YN0000: in ../../.yarn/unplugged/@mongodb-js-zstd-darwin-arm64-npm-1.0.0-df74982a20/node_modules/@mongodb-js/zstd-darwin-arm64/zstd.darwin-arm64.node 1:0
➤ YN0000:   Module parse failed: Unexpected character '�' (1:0)
➤ YN0000:   You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
➤ YN0000:   (Source code omitted for this binary file)
➤ YN0000:   
➤ YN0000:   in ../../.yarn/unplugged/@napi-rs-snappy-darwin-arm64-npm-7.1.2-605592bf69/node_modules/@napi-rs/snappy-darwin-arm64/snappy.darwin-arm64.node 1:0
➤ YN0000:   Module parse failed: Unexpected character '�' (1:0)
➤ YN0000:   You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
➤ YN0000:   (Source code omitted for this binary file)
➤ YN0000: 
➤ YN0000: 1 deprecation found: run 'serverless doctor' for more details
➤ YN0000: Done in 17s 69ms

Environment

System:
    OS: macOS 13.0
    CPU: (10) arm64 Apple M1 Max
  Binaries:
    Node: 16.13.2 - /private/var/folders/mv/dtw39jks6ml64s3pd1pnv_640000gn/T/xfs-7dfef409/node
    Yarn: 3.2.3 - /private/var/folders/mv/dtw39jks6ml64s3pd1pnv_640000gn/T/xfs-7dfef409/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm

Additional context

Mongodb has created 3 PRs to "support PnP", but have written no documentation. But this issue is not specific to them, it is more about how we handle libraries that have missing peer dependencies that happen to be binaries.

https://github.com/mongodb/node-mongodb-native/pulls?q=pnp

I also posted a comment on this PR as Mongodb and Mongodb'sissue section on Jira.

mongodb/node-mongodb-native#2606
https://jira.mongodb.org/browse/NODE-4659

I am also using pnp-webpack-plugin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gruckion