expo@37 support

[vjpr]

Just keeping a log of all the difficulties encountered running expo with pnpm.

An existing issue has notes for expo@31 (#1501) but was a bit complicated, so I thought I would do a fresh one.

[vjpr]

NOTE: This thread has lots of good information: facebook/metro#1

Error: Unable to resolve module ./node_modules/expo/AppEntry from ``:

Full stack trace

Error: Unable to resolve module `./node_modules/expo/AppEntry` from ``:

None of these files exist:
  * node_modules/expo/AppEntry(.native|.ios.expo.ts|.native.expo.ts|.expo.ts|.ios.expo.tsx|.native.expo.tsx|.expo.tsx|.ios.expo.js|.native.expo.js|.expo.js|.ios.expo.jsx|.native.expo.jsx|.expo.jsx|.ios.ts|.native.ts|.ts|.ios.tsx|.native.tsx|.tsx|.ios.js|.native.js|.js|.ios.jsx|.native.jsx|.jsx|.ios.json|.native.json|.json|.ios.wasm|.native.wasm|.wasm)
  * node_modules/expo/AppEntry/index(.native|.ios.expo.ts|.native.expo.ts|.expo.ts|.ios.expo.tsx|.native.expo.tsx|.expo.tsx|.ios.expo.js|.native.expo.js|.expo.js|.ios.expo.jsx|.native.expo.jsx|.expo.jsx|.ios.ts|.native.ts|.ts|.ios.tsx|.native.tsx|.tsx|.ios.js|.native.js|.js|.ios.jsx|.native.jsx|.jsx|.ios.json|.native.json|.json|.ios.wasm|.native.wasm|.wasm)
    at ModuleResolver.resolveDependency (/Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/node-haste/DependencyGraph/ModuleResolution.js:164:15)
    at ResolutionRequest.resolveDependency (/Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/node-haste/DependencyGraph/ResolutionRequest.js:52:18)
    at DependencyGraph.resolveDependency (/Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/node-haste/DependencyGraph.js:282:16)
    at /Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/lib/transformHelpers.js:267:42
    at Server.<anonymous> (/Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/Server.js:1088:41)
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/Server.js:99:24)
    at _next (/Users/Vaughan/dev-mono/thirtyfive/node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/Server.js:119:9)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Solution

Copy ./node_modules/expo/AppEntry to the expo project dir, and change package.json#main to point to it.

---

See facebook/metro#1 (comment)

Add metro.config.js and add extraNodeModules.

Unable to resolve "@babel/runtime/helpers/interopRequireDefault" from "AppEntry.js"

Logged to terminal:

NOTE: The expo error logger is not able to handle the real error so it just shows Failed building JavaScript bundle.. See expo/expo-cli#206.

Unable to resolve "@babel/runtime/helpers/interopRequireDefault" from "AppEntry.js"
Failed building JavaScript bundle.

Logged to simulator:

Unable to resolve module `@babel/runtime/helpers/interopRequireDefault` from `AppEntry.js`: @babel/runtime/helpers/interopRequireDefault could not be found within the project or in these directories:
  node_modules/@babel/runtime/helpers

Debug Setup

Modify this file and then step-through metro-resolver#resolve code in WebStorm debugger. You will need to place breakpoint at the .js file instead of the .js.flow file.

NOTE: Make sure you use the same version of metro-resolver as the version of metro that uses.

metro.config.js

const resolver = require('metro-resolver')

{
  resolver: {
    resolveRequest: (ctx, moduleName, platform) => {
      console.log({moduleName})
      // See: https://github.com/facebook/metro/blob/master/packages/metro-resolver/src/resolve.js#L95
      delete ctx.resolveRequest
      return resolver.resolve(ctx, moduleName, platform)
    },
  }
}

Debug Journey

If we trace the resolution through we see that context.doesFileExist is returning false for a file that is clearly present on disk.

We set a conditional breakpoint on this function to break on extension === '.js'.

node_modules/.pnpm/registry.npmjs.org/metro-resolver/0.59.0/node_modules/metro-resolver/src/resolve.js

function resolveSourceFileForExt(context, extension) {
  const filePath = `${context.filePathPrefix}${extension}`;

  if (context.doesFileExist(filePath)) {
    return filePath;
  }

  context.candidateExts.push(extension);
  return null;
}

doesFileExist() is defined here:

node_modules/.pnpm/registry.npmjs.org/metro/0.56.4/node_modules/metro/src/node-haste/DependencyGraph.js:99

_defineProperty(this, "_doesFileExist", filePath => {
  return this._hasteFS.exists(filePath);
});

_hasteFS is a JestHasteMap, and if you trace through to _crawl which builds that haste map, you will see it uses watchman that does not support symlinks: facebook/watchman#105.

Now, we can disable watchman and use Node.js for building the haste map by setting metro.config.js#resolver.useWatchman.

However, there is a check in this implementation that prevents following symlinks - maybe to maintain parity with watchman?

node_modules/.pnpm/registry.npmjs.org/jest-haste-map/24.9.0/node_modules/jest-haste-map/build/crawlers/node.js

        _fs().default.lstat(file, (err, stat) => {
          activeCalls--;

          if (!err && stat && !stat.isSymbolicLink()) {
            if (stat.isDirectory()) {
              search(file);
            } else {

NOTE: stat.isDirectory() returns false because they are using lstat which doesn't follow the symlink.

If we patch it to the following...then it works!

        //_fs().default.lstat(file, (err, stat) => {
        _fs().default.stat(file, (err, stat) => {
          activeCalls--;

          //if (!err && stat && !stat.isSymbolicLink()) {
          if (!err && stat) {
            if (stat.isDirectory()) {
              search(file);
            } else {

Also see here: facebook/metro#330 (comment)

By setting metro.config.js#watchFolders you can have your pnpm monorepo added to Haste. One problem though is that when resolving a file, when it checks whether the file exists it will look it up in the haste map by its logical path relative to the project rootDir, but it only exists in the haste map by its realpath.

node_modules/.pnpm/registry.npmjs.org/jest-haste-map/24.9.0/node_modules/jest-haste-map/build/HasteFS.js

  _getFileData(file) {
    // file => logical
    // this._files => contains only realpaths
    const relativePath = fastPath.relative(this._rootDir, file);
    return this._files.get(relativePath);
  }

NOTE: When loading file data from the haste map, you can use absolute path, or a project root relative path.

NOTE: There is a HasteFS file map and a Module Map. The module map is for tracking our modules (those that are not in node_modules). Every request for a file will go through the HasteFS file map which is causing problems.

---

Metro resolver

First it checks the haste map.

const result = resolveHasteName(context, normalizedName, platform);

const modulePath = context.resolveHasteModule(moduleName);
...
let packageJsonPath = context.resolveHastePackage(packageName);

this._options.moduleMap.getModule(name, platform, true),

moduleMap does not contain any packages inside node_modules because when the crawl is going on there is this block of code which just gets the sha1 without adding them as packages/modules.

_jest-haste-map/index.js#processFile

    if (this._options.retainAllFiles && this._isNodeModulesDir(filePath)) {
      if (computeSha1) {
        return this._getWorker(workerOptions)
          .getSha1({
            computeDependencies: this._options.computeDependencies,
            computeSha1,
            dependencyExtractor: this._options.dependencyExtractor,
            filePath,
            hasteImplModulePath: this._options.hasteImplModulePath,
            rootDir
          })
          .then(workerReply, workerError);
      }

      return null;
    }

[vjpr]

jestjs/jest#10063

jest-haste-map ignores files and dirs starting with dots, preventing them being included in the haste map which causes "SHA-1 missing" errors (facebook/metro#330).

Took me so long to track this down and it turns out that it is a one-line fix.

"glob_includedotfiles": true needs to be added to jest-haste-map/build/crawlers/watchman.js or https://github.com/facebook/jest/blob/v24.9.0/packages/jest-haste-map/src/crawlers/watchman.ts#L130. This PR should be merged soon but will take a while to make it into expo because of the transitive dependency chain.

[zkochan]

Looks like this library help with react native, no?
https://github.com/microsoft/rnx-kit/tree/main/packages/metro-resolver-symlinks