[WIP] Upgrade entire suite to Babel 7 #6949

milesj · 2018-09-05T19:46:07Z

🚨 Note that this PR is not needed for projects using Jest to use Babel 7 🚨

Summary

Had a bit of downtime so thought I would look into this a bit. At the moment, the migration was easy, but there are a few troublesome deps that are blocking this.

Resolve babel-plugin-transform-inline-imports-commonjs incompatabilities. (Switched to @babel/plugin-transform-modules-commonjs + lazy mode).
Resolve babel-preset-react-native incompatabilities. (Switched to metro-react-native-babel-preset).
Resolve react-native (Using 0.57).

Test plan

In progress.

milesj · 2018-09-05T19:55:21Z

packages/jest-runtime/src/__tests__/test_root/RegularModule.js

-if (exports !== this) {
-  throw new Error('Invalid module context');
-}
+// if (exports !== this) {


TODO, bring back.

SimenB · 2018-09-06T09:39:37Z

e2e/snapshot-escape/__tests__/snapshot.test.js

@@ -8,3 +8,4 @@

 // prettier-ignore
 test('escape strings', () => expect('one: \\\'').toMatchSnapshot());
+test('escape strings two', () => expect('two: \'"').toMatchSnapshot());


should not be commited

SimenB · 2018-09-06T09:41:07Z

scripts/build.js

-const transformOptions = JSON.parse(
-  fs.readFileSync(path.resolve(__dirname, '..', '.babelrc'), 'utf8')
-);
+const transformOptions = require(path.resolve(


require('../babel.config.js')

SimenB · 2018-09-06T09:42:08Z

I'm a fan of this, thanks for working on it! Would be nice to just drop babel 6 and not worry about it in a major.

@cpojer @mjesun @thymikee @rickhanlonii thoughts on that? Not sure if that works out for FB?

mjesun · 2018-09-06T09:58:33Z

❤️ We're on 7, so feel free to go! That'd probably be a major, so we can also open the gates for breaking changes 😆

milesj · 2018-09-06T18:56:35Z

babel.config.js

+      '@babel/preset-env',
+      {
+        shippedProposals: true,
+        targets: {node: '6'},


What's the lowest supported node version for Jest?

It's 6 now.

SimenB · 2018-09-07T08:02:43Z

Resolve babel-plugin-transform-inline-imports-commonjs incompatabilities

We can use the official babel plugin, it has a lazy mode: https://babeljs.io/docs/en/babel-plugin-transform-modules-commonjs#lazy. Not sure if the env preset supports it as an option or not

loganfsmyth · 2018-09-07T20:50:51Z

packages/babel-jest/src/index.js

 import jestPreset from 'babel-preset-jest';
-import {transform as babelTransform, util as babelUtil} from 'babel-core';


Is it possible for process to be async? Babel now exposes an async version of this function so that in the future we can start optionally allowing plugins some amount of async initialization. The more tooling that uses the async transforms, the better.

In our case no, as we call it through require which must be synchronous

loganfsmyth · 2018-09-07T20:53:06Z

packages/jest-runtime/src/script_transformer.js

@@ -20,7 +20,7 @@ import path from 'path';
 import vm from 'vm';
 import {createDirectory} from 'jest-util';
 import fs from 'graceful-fs';
-import {transform as babelTransform} from 'babel-core';
+import {transform as babelTransform} from '@babel/core';


FYI, babelTransform may return null if the file being compiled has been ignoreed in Babel's config. It looks like this transformer doesn't handle that case. Looks like you'd want to return content; in that case.

Submitting this again since GitHub's review system is still pretty wonky...

I think we handle it, see https://github.com/facebook/jest/blob/025c6af5e4bd2a971f295fe9408f3d8c77ad187b/packages/jest-runtime/src/script_transformer.js#L214-L234

We only assign to transformed if the transform returns something

Oh you're right. The line I was worried about is https://github.com/facebook/jest/blob/025c6af5e4bd2a971f295fe9408f3d8c77ad187b/packages/jest-runtime/src/script_transformer.js#L155, but that actually shouldn't ever have any ignores anyway. There is an issue with that call though, so I'll add a comment down there.

loganfsmyth · 2018-09-07T20:55:44Z

scripts/build.js


-const transformOptions = JSON.parse(


Here, my recommendation would be

const transformOptions = { cwd: path.resolve(__dirname, ".."), // configFile: "babel.config.js" // optional, since the `cwd` is the root and it will autodetect. babelrc: false, };

milesj · 2018-09-10T06:54:12Z

Been out of town the last few days but thanks for the tips! Will implement these.

SimenB

Thanks for chiming in @loganfsmyth!

SimenB · 2018-09-10T08:34:51Z

@milesj 2 changes I'd love to see (which don't have to come as part of this PR) is:

Add caller to our invocations of babel so it can handle module transform (and maybe future things) automatically: Allow preset-env to toggle module handling based on flags from the caller (like babel-loader) babel/babel#8485
Use loadPartialConfig to load the config for cache busting instead of looking up .babelrcs and babel.config.jss manually (and then pass that into the transform if possible so babel doesn't have to look it up again): Expose the partial Babel config for callers to load and mutate. babel/babel#7472

@loganfsmyth thoughts on those 2?

milesj · 2018-09-10T18:32:47Z

packages/babel-jest/src/index.js

  delete options.cacheDirectory;
  delete options.filename;

+  const loadBabelOptions = filename =>


@SimenB Updated to use loadPartialConfig.

please add supportsStaticESM: false as well

milesj · 2018-09-10T18:55:41Z

I'm getting a lot of spurious test failures locally, especially for stuff like Mercurial and Haste. I've tried installing the hg binary, but that doesn't resolve it. Are there docs on setting up the test suite locally?

Also can't seem to get Rollup to build no matter what config I try.

Edit: Got Rollup working, had to set configFile: false.

SimenB · 2018-09-10T21:14:35Z

Are there docs on setting up the test suite locally?

Not beyond installing mercury as you've already tried. What failures are you getting?

milesj · 2018-09-10T21:27:37Z

Stuff like this:

 FAIL  packages/jest-haste-map/src/__tests__/index.test.js
  ● HasteMap › builds a haste map on a fresh cache

    TypeError: Cannot read property 'clocks' of null

      at hasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:272:19)

  ● HasteMap › builds a haste map on a fresh cache with SHA-1s › uses watchman: false

    TypeError: Cannot read property 'files' of null

      at Object.<anonymous> (packages/jest-haste-map/src/__tests__/index.test.js:359:21)

  ● HasteMap › builds a haste map on a fresh cache with SHA-1s › uses watchman: true

    TypeError: Cannot read property 'files' of null

      at Object.<anonymous> (packages/jest-haste-map/src/__tests__/index.test.js:359:21)

  ● HasteMap › does not crawl native files even if requested to do so

    TypeError: Cannot read property 'map' of null

      at Object.<anonymous> (packages/jest-haste-map/src/__tests__/index.test.js:419:17)

  ● HasteMap › retains all files if `retainAllFiles` is specified

    TypeError: Cannot read property 'files' of null

      at hasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:441:19)

  ● HasteMap › warns on duplicate module ids

    TypeError: Cannot read property 'map' of null

      at HasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:492:21)

  ● HasteMap › splits up modules by platform

    TypeError: Cannot read property 'files' of null

      at HasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:542:21)

  ● HasteMap › does not access the file system on a warm cache with no changes

    TypeError: Cannot read property 'clocks' of null

      at HasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:598:25)

  ● HasteMap › only does minimal file system access when files change

    TypeError: Cannot read property 'clocks' of null

      at HasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:638:25)

  ● HasteMap › correctly handles file deletions

    TypeError: Cannot read property 'files' of null

      at HasteMap.build.then (packages/jest-haste-map/src/__tests__/index.test.js:674:46)

milesj · 2018-09-10T21:36:49Z

packages/jest-editor-support/src/parsers/babylon_parser.js

 import {Expect, ItBlock} from './parser_nodes';
+// TODO: Change import to @babel/parser once types exist
 import type {File as BabylonFile} from 'babylon';

 export type BabylonParserResult = {


@loganfsmyth @SimenB In Babylon, we were passing plugins: ['*'], but this was removed in Babel 7. What's the best way to infer the plugin list from the Babel config? Since I can't hardcode all of these plugins because flow vs typescript, etc.

@orta thoughts?

We've added a babel.parse that takes all of Babel's normal options and will load your plugins and such to decide how to parse, so that's the recommendation for this type of thing now.

One thing to note is, like my other comment, this will return null if the file is ignoreed in the Babel config. Your call if you want to then fall back to something else or parse with @babel/parser with no plugins or what.

loganfsmyth · 2018-09-11T00:41:12Z

packages/jest-runtime/src/script_transformer.js

-    }).code;
+    });
+
+    return result ? result.code : content;


As noted in #6949 (comment), I was actually mistaken about needing to handle null values here.

Turns out this is required. Some tests broke after not checking for null.

loganfsmyth · 2018-09-11T00:42:06Z

packages/jest-runtime/src/script_transformer.js

@@ -152,9 +152,10 @@ export default class ScriptTransformer {
  }

  _instrumentFile(filename: Path, content: string): string {
-    return babelTransform(content, {
+    const result = babelTransform(content, {
      auxiliaryCommentBefore: ' istanbul ignore next ',
      babelrc: false,


This call should have a configFile: false alongside the babelrc: false on it or else it risks applying the user's project-wide config during instrumentation.

@loganfsmyth Perhaps you can help me debug the failing test for this, as it's currently throwing this.

● ScriptTransformer › writes source map if preprocessor supplies it TypeError: Cannot read property 'mtime' of undefined at fileMtime (node_modules/@babel/core/lib/config/files/utils.js:39:45)

While debugging, I noticed that console logs before the babelTransform would display, but not after. I also noticed that /fruits/package.json is being statSyncd (via makeStaticFileCache), yet that path is NOT defined within the tests. That leads me to believe that it's caused by Babel, or the Istanbul plugin, but after debugging the plugin, it's never even hit. At this point I'm leaning towards it being a Babel config loading problem, but both babelrc and configFile are false, so I'm a bit lost here.

Is something mocking statSync maybe? That function should always return an object, or throw: https://github.com/babel/babel/blob/13798feefbf19389cb692aa0b5157276ce7cdf4f/packages/babel-core/src/config/files/utils.js#L21

@loganfsmyth We do mock statSync, but the weird thing is that /fruits/package.json returns undefined while all other calls to statSync return the mock. It's very confusing.

Documenting for future debugging.

When I console log fs before this line https://github.com/babel/babel/blob/13798feefbf19389cb692aa0b5157276ce7cdf4f/packages/babel-core/src/config/files/utils.js#L21, I can successfully see the entire module being mocked. However, when /fruits/package.json is statSyncd, the value is no longer mocked and undefined is returned. I'm leaning towards this being an issue with the make*Cache calls. Perhaps the cache is interfering?

loganfsmyth · 2018-09-11T01:03:30Z

packages/babel-jest/src/index.js

@@ -18,75 +18,34 @@ import type {
 import crypto from 'crypto';
 import fs from 'fs';
 import path from 'path';
+import {
+  transform as babelTransform,
+  util as babelUtil,


These utils don't exist in 7.x FYI.

@SimenB Remove the util functionality completely? I noticed the canCompile function being set in a few places.

I suppose? It was probably an optimization, but I guess it doesn't matter as it's been removed

All it did was check our list of suggested list of extensions, so we just exposed that instead: https://github.com/babel/babel/blob/master/packages/babel-core/src/index.js#L37 I'm not sure it's that useful for Jest anyway though, since you already limit it to .js files by default.

loganfsmyth · 2018-09-11T01:04:53Z

packages/babel-jest/src/index.js

@@ -95,19 +54,22 @@ const createTransformer = (options: any): Transformer => {
      configString: string,
      {instrument, rootDir}: CacheKeyOptions,
    ): string {
+      const babelOptions = loadBabelOptions(filename);
+      const configPath = babelOptions.config || babelOptions.babelrc || '';


Both of these could exist at the same time, so probably best to just hash both.

SimenB · 2018-09-11T06:48:53Z

docs/TutorialReact.md

-      });
-    }
-    return src;
+    return transform(src, {


if the return-value is falsy, this should return src

SimenB · 2018-09-11T06:49:47Z

e2e/babel-plugin-jest-hoist/.babelrc

-    }
-  ],
+  "presets": ["@babel/preset-env", "@babel/preset-flow"],
+  "plugins": ["jest-hoist"],
  "retainLines": true


can you remove this? shouldn't be needed

milesj · 2018-09-21T01:15:23Z

I'll be out of town for the next few weeks, so won't be around to work on this. Feel free to merge it into the branch.

SimenB · 2018-09-21T06:23:37Z

Ok, let's do it then.

SimenB · 2018-09-21T06:28:35Z

Opened up #7016 to have a place to keep discussing the code in context. Anyone wanting to help can open up PRs against the same branch, and it'll show up in that PR as we merge 🙂

github-actions · 2021-05-12T09:06:42Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

milesj requested a review from orta as a code owner September 5, 2018 19:46

facebook-github-bot added the cla signed label Sep 5, 2018

milesj commented Sep 5, 2018

View reviewed changes

SimenB reviewed Sep 6, 2018

View reviewed changes

matthamil mentioned this pull request Sep 6, 2018

Update babel-core dependency to v7 for jest-config #6926

Closed

milesj commented Sep 6, 2018

View reviewed changes

milesj mentioned this pull request Sep 6, 2018

WIP: Support Babel 7 zertosh/babel-plugin-transform-inline-imports-commonjs#11

Closed

SimenB mentioned this pull request Sep 7, 2018

jest-runtime: move babel-core to peer dependencies #4557

Merged

loganfsmyth reviewed Sep 7, 2018

View reviewed changes

SimenB reviewed Sep 10, 2018

View reviewed changes

DBosley mentioned this pull request Sep 10, 2018

Requires Babel "^7.0.0-0", but was loaded with "6.26.3". #6913

Closed

milesj commented Sep 10, 2018

View reviewed changes

loganfsmyth reviewed Sep 11, 2018

View reviewed changes

SimenB reviewed Sep 11, 2018

View reviewed changes

docs/TutorialReact.md Outdated

});

}

return src;

return transform(src, {

Copy link

Member

SimenB Sep 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the return-value is falsy, this should return src

SimenB reviewed Sep 11, 2018

View reviewed changes

Miles Johnson and others added 21 commits September 20, 2018 17:43

Update RN support.

7ea038b

Rework Babel config loading.

c7232be

Update deps.

3b0f790

Update some deps.

c1a9a2c

Fix rollup build.

a28d236

Remove browser build plugin.

7dd89ee

Target IE8.

dde23de

Update to beta RN.

0e6cfda

Parser updates.

dab8e57

Upgrade istanbul.

9abd68b

Rework lazy.

141c901

More work and polish.

ddb7d28

Use transformSync.

6f7d4ab

Update react-native.

765a38b

Fix RN tests.

3927ce4

Final fixes.

34e0e44

Minor changes.

e0cd570

Use babel for TS.

826f29f

Update deps. Run install.

0deb91e

Update editor support.

294ea06

Add TS argument.

8e3a39d

SimenB merged commit e47e545 into jestjs:babel-7 Sep 21, 2018

milesj deleted the babel7 branch September 21, 2018 17:20

connectdotz mentioned this pull request Oct 18, 2018

Babel 7 #7016

Merged

6 tasks

connectdotz mentioned this pull request Jan 26, 2020

update babylon to babel/parser 7.x jest-community/jest-editor-support#30

Merged

github-actions bot locked as resolved and limited conversation to collaborators May 12, 2021

		import jestPreset from 'babel-preset-jest';
		import {transform as babelTransform, util as babelUtil} from 'babel-core';

[WIP] Upgrade entire suite to Babel 7 #6949

[WIP] Upgrade entire suite to Babel 7 #6949

Conversation

milesj commented Sep 5, 2018 • edited by SimenB

🚨 Note that this PR is not needed for projects using Jest to use Babel 7 🚨

Summary

Test plan

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimenB commented Sep 6, 2018 • edited

mjesun commented Sep 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimenB commented Sep 7, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milesj commented Sep 10, 2018

SimenB left a comment

Choose a reason for hiding this comment

SimenB commented Sep 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milesj commented Sep 10, 2018 • edited

SimenB commented Sep 10, 2018

milesj commented Sep 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

loganfsmyth Sep 11, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milesj Sep 12, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milesj commented Sep 21, 2018

SimenB commented Sep 21, 2018

SimenB commented Sep 21, 2018

github-actions bot commented May 12, 2021

milesj commented Sep 5, 2018 •

edited by SimenB

SimenB commented Sep 6, 2018 •

edited

SimenB commented Sep 7, 2018 •

edited

milesj commented Sep 10, 2018 •

edited

loganfsmyth Sep 11, 2018 •

edited

milesj Sep 12, 2018 •

edited