feat(gatsby): enable babel-loader for all dependencies #14111

wardpeet · 2019-05-17T09:33:47Z

Description

I ran gatsby through babel-loader to make sure babel-preset-env can adds it's polyfills for all dependencies so we don't break on older browsers. This will also help us for modern builds so our modern build will have fewer polyfills than our classic one.

To give you an idea on how much slower it is. The first run on a big website takes about double so that's something to wonder about.

default-starter	no babel cache	babel cache
transpile node_modules (no babel-cache)	5,5623s	2,887s
exclude node_modules (no babel-cache)	3,148s	2,224s
diff	76%	29%

gatsbyjs.org	no babel cache	babel cache
transpile node_modules (no babel-cache)	31,487s	14,684s
exclude node_modules (no babel-cache)	15,896s	10,804s
diff	98%	36%

default-starter	runtime 1	runtime 2	runtime 3
transpile node_modules (no babel-cache)	5,198s	5,768s	5,721s
transpile node_modules (babel-cache)	2,771s	2,958s	2,932s
exclude node_modules (no babel-cache)	2,937s	3,339s	3,168s
exclude node_modules (babel-cache)	2,172s	2,283s	2,217s

gatsbyjs.org	runtime 1	runtime 2	runtime 3
transpile node_modules (no babel-cache)	30.013s	32.846s	31.602s
transpile node_modules (babel-cache)	14.761s	15.519s	13.771s
exclude node_modules (no babel-cache)	15.686s	16.385s	15.617s
exclude node_modules (babel-cache)	12.336s	10.532s	9.545s

bundle info

http://wardpeet-filestorage.surge.sh/gatsby-store/pr-14111

Related Issues

#12409
#12399
#13427
#5553
#7064
#13241

wardpeet · 2019-05-17T09:35:21Z

packages/gatsby/src/utils/webpack-utils.js

@@ -292,7 +292,7 @@ module.exports = async ({
    let js = (options = {}) => {
      return {
        test: /\.jsx?$/,
-        exclude: vendorRegex,
+        exclude: /core-js/,


Somehow core-js needs to be excluded because babel will try to polyfill core-js with core-js XD also mini-extra-css-plugin doesn't work well without this. This magically fixes it, i'm not 100% sure so the above comments are just speculation 😄 .

I'll add a comment when it's getting out of draft mode.

will try to polyfill core-js with core-js

😂

sidharthachatterjee · 2019-05-17T09:37:43Z

This will finally allow dependencies (themes and others) to have queries! Woohoo 🚀

sidharthachatterjee · 2019-05-17T09:42:30Z

Want to also post diff of bundle size from before and after? Curious how much this adds or removes

sidharthachatterjee · 2019-05-17T09:52:04Z

Can also remove note in https://www.gatsbyjs.org/docs/browser-support/#polyfills

wardpeet · 2019-05-17T12:12:00Z

I've added a list of changes that this PR creates. It only adds 12kb to gatsbyjs.org which is extremely low minified so I guess we should just get this in and reduce it again when using modern builds.

Babel all the things!

http://wardpeet-filestorage.surge.sh/gatsby-store/pr-14111/

wardpeet · 2019-06-07T20:59:00Z

I've edited above comment with new elements! Lets get this in! ^^

KyleAMathews · 2019-06-08T20:07:55Z

💯yeah, let's do this! The extra build time is a bummer but so is sites not working on older browsers.

Does this PR disable transpiling node_modules during development? That'd be something to consider as people are more sensitive to build times when trying to start the develop server vs. doing a production build (which probably is happening on a CI server somewhere)

wardpeet · 2019-06-11T12:15:34Z

I've found a bug on develop where somehow react-hot-loader is causing issues. trying to find out why

wardpeet · 2019-06-11T14:05:25Z

seems like I needed to add event-source-polyfill & webpack-hot-middleware/client as we don't want them to get polyfilled. As they act as HMR polyfills. If we don't exclude we get some core-js loops and errors.

wardpeet · 2019-06-25T15:21:00Z

I haven't tested the latest change, I'll do that tomorrow

sidharthachatterjee

I think this is ready to go now! Thank you @wardpeet ❤️

wardpeet · 2019-06-27T06:35:15Z

Published in gatsby@2.11.0

monsieurnebo · 2019-06-27T08:40:21Z

That's great, thanks for this @wardpeet !

Because of gatsbyjs#14111, we can remove the custom theme processing.

jquense · 2019-06-27T19:01:52Z

this is kinda of a breaking change for sites with custom babel configs ya'll

DSchau · 2019-06-27T19:30:18Z

@jquense yikes —- we’re sorry to hear that!

Would you mind opening up an issue and showing us exactly how this is breaking?

jquense · 2019-06-27T20:00:49Z

It wasn't a huge deal for us, but it's not obvious what was happening. There isn't a lot to how it breaks unfortunately. It's just not safe to apply any babel config to any code.

For instance: babel-dev-expression breaks trying to compile warning

joealden · 2019-06-28T14:07:10Z

@wardpeet the increased build times are a pretty big issue for us, as our build times were long enough as they were. They have gone from about 6 minutes all the way up to 11 minutes (and rebuilds are triggered quite regularly). As we are paying for compute time per minute spent building, this drives up the costs for CI builds to nearly double.

While I understand why the changes in this PR were introduced, would it not be possible to have an option to turn this off (in gatsby-config.js or as a env var)?

wardpeet · 2019-06-28T14:51:34Z

@joealden thanks, i've created an issue about this #15213

Gatsby@2.11.0 Build failure gatsbyjs/gatsby#15190 (comment) > gatsby-plugin-netlify-cms runs an independent Webpack build within > Gatsby's Webpack build (insert "yo dawg" reference here) using a > modified version of the Gatsby Webpack config. Gatsby 2.11.0 no longer > excludes node_modules, which causes the plugin's Webpack build to attempt > processing the Netlify CMS modules with Babel. The modules are massive > and also prebuilt, so it will (and should) crash your system. > The PR I raised fix(gatsby-plugin-netlify-cms): exclude node_modules from cms builds gatsbyjs/gatsby#15191 > just adds that exclusion back in to the plugin's Webpack config. > Please don't override your node max file size limit to fix this! feat(gatsby): enable babel-loader for all dependencies gatsbyjs/gatsby#14111

Because of gatsbyjs#14111, we can remove the custom theme processing.

wardpeet commented May 17, 2019

View reviewed changes

m-allanson mentioned this pull request May 20, 2019

Investigate compiling node_modules code #7064

Closed

wardpeet added this to In progress in OSS Roadmap via automation Jun 4, 2019

wardpeet added the status: awaiting author response Additional information has been requested from the author label Jun 4, 2019

wardpeet added status: awaiting reviewer response A pull request that is currently awaiting a reviewer's response and removed status: awaiting author response Additional information has been requested from the author labels Jun 7, 2019

wardpeet marked this pull request as ready for review June 7, 2019 22:18

wardpeet requested a review from a team as a code owner June 7, 2019 22:18

pieh mentioned this pull request Jun 10, 2019

chore(gatsby): add ie polyfill to webpack entry #13241

Closed

wardpeet requested review from a team June 11, 2019 14:01

wardpeet requested review from a team as code owners June 11, 2019 14:01

wardpeet mentioned this pull request Jun 11, 2019

Gatsby's Use Of Polyfills and the 'Missing Resources for x' error #12399

Closed

wardpeet force-pushed the feat/babel-transpile-all branch from e943596 to bd335a9 Compare June 14, 2019 10:56

wardpeet added 4 commits June 25, 2019 15:29

feat(gatsby): enable babel-loader for all dependencies

f410ddb

fix excludes for develop

0563135

also compile mjs files

d46f17b

fix tests

d8df2e0

wardpeet force-pushed the feat/babel-transpile-all branch from bd335a9 to d8df2e0 Compare June 25, 2019 13:38

only transpile node_modules on production

4210885

sidharthachatterjee approved these changes Jun 26, 2019

View reviewed changes

wardpeet merged commit 268ed27 into gatsbyjs:master Jun 27, 2019

OSS Roadmap automation moved this from In progress to Done Jun 27, 2019

wardpeet deleted the feat/babel-transpile-all branch June 27, 2019 06:19

sidharthachatterjee mentioned this pull request Jun 27, 2019

feat(gatsby): Handle duplicated fragment definitions #15179

Merged

2 tasks

smakosh mentioned this pull request Jun 27, 2019

upgrade gatsby smakosh/old-smakosh.com#26

Merged

ChristopherBiscardi added a commit to ChristopherBiscardi/gatsby that referenced this pull request Jun 27, 2019

Remove duplicate theme .js processing

159d5c6

Because of gatsbyjs#14111, we can remove the custom theme processing.

moonmeister mentioned this pull request Jun 27, 2019

Gatsby@2.11.0 Build failure (JavaScript heap out of memory) #15190

Closed

erquhart mentioned this pull request Jun 27, 2019

fix(gatsby-plugin-netlify-cms): exclude node_modules from cms #15191

Merged

wardpeet mentioned this pull request Jun 28, 2019

Making babel node_modules transpilation better #15213

Closed

wardpeet mentioned this pull request Jul 1, 2019

fix(*): enable separate config for node_modules transpilation #15270

Merged

ChristopherBiscardi added a commit to ChristopherBiscardi/gatsby that referenced this pull request Jul 1, 2019

Remove duplicate theme .js processing

79da69b

Because of gatsbyjs#14111, we can remove the custom theme processing.

grubersjoe mentioned this pull request Jul 2, 2019

Netlify wont build. Building gatsby-image and production js / css bundles now takes substantially longer time. #15278

Closed

harmon25 mentioned this pull request Jul 5, 2019

Build fail with heap out of memory error in gatsby 2.11.6 #15256

Closed

johno pushed a commit to johno/gatsby that referenced this pull request Jul 17, 2019

feat(gatsby): enable babel-loader for all dependencies (gatsbyjs#14111)

fdd44d5

wardpeet mentioned this pull request Aug 5, 2019

node_module transpile causing errors in develop mode #16396

Closed

This was referenced Aug 9, 2019

JavaScript heap out of memory wKovacs64/gatsby-plugin-use-dark-mode#23

Closed

Build failing with heap out of memory error, gatsby@>=2.11.0 #16499

Closed

StarpTech mentioned this pull request Aug 20, 2019

Optimize Next.js polyfilling behavior (Babel) vercel/next.js#8303

Closed

toolness mentioned this pull request Oct 12, 2019

Use @justfixnyc/geosearch-requester JustFixNYC/justfix-website#40

Merged

trevorblades mentioned this pull request Jan 29, 2020

fix: Make the Gatsby plugin's theme configurable chakra-ui/chakra-ui#362

Merged

vladar mentioned this pull request Dec 4, 2020

TypeError on live reload #28429

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gatsby): enable babel-loader for all dependencies #14111

feat(gatsby): enable babel-loader for all dependencies #14111

wardpeet commented May 17, 2019 •

edited

wardpeet May 17, 2019

sidharthachatterjee May 17, 2019 •

edited

sidharthachatterjee commented May 17, 2019

sidharthachatterjee commented May 17, 2019

sidharthachatterjee commented May 17, 2019

wardpeet commented May 17, 2019 •

edited

wardpeet commented Jun 7, 2019

KyleAMathews commented Jun 8, 2019

wardpeet commented Jun 11, 2019

wardpeet commented Jun 11, 2019

wardpeet commented Jun 25, 2019

sidharthachatterjee left a comment

wardpeet commented Jun 27, 2019

monsieurnebo commented Jun 27, 2019

jquense commented Jun 27, 2019

DSchau commented Jun 27, 2019

jquense commented Jun 27, 2019 •

edited

joealden commented Jun 28, 2019

wardpeet commented Jun 28, 2019

feat(gatsby): enable babel-loader for all dependencies #14111

feat(gatsby): enable babel-loader for all dependencies #14111

Conversation

wardpeet commented May 17, 2019 • edited

Description

bundle info

Related Issues

wardpeet May 17, 2019

Choose a reason for hiding this comment

sidharthachatterjee May 17, 2019 • edited

Choose a reason for hiding this comment

sidharthachatterjee commented May 17, 2019

sidharthachatterjee commented May 17, 2019

sidharthachatterjee commented May 17, 2019

wardpeet commented May 17, 2019 • edited

Babel all the things!

wardpeet commented Jun 7, 2019

KyleAMathews commented Jun 8, 2019

wardpeet commented Jun 11, 2019

wardpeet commented Jun 11, 2019

wardpeet commented Jun 25, 2019

sidharthachatterjee left a comment

Choose a reason for hiding this comment

wardpeet commented Jun 27, 2019

monsieurnebo commented Jun 27, 2019

jquense commented Jun 27, 2019

DSchau commented Jun 27, 2019

jquense commented Jun 27, 2019 • edited

joealden commented Jun 28, 2019

wardpeet commented Jun 28, 2019

wardpeet commented May 17, 2019 •

edited

sidharthachatterjee May 17, 2019 •

edited

wardpeet commented May 17, 2019 •

edited

jquense commented Jun 27, 2019 •

edited