Compile dependencies with babel-preset-env

[gaearon]

Proof of concept for #1125.
This currently runs all node_modules through babel-preset-env.

(Which was unsafe in Babel 6, but should—maybe—be safe in Babel 7.)

I don't know the impact on the build time. It'll definitely make the cold start slower but hopefully repeated builds will be okay. We need some measurements with larger projects here: https://mobile.twitter.com/dan_abramov/status/952248179478745089.

I intentionally only did this for webpack. I don’t think we need to do this for tests because you control the Node version on which you run the tests, unlike the user’s browser. And it’s never required to compile because we intentionally don’t allow any custom transforms (like JSX). Only “real” JS gets compiled.

TODO:

• Measure build time impact
• Consider opt-out heuristics (e.g. "too old" engines field that implies we don't need compilation)
• Verify this doesn't break third-party code
• Verify generators and async/await work in deps
• Verify helpers for the above are being shared (e.g. runtime transform)
• Verify .babelrc in packages gets ignored
• Create end-to-end tests for these situations

[Timer]

Maybe we should?

[gaearon]

Since Node hasn't decided how to handle ESM I don't think we should support it

[gaearon]

At least it passes the CI, not bad

[Timer]

yarn build:

next cold: 110s
next warm: 75s
compile-deps cold: 136s
compile-deps warm: 84s

File sizes after gzip:

  381.63 KB (+634 B)  build/static/js/0.531211dc.chunk.js
  278.26 KB (+832 B)  build/static/js/3.e5d787e7.chunk.js
  277.21 KB (+160 B)  build/static/js/2.e9776a55.chunk.js
  273.4 KB (+89 B)    build/static/js/4.24972f35.chunk.js
  268.39 KB (+305 B)  build/static/js/1.20148f30.chunk.js
  199.08 KB (+904 B)  build/static/js/main.29f461ce.js
  31.32 KB            build/static/css/main.e5579a8c.css

[gaearon]

I think we can live with these numbers given how much demand there is for this feature.

[gaearon]

It would be nice to see if updating webpack to latest 3.x (or 4.x prereleases) could offset some of the hit.

[Timer]

We could use webpack's new thread-loader to multi-thread compilation. Last time I played with it there were pretty promising results.

[Timer]

Sticking thread-loader in front removes the hit we see from compiling node_modules:

cold: 107s
warm: 71s

[Timer]

😭 🎉

[alzalabany]

@ljharb sometimes its not an external package that is using > es5, its your own packages in an monorepo while you working on a new feature back and forth between ui-component-lib and main webapp

[ljharb]

@alzalabany sure, but that's a special case, and deserves an explicit opt in - that's not a safe default nor a reasonable thing to expect as a common case.

[franciscop]

@ljharb how long should 3rd party libraries support ES5? IMHO this question does not have a decent answer since it depends on the actual devs using the libraries and what they support...

• Forever - no, since support for ES3-4 is not needed we can predict some day support for ES5 will not be needed.
• N years - no, since different users of my library will have vastly different support needs based on their user base.
• No compiling - well, the only problem is webpack/create-react-app don't support this.

Compiled ES5 has a quantitative disadvantage: it is larger (and slower) than ES6+. Support for ES6+ just requires time to come, but if every library author compiles to ES5 it'll take really long time to fix in the npm community and we don't want another callbacks vs promises issue. I think it's better to support Javascript, and that libraries should do so, instead of trying to force the whole ecosystem to be locked into a single version.

[ljharb]

@franciscop much much longer than you can drop it in your app, and yes, effectively forever (es4 never existed). Also, compiled es5 isn’t necessarily slower - browsers optimize old patterns, not new patterns, because more people use them. Publishing a dep that’s not compiled down is what’s forcing the ecosystem to a specific level of javascript, because it means people can’t safely use your code when they want to support older engines.

Either way, this isn’t something we should be debating in this thread; feel free to find me on irc.

[JesseChrestler]

Having the ability to opt-in with a list of packages that need to be transpiled would actually help both arguments. You won't have to eject to transpile node_modules but aren't unnecessarily transpiling packages. My project has a ton of dependencies but only transpiling 3 packages. We had to fork the repo so we could get this behavior in our monorepo. However on the opposite side I wouldn't want to transpile everything. 🤔

[gaearon]

That's tangential, so feel free to file a new issue.

This behavior shipped in 2.0.
https://reactjs.org/blog/2018/10/01/create-react-app-v2.html

[conartist6]

When I search on google I end up here, but all that's here is a link to release notes and the blog post, neither of which mentions this presumably huge important feature.

I'll just add what I know: as far as I can tell, create-react-app@2 just does this, no additional configuration is required. It uses a slightly more conservative config, transpiling only core es features (those which preset-env supports by default).