Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build reform #20675

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Build reform #20675

wants to merge 2 commits into from

Conversation

ef4
Copy link
Contributor

@ef4 ef4 commented Apr 2, 2024

  1. This makes Ember's test suite build with Vite, with no AMD loader present. Which ensures that Ember is strictly following the ES module spec.

  2. It also makes Ember's prepublication build use Rollup instead of a custom broccoli pipeline. Again, because we want the ES module graph driving the build. This step is Rollup instead of Vite because there's enough backward-compatibility weirdness that Vite's more opinionated packaging up of Rollup features was not helpful.

TODOs:

  • address dependency on decorator-transforms (which is breaking for browser support unless we're careful to only use it in prebuild)
  • do size comparison on prod bundles (to ensure we properly remove ember-testing, etc)
  • do a size comparison on an app's required total node_modules
  • expand the smoke-tests so they exercise the non-prebuilt legacy bundle code path (this is not a thing now, we've returned to shipping ember.prod.js for the legacy bundles so we can avoid shipping our entire legacy bundle build into apps)
  • expand the smoke-tests so they exercise the embroider strictEmberSource code path
  • before merging do a final comparison of the defines in the bundles, before and after the pr
  • replace the PREBUILT variant (which is currently being ignored). We could reimplement prebuilt, or we could expand the end-to-end app tests since they would more naturally cover that case.
  • remove vestigial broccoli build code and audit for unused dependencies
  • update to stable releases of babel-plugin-debug-macros and decorator-transforms
  • restore compatibility with old ember-cli-htmlbars versions

@ef4
Copy link
Contributor Author

ef4 commented Apr 6, 2024

Notes to self on next steps:

  • the prebuilt ember.debug.js and ember-testing.js and ember-template-compiler.js are all traditionally built with hard-coded DEBUG=true and assertions not stripped. We can keep that behavior easily enough.
  • in the ESM output, we should implement the DEBUG and assertion stripping in terms of embroider/macros in preparation for being a real v2 addon. Plus, that will work correctly after the build has been reduced by rollup at prepublication time, since embroider/macros is wholly external, whereas @ember/debug is something rollup is itself building and trying to optimize away.
  • treeForVendor will need to manually run the embroider macros. We can't move ember's code into treeForAddon until we have a strict-es-modules flag to flip (too much stuff expects them to exist early in vendor.js).
  • need to re-add the canary-features.js support. It applies to the ES packages build
  • classically, the ES packages build serves as input to the legacy bundles build. This branch separates them. It's probably worth keeping them layered.

@ef4
Copy link
Contributor Author

ef4 commented Apr 19, 2024

I think "All Tests" jobs are taking forever because filtering down suites by URL isn't working, so we end up running the whole suite many times.

@ef4 ef4 mentioned this pull request Apr 30, 2024
ef4 added a commit to embroider-build/embroider that referenced this pull request May 7, 2024
…anges

I'm working to land a [Build Reform](emberjs/ember.js#20675) branch in ember-source that, among other things, uses only rollup for Ember's prepublication build, ensuring that it's all ES-module clean.

The inter-package imports within `ember-source/dist/packages` switch from being package names (which require non-standard resolving to work) to relative imports (which do not). As a consequence of that it's simpler to ship all of `dist/packages` and `dist/dependencies` together as `dist/packages`. So our compat adapter needs to tolerate `dist/dependencies` not existing.

The special handling we had for enumerating the contents of `dist/dependencies` and removing them from package.json dependencies was only needed to deal with the magical inter-package resolving, so it's correct that it becomes a no-op for these new ember versions.
ef4 added a commit to embroider-build/embroider that referenced this pull request May 7, 2024
…anges

I'm working to land a [Build Reform](emberjs/ember.js#20675) branch in ember-source that, among other things, uses only rollup for Ember's prepublication build, ensuring that it's all ES-module clean.

The inter-package imports within `ember-source/dist/packages` switch from being package names (which require non-standard resolving to work) to relative imports (which do not). As a consequence of that it's simpler to ship all of `dist/packages` and `dist/dependencies` together as `dist/packages`. So our compat adapter needs to tolerate `dist/dependencies` not existing.

The special handling we had for enumerating the contents of `dist/dependencies` and removing them from package.json dependencies was only needed to deal with the magical inter-package resolving, so it's correct that it becomes a no-op for these new ember versions.
@ef4
Copy link
Contributor Author

ef4 commented May 8, 2024

Confirmed in a real app's production build that this reduces vendor.js by 48kB. Most of that is probably from using decorator-transforms internally, so that we're not forced to transpile all other class features.

@ef4
Copy link
Contributor Author

ef4 commented May 8, 2024

I audited the differences caused by this PR in the AMD loader contents of a real app in both dev and prod.

The only change is that ember-babel disappears, which is fine.

@ef4 ef4 marked this pull request as ready for review May 8, 2024 16:27
// ember.debug.js that publishes in the ember-source NPM package. That copy is
// essentially an optimization if you happen to be doing development under the
// default babel targets.
'PREBUILT',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This concept stops being useful in this PR. It was very specific to the old optimization where sometimes ember would use a prebuilt bundle and other times it wouldn't.

@@ -0,0 +1,202 @@
/* eslint-disable */

// This file was derived from the output of the classic broccoli-based build of
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intent for these three amd-compat-entrypoints files is that they're capturing a final snapshot of what we ship in the backward-compatible AMD bundles. We should consider them frozen going forward. New modules shouldn't go into them.

Instead, new modules should ship like normal addon modules from treeForAddon.

The rationale here is that moving existing modules out of vendor.js is a breaking change. But new modules that have never been in vendor.js can avoid ever going there.

This PR doesn't introduce any new modules doing things the new way, but I intend to do that in my next PR.

@@ -1,12 +0,0 @@
'use strict';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bundle the babel helpers is now handled by rollup.

This causes the one visible difference in the contents of the AMD loader after this PR: we used to expose ember-babel there, just because that's where we happened to store our helpers.

// transform, since some tests use decorators and class fields
['@babel/plugin-proposal-decorators', { legacy: true }],
['@babel/plugin-proposal-class-properties', { loose: true }],
module.exports = function canaryFeatures() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just changing from a broccoli transform to a piece of babel config.

const isProduction = process.env.EMBER_ENV === 'production';
let ember = new Funnel(tree, {
destDir: 'ember',
include: [`ember.${which}.js`, `ember.${which}.map`, 'ember-testing.js', 'ember-testing.map'],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I'm returning to a simpler pattern that we've used in the past, which is that the prepublication build produces both ember.debug.js and ember.prod.js. The benefit of this is that we don't need to bring the complete prepublication build machinery into every app.

This works now because we don't need @babel/preset-env to do anything in our prepublication build. We can produce complete bundles and still run them through preset-env at app-build time.

if (
!isProduction &&
// do the running applications targets match our prebuilt assets targets?
PRE_BUILT_TARGETS.every((target) => targets.includes(target)) &&
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This optimization depends on an assumption that is no longer true. Which is that producing an ember bundle necessarily implies using some preset-env targets.

There are no targets applied to our prebuilt bundles. It's now possible and desirable to instead pass the whole bundle through preset-env at the point where the app provides the targets.

Overall this generates production output that's about 48kB smaller.


moduleFor(
moduleForDevelopment(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, these modules were stripped from production-mode builds of the test suite using broccoli shenanigans.

Now all test modules are pulled into the test suite via import.meta.glob, and we use this moduleForDevelopment utility to control whether they execute.

var result = callback.apply(this, reified);
if (!deps.includes('exports') || result !== undefined) {
exports = result;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a whole extra AMD loader that ships in bundled ember that is inert in typical usage in the browser (because the AMD loader from loader.js is present instead). It apparently still exists to make the bundles evaluatble in node.

But it lacked a feature found in loader.js that we happen to want to use. So this ports that feature into it.

@@ -0,0 +1,77 @@
<!DOCTYPE html>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't really new, it moved from tests/index.html to follow vite conventions.

…sent. Which ensures that Ember is strictly following the ES module spec.

It also makes Ember's prepublication build use Rollup instead of a custom broccoli pipeline. Again, because we want the ES module graph driving the build. This step is Rollup instead of Vite because there's enough backward-compatibility weirdness that Vite's more opinionated packaging up of Rollup features was not helpful.
index.html Outdated Show resolved Hide resolved
Co-authored-by: Katie Gengler <katie@kmg.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants