From de5b2ec8513e5cd02e84fd95a56f017d4513db71 Mon Sep 17 00:00:00 2001 From: Mike Reinstein Date: Mon, 14 Dec 2020 10:41:07 -0800 Subject: [PATCH 1/7] New: provide an es module --- designs/2020-es-module-support/README.md | 159 +++++++++++++++++++++++ 1 file changed, 159 insertions(+) create mode 100644 designs/2020-es-module-support/README.md diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md new file mode 100644 index 00000000..6489a8a3 --- /dev/null +++ b/designs/2020-es-module-support/README.md @@ -0,0 +1,159 @@ +- Repo: eslint/espree +- Start Date: 2020-12-13 +- RFC PR: `TODO` +- Authors: Mike Reinstein (https://github.com/mreinstein) + +# Adding es module support + +## Summary + +Provide an ecmascript module for espree. + + +## Motivation + +Historically there have been a number of competing module standards: commonjs, requireJs, iife wrapped script tags, etc. + +This has contributed to a very complicated tooling scenario for node/npm/javascript based projects in general: +* every module has it's own unique build system, with no consistency across modules. +* projects are packed with complex configurations, much of which is plumbing related to packaging/bundling. + +Over time, browsers have adopted the ecma module standard. Today, support is pretty good: +https://caniuse.com/es6-module + +Node also added support in v12.17.1 and up, and has officially marked the API stable: +https://nodejs.org/dist/latest-v15.x/docs/api/esm.html#esm_modules_ecmascript_modules + + +By adopting this new standard, we can achieve a few things: + +* eventual simplification of espree's build process by eliminating the need for multiple module formats +* make it easier to directly importing espree in browser environments, deno, etc. Skypack is a CDN that already enables this, but they have internal logic that looks at every package in npm and tries to build an es module. https://www.skypack.dev/ + +As more projects adopt this standardized module format, the hope is a lot of this "connective glue" will fall away, leaving us with leaner modules and less build tooling across our stack. + + +## Detailed Design + +### Nomenclature + +* `ESM` ecmascript module. The standardized module format for javascript +* `CJS` commonjs. Node's historical module format (i.e., `module.exports = ...`, `const a = require('foo')`) +* `UMD` universal module definition. A legacy format for specifying javascript code as a module in a way that tries to guarantee compatibility with many loader formats including requireJS, commonjs, and global script tags. +* `IIFE` immediately invoked function expressions. A legacy strategy for encapsulating javascript into a module that won't leak global variables into the execution environment. It is still the primary way of packaging global script tags for inclusion in a web page. `UMD` is usually wrapped in an `IIFE` +* `CDN` content delivery network. can be thought of as a set of servers that host static content all around the globe to deliver assets quickly and at scale. + + +### build process + +Today, espree is written in commonjs (CJS) format, and a universal module definition (UMD) build is created too. + + +today's build process: +``` +┌-------------------┐ ┌-----------------┐ +│ │ │ │ package.json: +│ espree.js | │ build/espree.js │ +│ (CJS entry point) ├──BUILD_STEP──▶│ (UMD bundle) │ CJS ---▶ "main": "espress.js", +│ │ │ │ +└-------------------┘ └-----------------┘ +``` + + +The strategy that probably makes the most sense in this initial work is to do what is known as a "dual package" solution, where +espree provides an ESM, and a CJS module. + +Providing a dual package shouldn't affect any existing espree users, but provies an ESM option for users that want it. + + +proposed build process in this RFC: +``` + ┌-------------------┐ + │ │ + │ dist/espree.cjs │ + │ (CJS entry point) │ package.json: +┌-------------------┐ │ │ +│ │ └--▲----------------┘ CJS ---▶ "main": "dist/espress.cjs", +│ espree.js │ │ "exports": { +│ (ESM entry point) ├───BUILD_STEP───┤ ESM ---▶ "import": "espree.js", +│ │ │ CJS ---▶ "require": "./dist/espree.cjs" +└-------------------┘ ▼ }, + ┌----------------┐ "type": "module" + │ │ + │ dist/espree.js │ + │ (UMD bundle) │ + │ │ + └----------------┘ +``` + +browserify is specifically oriented around commonjs as the input format, so I also propose replacing it with rollup, which understands and expects ESM as it's input format. + + +## Documentation + +The changes should be described in the migration guide for whichever major version of espree this goes into (currently considering the upcoming 8.x line.) + +Despite the fact that we don't expect this to break the existing API, it probably makes sense to formally announce this via blog post as well, because it is a somewhat dramatic change. + + +## Drawbacks + +The javascript build/tooling ecosystem is already monstrously complex, and this change adds another output format, which further complicates things (compare the 2 graphs above to see this additional complexity visualized.) + +I do believe this is a temporary scenario; as time goes on, the UMD and CJS entry points could be dropped altogether, along with the build step. But that will take months, maybe even years depending on how comfortable the community is with this change and how widely adopted the ESM format becomes. + + +## Backwards Compatibility Analysis + +The dual package approach _should_ not break espree for anyone currently using espree. + + +## Alternatives + +Another option is to just drop CJS, UMD and the build system altogether, provide an ESM only implementation, +and make this a part of a semver-major version bump. (e.g., espree@8 is ESM only.) + +This would essentially eliminate the build step and make things very simple, but it has a big downside: +* users that don't support ESM only projects would be stuck on `espree <= 7.x` forever (or at least until they added the ability to use ESM in their own stuff.) + + +## Open Questions + +* Do we still care about UMD? Maybe it makes sense to just drop this too if it's not something people are using much. + + +## Help Needed + +None yet. + + +## Frequently Asked Questions + +> what is this "dual package" stuff? + +It's a proposal by the node community on how a module may go about adopting ESM without totally breaking CJS for their users. https://nodejs.org/api/packages.html#packages_writing_dual_packages_while_avoiding_or_minimizing_hazards + + +> are any other well known modules using this approach? + +acorn is producing a dual module, and has been the inspiration for my own work around the forthcoming espree PR https://github.com/acornjs/acorn/blob/master/acorn/package.json#L5-L11 + + +> npm modules reference each other via commonjs, and the browser's loader expects URLs, so why are we concerned with making npm modules work in a browser? + +that's true, you can't simply do `import espree from 'https://github/eslint/espree.js'` naively because it's referencing other npm modules in it, such as `acorn`. + +However there are some CDNs which will transform npm modules to refer to URLs. For example, this works today: + +```javascript +import espree from 'https://cdn.skypack.dev/espree' +``` + +By making espree a standard ESM it reduces the amount of transforms that need to be run on it to make it usable. + +It's also possible that in the semi-near future, node may consider offering URL loaders. Deno is already doing this. + + +## Related Discussions + +- [1] The discussion genesis https://github.com/eslint/espree/issues/457 From 6758ec60f211cc148bdc08e5dd4d20318d953e93 Mon Sep 17 00:00:00 2001 From: Mike Reinstein Date: Sat, 26 Dec 2020 09:43:32 -0800 Subject: [PATCH 2/7] Update designs/2020-es-module-support/README.md Co-authored-by: Nicholas C. Zakas --- designs/2020-es-module-support/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md index 6489a8a3..1791a515 100644 --- a/designs/2020-es-module-support/README.md +++ b/designs/2020-es-module-support/README.md @@ -1,6 +1,6 @@ - Repo: eslint/espree - Start Date: 2020-12-13 -- RFC PR: `TODO` +- RFC PR: https://github.com/eslint/rfcs/pull/72 - Authors: Mike Reinstein (https://github.com/mreinstein) # Adding es module support From 1d18c2c4745f6ca32504e63e4f5f0f83a48cab36 Mon Sep 17 00:00:00 2001 From: Mike Reinstein Date: Sat, 26 Dec 2020 10:59:00 -0800 Subject: [PATCH 3/7] Update: drop UMD build from es module proposal and clarify that it is indeed a breaking change. --- designs/2020-es-module-support/README.md | 35 +++++++++++------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md index 1791a515..80adf390 100644 --- a/designs/2020-es-module-support/README.md +++ b/designs/2020-es-module-support/README.md @@ -63,42 +63,39 @@ today's build process: The strategy that probably makes the most sense in this initial work is to do what is known as a "dual package" solution, where espree provides an ESM, and a CJS module. -Providing a dual package shouldn't affect any existing espree users, but provies an ESM option for users that want it. +Providing a dual package shouldn't affect any existing espree users, but provies an ESM option for users that want it. proposed build process in this RFC: ``` - ┌-------------------┐ - │ │ - │ dist/espree.cjs │ - │ (CJS entry point) │ package.json: -┌-------------------┐ │ │ -│ │ └--▲----------------┘ CJS ---▶ "main": "dist/espress.cjs", + ┌-------------------┐ + │ │ + │ dist/espree.cjs │ + │ (CJS entry point) │ package.json: +┌-------------------┐ │ │ +│ │ └--▲----------------┘ CJS ---▶ "main": "dist/espress.cjs", │ espree.js │ │ "exports": { -│ (ESM entry point) ├───BUILD_STEP───┤ ESM ---▶ "import": "espree.js", -│ │ │ CJS ---▶ "require": "./dist/espree.cjs" -└-------------------┘ ▼ }, - ┌----------------┐ "type": "module" - │ │ - │ dist/espree.js │ - │ (UMD bundle) │ - │ │ - └----------------┘ +│ (ESM entry point) ├───BUILD_STEP───┘ ESM ---▶ "import": "espree.js", +│ │ CJS ---▶ "require": "./dist/espree.cjs" +└-------------------┘ }, + "type": "module" ``` browserify is specifically oriented around commonjs as the input format, so I also propose replacing it with rollup, which understands and expects ESM as it's input format. +`UMD` is not part of the formal package interface and it's usage is questionable, so I propose we drop it altogether. + ## Documentation The changes should be described in the migration guide for whichever major version of espree this goes into (currently considering the upcoming 8.x line.) -Despite the fact that we don't expect this to break the existing API, it probably makes sense to formally announce this via blog post as well, because it is a somewhat dramatic change. +Adding an `exports` field to `package.json` will break the existing API, it makes sense to formally announce this via blog post. ## Drawbacks -The javascript build/tooling ecosystem is already monstrously complex, and this change adds another output format, which further complicates things (compare the 2 graphs above to see this additional complexity visualized.) +The javascript build/tooling ecosystem is already monstrously complex, and this change adds another output format, which further complicates things (compare the 2 graphs above to see this additional complexity visualized.) I do believe this is a temporary scenario; as time goes on, the UMD and CJS entry points could be dropped altogether, along with the build step. But that will take months, maybe even years depending on how comfortable the community is with this change and how widely adopted the ESM format becomes. @@ -119,7 +116,7 @@ This would essentially eliminate the build step and make things very simple, but ## Open Questions -* Do we still care about UMD? Maybe it makes sense to just drop this too if it's not something people are using much. +None yet. ## Help Needed From c43b807480b21e5ca2ce625cf3ed1ca44df43f3e Mon Sep 17 00:00:00 2001 From: Mike Date: Wed, 3 Feb 2021 00:58:30 -0800 Subject: [PATCH 4/7] Docs: fix typo in esm build diagrams (#72) --- designs/2020-es-module-support/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md index 80adf390..36ec191c 100644 --- a/designs/2020-es-module-support/README.md +++ b/designs/2020-es-module-support/README.md @@ -54,7 +54,7 @@ today's build process: ┌-------------------┐ ┌-----------------┐ │ │ │ │ package.json: │ espree.js | │ build/espree.js │ -│ (CJS entry point) ├──BUILD_STEP──▶│ (UMD bundle) │ CJS ---▶ "main": "espress.js", +│ (CJS entry point) ├──BUILD_STEP──▶│ (UMD bundle) │ CJS ---▶ "main": "espree.js", │ │ │ │ └-------------------┘ └-----------------┘ ``` @@ -73,7 +73,7 @@ proposed build process in this RFC: │ dist/espree.cjs │ │ (CJS entry point) │ package.json: ┌-------------------┐ │ │ -│ │ └--▲----------------┘ CJS ---▶ "main": "dist/espress.cjs", +│ │ └--▲----------------┘ CJS ---▶ "main": "dist/espree.cjs", │ espree.js │ │ "exports": { │ (ESM entry point) ├───BUILD_STEP───┘ ESM ---▶ "import": "espree.js", │ │ CJS ---▶ "require": "./dist/espree.cjs" From c6d3e664d66f67d94db74ac6c378dda5f25eba2e Mon Sep 17 00:00:00 2001 From: Mike Date: Wed, 3 Feb 2021 01:01:50 -0800 Subject: [PATCH 5/7] Docs: declare intent to remove umd bundle (#72) --- designs/2020-es-module-support/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md index 36ec191c..5219069d 100644 --- a/designs/2020-es-module-support/README.md +++ b/designs/2020-es-module-support/README.md @@ -83,7 +83,7 @@ proposed build process in this RFC: browserify is specifically oriented around commonjs as the input format, so I also propose replacing it with rollup, which understands and expects ESM as it's input format. -`UMD` is not part of the formal package interface and it's usage is questionable, so I propose we drop it altogether. +`UMD` is not part of the formal package interface and will be dropped altogether. ## Documentation From e33981e4b0534e1aaf7e33cdf256295e8befd03d Mon Sep 17 00:00:00 2001 From: Mike Date: Wed, 3 Feb 2021 01:15:51 -0800 Subject: [PATCH 6/7] Docs: add FAQ item about tests (#72) --- designs/2020-es-module-support/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md index 5219069d..40e66c19 100644 --- a/designs/2020-es-module-support/README.md +++ b/designs/2020-es-module-support/README.md @@ -136,6 +136,11 @@ It's a proposal by the node community on how a module may go about adopting ESM acorn is producing a dual module, and has been the inspiration for my own work around the forthcoming espree PR https://github.com/acornjs/acorn/blob/master/acorn/package.json#L5-L11 +> does this mean local development now requires a build step? + +no. Tests should be written to run against the ESM source in `lib/`. The pre-release tests should be run against the CJS bundle as part of the pre-publish step. + + > npm modules reference each other via commonjs, and the browser's loader expects URLs, so why are we concerned with making npm modules work in a browser? that's true, you can't simply do `import espree from 'https://github/eslint/espree.js'` naively because it's referencing other npm modules in it, such as `acorn`. From 901d97f6b047b94814ac2257c859e7f622677536 Mon Sep 17 00:00:00 2001 From: Mike Date: Wed, 3 Feb 2021 01:23:17 -0800 Subject: [PATCH 7/7] Docs: add FAQ item explaining use of named exports (#72) --- designs/2020-es-module-support/README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/designs/2020-es-module-support/README.md b/designs/2020-es-module-support/README.md index 40e66c19..f975aafe 100644 --- a/designs/2020-es-module-support/README.md +++ b/designs/2020-es-module-support/README.md @@ -141,6 +141,15 @@ acorn is producing a dual module, and has been the inspiration for my own work a no. Tests should be written to run against the ESM source in `lib/`. The pre-release tests should be run against the CJS bundle as part of the pre-publish step. +> how would one import espree now that it's providing an esm interface? + +There will be named exports. i.e., + +```javascript +import { parse, tokenize /*, ... */ } from 'espree'; +``` + + > npm modules reference each other via commonjs, and the browser's loader expects URLs, so why are we concerned with making npm modules work in a browser? that's true, you can't simply do `import espree from 'https://github/eslint/espree.js'` naively because it's referencing other npm modules in it, such as `acorn`.