fix #334: support automatic JSX runtime #2349

jgoz · 2022-06-25T20:41:38Z

fixes #334
fixes #718
fixes #1172
fixes #2318

This adds support for the new (automatic) JSX runtime to esbuild for both the build and transform APIs.

New CLI flags and API options:

--jsx-runtime, jsxRuntime — "automatic" or "classic" (default)
--jsx-development, jsxDevelopment — toggles development mode for the automatic runtime
--jsx-import-source, jsxImportSource — Overrides the root import for runtime functions (default "react")

New JSX pragmas:

@jsx — sets the runtime ("automatic" or "classic")
@jsxImportSource — sets the import source (only valid with automatic runtime)

@jsxFragment and @jsxFactory are only valid with classic runtime.

Implementation details:
Most of the work to support the automatic runtime happens in the second parsing pass. Here is a high level overview of the logic when automatic runtime is enabled:

Determine whether Fragment needs to be imported (from the appropriate runtime import)
Look for key prop and extract it from the props argument, since it's now passed directly to a jsx function
Inspect children and determine whether they are considered static (generally, more than 1 child) or not
Build up arguments list to pass to the appropriate jsx function
- In development mode, this includes "source" and "self" args and "isStaticChildren" is an argument
Determine which jsx function to use:
- jsxDEV — development mode (from {importSource}/jsx-dev-runtime)
- jsxs — production mode, static children (from {importSource}/jsx-runtime)
- jsx — production mode, non-static children (from {importSource}/jsx-runtime)
- createElement — fallback mode* (from {importSource})
Record an import of the appropriate function from the appropriate source
Insert a function call with the assembled arguments

At the end of parsing, if any JSX symbols had recorded usage, the appropriate import statement is generated for only those symbols that were actually used within the file. These are generated as external imports, unlike esbuild's runtime.

* Fallback mode — If a component includes spread props followed by a key prop, {...props} key={1}, Babel triggers a fallback mode to createElement. Apparently this is a temporary special case that will be removed once spreading "key" from props is no longer supported. It looks like the TypeScript compiler also implements this special case, so it seemed prudent to be compatible here.

Development mode:
Aside from importing a different function from a different source ({ jsxDEV } from 'react/jsx-dev-runtime'), development mode also passes more arguments, one of which is the "source" argument. This contains the current filename, line number, and column number mapping to the original source location of the element.

I used the LineColumnTracker to extract those details from the element's location, which will probably incur a minor performance cost that should be documented for the jsxDevelopment option. The alternative was to simply omit those details, but that would defeat the purpose of supporting development mode.

TSConfig resolving:
Along with accepting the new options directly via CLI or API, I implemented option inference from tsconfig compiler options.

"jsx": "preserve" or "jsx": "react-native" → preserve
"jsx": "react" → classic runtime
"jsx": "react-jsx" → automatic runtime
"jsx": "react-jsxdev" → automatic runtime and development mode enabled

It also reads the value of "jsxImportSource" from tsconfig if specified.

For react-jsx it's important to note that it doesn't implicitly set jsxDevelopment to false. This is to support the case where a user sets react-jsx in their tsconfig but then toggles development mode directly in esbuild, e.g., by environment variable or some other means.

esbuild vs Babel vs TS vs...

There are a few differences between the various technologies that implement automatic JSX runtimes. Though esbuild generally follows TypeScript's behavior when there is a disagreement, I chose to follow Babel in cases where it might help the user catch invalid or undesirable behavior.

Here are the notable differences:

Element has __source or __self props:
- Babel: print an error about a deprecated transform plugin
- TypeScript: allow the props
- swc: Hard crash
- esbuild: print an error — I chose to follow Babel for this one because this might help people catch configuration issues where JSX files are being parsed by multiple tools
Element has an "implicit true" key prop, e.g. <a key />:
- Babel: print an error indicating that "key" props require an explicit value
- TypeScript: silently omit the "key" prop
- swc: Hard crash
- esbuild: print an error like Babel — this might help catch legitimate programming mistakes
Element has spread children, e.g. <a>{...children}</a>
- Babel: print an error stating that React doesn't support spread children
- TypeScript: use static jsx function and pass children as-is, including spread operator
- swc: same as Babel
- esbuild: same as TypeScript

jgoz · 2022-06-25T20:44:49Z

I should note that I tested this successfully on a large internal codebase that uses the automatic runtime.

evanw · 2022-06-30T02:51:43Z

Heads up that I'm currently traveling. Given that this is a big PR, I will not be able to look at this until I get back.

jgoz · 2022-06-30T03:13:27Z

@evanw Thanks for the heads up — safe travels.

I thought about splitting the work into smaller chunks, but I couldn't think of a natural way of doing it. If there's anything I can do to make reviewing this easier when you get back, please let me know.

httpete · 2022-07-12T19:50:47Z

This will be a huge improvement. I have always felt squishy about the inject and react-shim. I am using it on a large project that uses mui and the jsx from @emotion/react.

import { jsx } from '@emotion/react';
import { Fragment } from 'react';
export { jsx, Fragment };

alexgorbatchev · 2022-07-13T20:48:51Z

This contains the current filename, line number, and column number mapping to the original source location of the element.

Thank you so much for this! 🙏

alexgorbatchev · 2022-07-19T22:37:07Z

@evanw want to bump this PR, just in case you may be back from traveling! 🗺️

# Conflicts: # internal/js_parser/js_parser.go

jgoz · 2022-07-23T15:23:03Z

internal/js_parser/js_parser.go

+	p.recordUsage(ref)
+	return p.handleIdentifier(loc, &js_ast.EIdentifier{Ref: ref}, identifierOpts{
+		wasOriginallyIdentifier: true,
+		matchAgainstDefines:     true, // Allow defines to rewrite imported JSX symbols


I added this after resolving conflicts because it seems like if we support rewriting Fragment and createElement, we should support rewriting jsx, jsxs, and jsxDEV.

evanw · 2022-07-25T04:31:20Z

Didn't get to this for this release cycle, sorry. Hoping to look at this soon (this week). Don't worry about resolving merge conflicts. I can do that when I process this PR.

* Improve line/column performance by not repeatedly scanning backward * Column numbers are supposed to be 1-based to match Babel, not 0-based * Column numbers should be in UTF-16 code units to match Babel, not bytes

The define feature only applies to globals and "automatic" JSX symbols are never globals, so this never did anything.

I find it confusing to have both `--jsx` and `--jsx-runtime` since the underlying state is really only one of three settings. This merges both of them, which is similar to what TypeScript does. I have kept the `--jsx-development` option separate because the desire is to be able to override TypeScript's settings. But I have shortened it to `--jsx-dev` to make specifying it on the command line easier.

evanw

I have gone through all of the code in this PR and made various changes. Most of them are minor bug fixes or tweaks. I will also take care of updating the documentation about all of this on esbuild's website.

One bigger change to call out is the removal of the --jsx-runtime setting. I merged it into the existing --jsx option, which was changed into a three-state value. This is similar to how TypeScript specifies this, and seemed simpler conceptually. As a bonus, it's also shorter to specify it on the command line. So the --jsx setting now has three values:

--jsx=transform (the default)
--jsx=preserve
--jsx=automatic

I also wanted to say that this PR was really well done! It's obvious that a lot of thought and care went into this. I really appreciate your clear documentation about the changes in the PR description, the clean and straightforward implementation, and all of the added tests.

pleunv · 2022-07-28T14:54:18Z

Hi @evanw, really appreciate you taking the time to get this merged. I think this was for many people the last missing puzzle piece to make a switch to esbuild, and it's great to see this land!

edit: and a massive thanks to @jgoz of course for actually implementing this!

mbrevda · 2022-07-28T14:58:54Z

Yup - great to see this! #312 next?

jgoz

I will also take care of updating the documentation about all of this on esbuild's website.

Thank you 🙇

One bigger change to call out is the removal of the --jsx-runtime setting.

This makes complete sense. I was trying to avoid changes to existing features/flags as much as possible, but this is clearly better.

I also wanted to say that this PR was really well done! It's obvious that a lot of thought and care went into this. I really appreciate your clear documentation about the changes in the PR description, the clean and straightforward implementation, and all of the added tests.

Thank you for being receptive to this change and for taking the time to review this PR so thoroughly. And, more broadly, thank you for esbuild itself — it is a significant achievement in web tooling. I've learned as much about the web platform just from your release notes as I have from any other source.

jgoz · 2022-07-28T14:54:38Z

internal/js_parser/js_parser_test.go


 	expectParseErrorJSXAutomatic(t, d, "<a key/>", "<stdin>: ERROR: Please provide an explicit key value. Using \"key\" as a shorthand for \"key={true}\" is not allowed.\n<stdin>: NOTE: The property \"key\" was defined here:\n")
 	expectParseErrorJSXAutomatic(t, d, "<div __self={self} />", "<stdin>: ERROR: Duplicate \"__self\" prop found. Both __source and __self are set automatically by esbuild. These may have been set automatically by a plugin.\n<stdin>: NOTE: The property \"__self\" was defined here:\n")
 	expectParseErrorJSXAutomatic(t, d, "<div __source=\"/path/to/source.jsx\" />", "<stdin>: ERROR: Duplicate \"__source\" prop found. Both __source and __self are set automatically by esbuild. These may have been set automatically by a plugin.\n<stdin>: NOTE: The property \"__source\" was defined here:\n")

+	// Line/column offset tests. Unlike Babel, TypeScript sometimes points to a
+	// location other than the start of the element. I'm not sure if that's a bug
+	// or not, but it seems weird. So I decided to match Babel instead.


jgoz · 2022-07-28T14:58:46Z

internal/js_parser/js_parser.go

+	// some buggy JavaScript toolchains such as the TypeScript compiler convert
+	// ESM into CommonJS by replacing "import" statements inline without doing
+	// any hoisting, which is incorrect. See the following issue for more info:
+	// https://github.com/microsoft/TypeScript/issues/16166. Since JSX-related


Oh wow, I didn't know about this and didn't consider that esbuild output would be used as input to tsc. Thank you for catching this.

jgoz · 2022-07-28T15:00:01Z

internal/js_parser/js_parser.go

@@ -1573,7 +1573,6 @@ func (p *parser) importJSXSymbol(loc logger.Loc, jsx JSXImport) js_ast.Expr {
 	p.recordUsage(ref)
 	return p.handleIdentifier(loc, &js_ast.EIdentifier{Ref: ref}, identifierOpts{
 		wasOriginallyIdentifier: true,
-		matchAgainstDefines:     true, // Allow defines to rewrite imported JSX symbols


Makes sense, I was clearly missing context here.

httpete · 2022-07-28T18:19:52Z

It’s marvelous. I set it to automatic and it did everything perfectly even with jsxImportSource to emotion/ MUI.

…

On Jul 28, 2022, at 11:20 AM, Evan Wallace ***@***.***> wrote: Merged #2349 into master. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

Reduces the amount of boilerplate to render the UI and makes it easier to respond to state changes (e.g. machine getting authorized, netmap changing, etc.) Preact adds ~13K to our bundle size (5K after Brotli) thus is a neglibible size contribution. We mitigate the delay in rendering the UI by having a static placeholder in the HTML. Required bumping the esbuild version to pick up evanw/esbuild#2349, which makes it easier to support Preact's JSX code generation. Fixes #5137 Fixes #5273 Signed-off-by: Mihai Parparita <mihai@tailscale.com>

jgoz added 13 commits June 24, 2022 14:07

Add new JSX options to API

90db88d

Resolve applicable jsx options from tsconfig

34a0667

Default options, parse pragmas

9d81ee7

Accept nil sourceIndex for external imports

9e00cd7

Parsing and import generation

2a6495a

tsconfig tests

032c9e2

JS API tests

fe60076

Parser tests

8e4a46c

Add sourcemap tests

ff5ec74

Fix handling of __self and __source

7934001

Add namespace tests

3676e5b

Clean up tests

428cdb1

Remove unused JSX consts

add10ea

pradel mentioned this pull request Jul 20, 2022

new JSX transformer not supported pradel/create-react-app-esbuild#7

Open

agriffis mentioned this pull request Jul 22, 2022

ESBuild support (non-Babel-builds) emotion-js/emotion#2474

Closed

jgoz added 2 commits July 23, 2022 09:10

Merge branch 'master' into jsx-runtime

f740295

# Conflicts: # internal/js_parser/js_parser.go

Set matchAgainstDefines: true for imported JSX symbols

1f9ddc9

jgoz commented Jul 23, 2022

View reviewed changes

evanw added 5 commits July 27, 2022 21:03

Merge branch 'master' into jsx-runtime

f430a16

Some fixes for jsx source positions

8ae6058

* Improve line/column performance by not repeatedly scanning backward * Column numbers are supposed to be 1-based to match Babel, not 0-based * Column numbers should be in UTF-16 code units to match Babel, not bytes

Add missing source map positions

7e6c025

Move imports to top to avoid bugs in other tools

033eaa1

Do not attempt to rewrite JSX symbols with define

431db4f

The define feature only applies to globals and "automatic" JSX symbols are never globals, so this never did anything.

evanw added 3 commits July 28, 2022 01:12

Update error messages to match esbuild's style

1cdd684

Add something to the release notes

93f9ea4

evanw approved these changes Jul 28, 2022

View reviewed changes

The automatic runtime implicitly sets strict mode

c20f3c2

jgoz commented Jul 28, 2022

View reviewed changes

evanw merged commit 0137224 into evanw:master Jul 28, 2022

jgoz deleted the jsx-runtime branch July 28, 2022 15:20

jgoz mentioned this pull request Jul 28, 2022

Use esbuild's automatic JSX runtime handling jgoz/esbuild-plugins#18

Merged

jonkoops mentioned this pull request Jul 29, 2022

Use automatic JSX runtime feature from esbuild vitejs/vite#9434

Closed

tylerbrostrom mentioned this pull request Jul 29, 2022

Cannot use automatic jsx runtime in fresh denoland/fresh#382

Closed

mihaip mentioned this pull request Aug 4, 2022

cmd/tsconnect: switch UI to Preact tailscale/tailscale#5300

Merged

lemonmade mentioned this pull request Aug 5, 2022

Use esbuild's native JSX runtime feature in development lemonmade/quilt#393

Closed

alex-kinokon mentioned this pull request Aug 21, 2022

React-Plugin does not respect the jsxImportSource from TSConfig vitejs/vite-plugin-react#9

Closed

4 tasks

sapphi-red mentioned this pull request Oct 7, 2022

feat: update esbuild compilation affecting fields vitejs/vite#10374

Merged

9 tasks

tmeasday mentioned this pull request Dec 13, 2022

[Bug]: esbuild fails with ERROR: Invalid option in build() call: "jsxImportSource" storybookjs/storybook#20215

Closed

jolle mentioned this pull request Dec 30, 2022

feat(remix-dev): add jsxImportSource config option remix-run/remix#2216

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix #334: support automatic JSX runtime #2349

fix #334: support automatic JSX runtime #2349

jgoz commented Jun 25, 2022 •

edited

jgoz commented Jun 25, 2022

evanw commented Jun 30, 2022

jgoz commented Jun 30, 2022

httpete commented Jul 12, 2022 •

edited

alexgorbatchev commented Jul 13, 2022

alexgorbatchev commented Jul 19, 2022

jgoz Jul 23, 2022

evanw commented Jul 25, 2022

evanw left a comment

pleunv commented Jul 28, 2022 •

edited

mbrevda commented Jul 28, 2022

jgoz left a comment

jgoz Jul 28, 2022

jgoz Jul 28, 2022

jgoz Jul 28, 2022

httpete commented Jul 28, 2022 via email

fix #334: support automatic JSX runtime #2349

fix #334: support automatic JSX runtime #2349

Conversation

jgoz commented Jun 25, 2022 • edited

jgoz commented Jun 25, 2022

evanw commented Jun 30, 2022

jgoz commented Jun 30, 2022

httpete commented Jul 12, 2022 • edited

alexgorbatchev commented Jul 13, 2022

alexgorbatchev commented Jul 19, 2022

jgoz Jul 23, 2022

Choose a reason for hiding this comment

evanw commented Jul 25, 2022

evanw left a comment

Choose a reason for hiding this comment

pleunv commented Jul 28, 2022 • edited

mbrevda commented Jul 28, 2022

jgoz left a comment

Choose a reason for hiding this comment

jgoz Jul 28, 2022

Choose a reason for hiding this comment

jgoz Jul 28, 2022

Choose a reason for hiding this comment

jgoz Jul 28, 2022

Choose a reason for hiding this comment

httpete commented Jul 28, 2022 via email

jgoz commented Jun 25, 2022 •

edited

httpete commented Jul 12, 2022 •

edited

pleunv commented Jul 28, 2022 •

edited