Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create parser plugin "topLevelAwait" #10449

Merged
merged 8 commits into from Oct 29, 2019
Merged

Create parser plugin "topLevelAwait" #10449

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7c95240
Create parser plugin "topLevelAwait"
nicolo-ribaudo Sep 17, 2019
14aa7e4
Update test262 whitelist
nicolo-ribaudo Sep 17, 2019
9a16a3e
Update ts typings
nicolo-ribaudo Sep 17, 2019
94cfa86
Fix "sourceType: unambiguous" with TLA
nicolo-ribaudo Oct 18, 2019
be80170
Ambiguous tokens after await
nicolo-ribaudo Oct 21, 2019
fd83281
Update await %x(0)
nicolo-ribaudo Oct 22, 2019
b73f31c
typo [skip ci]
nicolo-ribaudo Oct 26, 2019
c3d892e
Typo [skip ci]
nicolo-ribaudo Oct 29, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
25 changes: 21 additions & 4 deletions packages/babel-parser/src/index.js
View file
Open in desktop
Expand Up @@ -25,15 +25,32 @@ export function parse(input: string, options?: Options): File {
const parser = getParser(options, input);
const ast = parser.parse();

// Rather than try to parse as a script first, we opt to parse as a module and convert back
// to a script where possible to avoid having to do a full re-parse of the input content.
if (!parser.sawUnambiguousESM) ast.program.sourceType = "script";
if (parser.sawUnambiguousESM) {
return ast;
}

if (parser.ambiguousScriptDifferentAst) {
// Top level await introduces code which can be both a valid script and
// a valid module, but which produces different ASTs:
// await
// 0
// can be parser either as an AwaitExpression, or as two ExpressionStatements.
nicolo-ribaudo marked this conversation as resolved.
Show resolved Hide resolved
try {
options.sourceType = "script";
return getParser(options, input).parse();
} catch {}
} else {
// This is both a valid module and a valid script, but
// we parse it as a script by default
ast.program.sourceType = "script";
}

return ast;
} catch (moduleError) {
try {
options.sourceType = "script";
return getParser(options, input).parse();
} catch (scriptError) {}
} catch {}

throw moduleError;
}
Expand Down
1 change: 1 addition & 0 deletions packages/babel-parser/src/parser/base.js
View file
Open in desktop
Expand Up @@ -13,6 +13,7 @@ export default class BaseParser {
plugins: PluginsMap;
filename: ?string;
sawUnambiguousESM: boolean = false;
ambiguousScriptDifferentAst: boolean = false;

// Initialized by Tokenizer
state: State;
Expand Down
25 changes: 25 additions & 0 deletions packages/babel-parser/src/parser/expression.js
View file
Open in desktop
Expand Up @@ -2208,6 +2208,7 @@ export default class ExpressionParser extends LValParser {
isAwaitAllowed(): boolean {
if (this.scope.inFunction) return this.scope.inAsync;
if (this.options.allowAwaitOutsideFunction) return true;
if (this.hasPlugin("topLevelAwait")) return this.inModule;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If top-level await is only valid in modules, should seeing one set sawUnambiguousESM? In other words, how should this behave with a sourceType of unambiguous? Currently, parsing await 0 with unambiguous yields an AST with sourceType of script, even though if you'd specified script it would not have parsed:

> require('./').parse('await 0', { plugins: ['topLevelAwait'], sourceType: 'unambiguous' }).program.sourceType
'script'
> require('./').parse('await 0', { plugins: ['topLevelAwait'], sourceType: 'script' }).program.sourceType
Thrown:
{ [SyntaxError: Unexpected token, expected ";" (1:6)
] pos: 6, loc: Position { line: 1, column: 6 } }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tricky because, for exmaple, this is both a valid script and a valid module:

await
0

I can set it to a module when await the next invalid token is on the same line.

return false;
}

Expand All @@ -2234,9 +2235,33 @@ export default class ExpressionParser extends LValParser {
);
}

if (!this.scope.inFunction && !this.options.allowAwaitOutsideFunction) {
if (
this.hasPrecedingLineBreak() ||
// All the following expressions are ambiguous:
// await + 0, await - 0, await ( 0 ), await [ 0 ], await / 0 /u, await ``
this.match(tt.plusMin) ||
this.match(tt.parenL) ||
this.match(tt.bracketL) ||
this.match(tt.backQuote) ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! I missed await ` 0 ` .

I come up with a new case: If one turns on v8intrinsic, the following could be ambiguous, too.

await %a()

// Sometimes the tokenizer generates tt.slash for regexps, and this is
// handler by parseExprAtom
this.match(tt.regexp) ||
this.match(tt.slash) ||
// This code couls be parsed both as a modulo operator or as an intrinsic:
nicolo-ribaudo marked this conversation as resolved.
Show resolved Hide resolved
// await %x(0)
(this.hasPlugin("v8intrinsic") && this.match(tt.modulo))
) {
this.ambiguousScriptDifferentAst = true;
} else {
this.sawUnambiguousESM = true;
}
}

if (!this.state.soloAwait) {
node.argument = this.parseMaybeUnary();
}

return this.finishNode(node, "AwaitExpression");
}

Expand Down
6 changes: 1 addition & 5 deletions packages/babel-parser/src/parser/statement.js
View file
Open in desktop
Expand Up @@ -495,11 +495,7 @@ export default class StatementParser extends ExpressionParser {
this.state.labels.push(loopLabel);

let awaitAt = -1;
if (
(this.scope.inAsync ||
(!this.scope.inFunction && this.options.allowAwaitOutsideFunction)) &&
this.eatContextual("await")
) {
if (this.isAwaitAllowed() && this.eatContextual("await")) {
awaitAt = this.state.lastTokStart;
}
this.scope.enter(SCOPE_OTHER);
Expand Down
1 change: 1 addition & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/export-default/input.js
View file
Open in desktop
@@ -0,0 +1 @@
export default await 0;
4 changes: 4 additions & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/export-default/options.json
View file
Open in desktop
@@ -0,0 +1,4 @@
{
"plugins": ["topLevelAwait"],
"sourceType": "module"
}
85 changes: 85 additions & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/export-default/output.json
View file
Open in desktop
@@ -0,0 +1,85 @@
{
"type": "File",
"start": 0,
"end": 23,
"loc": {
"start": {
"line": 1,
"column": 0
},
"end": {
"line": 1,
"column": 23
}
},
"program": {
"type": "Program",
"start": 0,
"end": 23,
"loc": {
"start": {
"line": 1,
"column": 0
},
"end": {
"line": 1,
"column": 23
}
},
"sourceType": "module",
"interpreter": null,
"body": [
{
"type": "ExportDefaultDeclaration",
"start": 0,
"end": 23,
"loc": {
"start": {
"line": 1,
"column": 0
},
"end": {
"line": 1,
"column": 23
}
},
"declaration": {
"type": "AwaitExpression",
"start": 15,
"end": 22,
"loc": {
"start": {
"line": 1,
"column": 15
},
"end": {
"line": 1,
"column": 22
}
},
"argument": {
"type": "NumericLiteral",
"start": 21,
"end": 22,
"loc": {
"start": {
"line": 1,
"column": 21
},
"end": {
"line": 1,
"column": 22
}
},
"extra": {
"rawValue": 0,
"raw": "0"
},
"value": 0
}
}
}
],
"directives": []
}
}
1 change: 1 addition & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/for-await-module/input.js
View file
Open in desktop
@@ -0,0 +1 @@
for await (const a of b);
4 changes: 4 additions & 0 deletions ...ges/babel-parser/test/fixtures/experimental/top-level-await/for-await-module/options.json
View file
Open in desktop
@@ -0,0 +1,4 @@
{
"plugins": ["topLevelAwait"],
"sourceType": "module"
}
134 changes: 134 additions & 0 deletions ...ages/babel-parser/test/fixtures/experimental/top-level-await/for-await-module/output.json
View file
Open in desktop
@@ -0,0 +1,134 @@
{
"type": "File",
"start": 0,
"end": 25,
"loc": {
"start": {
"line": 1,
"column": 0
},
"end": {
"line": 1,
"column": 25
}
},
"program": {
"type": "Program",
"start": 0,
"end": 25,
"loc": {
"start": {
"line": 1,
"column": 0
},
"end": {
"line": 1,
"column": 25
}
},
"sourceType": "module",
"interpreter": null,
"body": [
{
"type": "ForOfStatement",
"start": 0,
"end": 25,
"loc": {
"start": {
"line": 1,
"column": 0
},
"end": {
"line": 1,
"column": 25
}
},
"await": true,
"left": {
"type": "VariableDeclaration",
"start": 11,
"end": 18,
"loc": {
"start": {
"line": 1,
"column": 11
},
"end": {
"line": 1,
"column": 18
}
},
"declarations": [
{
"type": "VariableDeclarator",
"start": 17,
"end": 18,
"loc": {
"start": {
"line": 1,
"column": 17
},
"end": {
"line": 1,
"column": 18
}
},
"id": {
"type": "Identifier",
"start": 17,
"end": 18,
"loc": {
"start": {
"line": 1,
"column": 17
},
"end": {
"line": 1,
"column": 18
},
"identifierName": "a"
},
"name": "a"
},
"init": null
}
],
"kind": "const"
},
"right": {
"type": "Identifier",
"start": 22,
"end": 23,
"loc": {
"start": {
"line": 1,
"column": 22
},
"end": {
"line": 1,
"column": 23
},
"identifierName": "b"
},
"name": "b"
},
"body": {
"type": "EmptyStatement",
"start": 24,
"end": 25,
"loc": {
"start": {
"line": 1,
"column": 24
},
"end": {
"line": 1,
"column": 25
}
}
}
}
],
"directives": []
}
}
1 change: 1 addition & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/for-await-script/input.js
View file
Open in desktop
@@ -0,0 +1 @@
for await (const a of b);
5 changes: 5 additions & 0 deletions ...ges/babel-parser/test/fixtures/experimental/top-level-await/for-await-script/options.json
View file
Open in desktop
@@ -0,0 +1,5 @@
{
"plugins": ["topLevelAwait"],
"sourceType": "script",
"throws": "Unexpected token, expected \"(\" (1:4)"
}
1 change: 1 addition & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/inside-arrow/input.js
View file
Open in desktop
@@ -0,0 +1 @@
() => await 0;
5 changes: 5 additions & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/inside-arrow/options.json
View file
Open in desktop
@@ -0,0 +1,5 @@
{
"plugins": ["topLevelAwait"],
"sourceType": "module",
"throws": "Can not use keyword 'await' outside an async function (1:6)"
}
3 changes: 3 additions & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/inside-block/input.js
View file
Open in desktop
@@ -0,0 +1,3 @@
if (true) {
await 0;
}
4 changes: 4 additions & 0 deletions packages/babel-parser/test/fixtures/experimental/top-level-await/inside-block/options.json
View file
Open in desktop
@@ -0,0 +1,4 @@
{
"plugins": ["topLevelAwait"],
"sourceType": "module"
}