feat(es/ast): Add `raw` to `Str` #4071

alexander-akait · 2022-03-17T21:23:35Z

Description:

feat - add raw property for StringLiteral

Also:

fix JSX with only \n in props
improve perf for normal printing strings + print them as they were written by developer
simplify TemplateLiteral - do not store StringLiteral (use JsWord), we just duplicate AST node, but it is unnecessary

~~Note - there is another improvement and simplification in TemplateLiteral, but I will do it in another PR~~ We need do it here to better rewrite code
Also we should add raw for numbers too, so we will support linting 1_000_000 ad also will print numbers as they were written by developer

@ArturAralin after this we can simplify quotes rule, no need to look at original code (it is slow)

BREAKING CHANGE:

Yes

Related issue (if exists):

No

Also:

we have duplicate IsDirective in parser and swc_ecma_utils

kdy1 · 2022-03-17T22:56:40Z

crates/swc_ecma_utils/src/macros.rs

@@ -31,8 +31,7 @@ macro_rules! quote_str {
        $crate::swc_ecma_ast::Str {
            span: $span,
            value: $s.into(),
-            has_escape: false,
-            kind: Default::default(),
+            raw: $s.into(),


Doesn't we need ' or "?

kdy1 · 2022-03-20T06:06:17Z

crates/swc/tests/fixture/issue-1227/case-1/output/index.js

@@ -1 +1 @@
-var foo = "\xa0";
+var foo = "\u{a0}";


This is wrong as the target is es5.

kdy1

Thank you so much!

kdy1 · 2022-03-20T15:10:49Z

crates/swc_ecma_parser/src/parser/typescript.rs

-                    kind: StrKind::Normal {
-                        contains_quote: false,
-                    },
+                    raw: Some(("\"".to_owned() + &v.to_string() + "\"").into()),


I think this would be bad for performance.

I think

raw: Some(format!("\"{}\"", v)),

would work

You can manually use String for better performance, but I don' think it will have a huge didfference.
(I mean String::push & String::push_str)

kdy1 · 2022-03-21T04:54:39Z

crates/swc_ecma_ast/src/lit.rs

 }

 impl Take for Str {
    fn dummy() -> Self {
        Str {
            span: DUMMY_SP,
            value: js_word!(""),
-            has_escape: Default::default(),
-            kind: Default::default(),
+            raw: Some("\"\"".into()),


I think this should be None

kdy1 · 2022-03-21T04:55:21Z

crates/swc_ecma_ast/src/expr.rs

 }

 impl Take for TplElement {
    fn dummy() -> Self {
        TplElement {
            span: DUMMY_SP,
            tail: Default::default(),
-            cooked: Take::dummy(),
-            raw: Take::dummy(),
+            cooked: Some("".into()),


kdy1 · 2022-03-21T12:39:06Z

crates/swc_ecma_parser/src/parser/typescript.rs

-                    kind: StrKind::Normal {
-                        contains_quote: false,
-                    },
+                    raw: Some(format!("\"{}\"", v).into()),


I think you missed this?

Is having this better?

@kdy1 Hm, do you mean

You can manually use String for better performance, but I don' think it will have a huge didfference.
(I mean String::push & String::push_str)

Just create String::new()? I missed that because it is recovery mode

I think in real code this will not happen... but I can refactor to using String

I mean something like

{

let mut buf = String::new();
// ...
}

(I'm afk now)

kdy1 · 2022-03-21T12:51:30Z

crates/swc_ecma_minifier/tests/terser/compress/unicode/issue_2242_3/output.js

@@ -1 +1 @@
-console.log("\ud83d\ude00", "\ud83d@\ude00");
+console.log("\\\0ud83d\\\0ude00", "\\\0ud83d@\\\0ude00");


I think this is wrong, as the input was

console.log("\ud83d" + "\ude00", "\ud83d" + "@" + "\ude00");

@kdy1 I see... They are surrogates pairs... Looking for workaround

kdy1 · 2022-03-21T14:15:48Z

crates/swc_ecma_minifier/tests/terser/compress/conditionals/cond_7/output.js

-condition(), (x = 20);
-z || condition(), (x = "fuji");
-x = (condition(), "foobar");
+x = y ? "foo" : "foo";


This should not be modified

crates/swc_ecma_minifier/tests/terser/compress/concat-strings/concat_1/output.js

alexander-akait · 2022-03-21T14:54:49Z

@kdy1 Fixed 👍

...s/compress/fixture/next/feedback-util-promisify/chunks/pages/_app-72ad41192608e93a/output.js

kdy1 · 2022-03-21T15:13:26Z

...a_minifier/tests/compress/fixture/projects/wmr/archive-1/chunks/meta-tags.c6b3dbd5/output.js

@@ -1,7 +1,7 @@
 import { m } from "../index.f66dda46.js";
 import { u as useLang, a as useTitleTemplate, b as useTitle, c as useMeta, d as useLink } from "./hoofd.module.6c5395cb.js";
 function MetaTags() {
-    return useLang("nl"), useTitleTemplate("%s | \u{1F4AD}"), useTitle("Welcome to hoofd"), useMeta({
+    return useLang("nl"), useTitleTemplate("%s | 💭"), useTitle("Welcome to hoofd"), useMeta({


I think this can be problematic

hm, it was original code, so we keep them as is, original code:

useTitleTemplate("%s | 💭")

Because target is not es5, it will not be problem

crates/swc_ecma_minifier/tests/terser/compress/object/property_with_unprintable/output.js

crates/swc_ecma_codegen/src/lib.rs

kdy1

LGTM.
Thank you so much for the hard work.

swc-bump:

swc_ecma_ast --breaking

kdy1 reviewed Mar 17, 2022

View reviewed changes

alexander-akait force-pushed the feat-improve-ast-strings branch 13 times, most recently from 73238f9 to 34c53c2 Compare March 19, 2022 22:36

kdy1 reviewed Mar 20, 2022

View reviewed changes

kdy1 requested changes Mar 21, 2022

View reviewed changes

alexander-akait force-pushed the feat-improve-ast-strings branch from c77ed1c to 629d8bc Compare March 21, 2022 12:15

kdy1 reviewed Mar 21, 2022

View reviewed changes

kdy1 requested changes Mar 21, 2022

View reviewed changes

kdy1 reviewed Mar 21, 2022

View reviewed changes

crates/swc_ecma_minifier/tests/terser/compress/concat-strings/concat_1/output.js Show resolved Hide resolved

kdy1 requested changes Mar 21, 2022

View reviewed changes

alexander-akait mentioned this pull request Mar 21, 2022

fix(ecma/parser): consume surrogates #4115

Merged

kdy1 requested changes Mar 22, 2022

View reviewed changes

crates/swc_ecma_codegen/src/lib.rs Show resolved Hide resolved

kdy1 previously approved these changes Mar 22, 2022

View reviewed changes

kdy1 dismissed their stale review via f98d571 March 22, 2022 07:27

alexander-akait added 3 commits March 22, 2022 16:27

test: more

554e597

feat: implement raw field for token

1c7d531

feat: implement raw

555db2e

alexander-akait and others added 20 commits March 22, 2022 16:27

fix: raw for acorn

1746d5f

fix: compatibility with acorn and raw

6327e33

test: fix

9627873

test: fix

f5a6fa7

test: fix

32c9112

refactor: use Some

79df77e

refactor: code

b9e1305

Wrong rebase

79c91d4

fixup

f5af826

fix: es5 compatibility codegen

144396c

test: fix

6a85382

refactor: code

908f4b8

test: update

2779b9b

test: update

7bdec61

test: update

11f8621

fix: workaround for surrogate pair

b763d7c

fix: eq ignore span

84072ff

refactor: fix cargo file

2e0ea5d

fix: source map

f008453

Simple

93d76b2

kdy1 force-pushed the feat-improve-ast-strings branch from f98d571 to 93d76b2 Compare March 22, 2022 07:28

Update test refs

cb4dfab

kdy1 changed the title ~~Feat improve ast strings~~ feat(es/ast): Add raw to Str Mar 22, 2022

kdy1 added this to the v1.2.160 milestone Mar 22, 2022

kdy1 enabled auto-merge (squash) March 22, 2022 07:52

kdy1 approved these changes Mar 22, 2022

View reviewed changes

kdy1 merged commit 634d732 into swc-project:main Mar 22, 2022

kdy1 mentioned this pull request Mar 22, 2022

Producing wrong string when minify enabled #4120

Closed

alexander-akait deleted the feat-improve-ast-strings branch March 22, 2022 10:27

swc-project locked as resolved and limited conversation to collaborators Oct 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(es/ast): Add `raw` to `Str` #4071

feat(es/ast): Add `raw` to `Str` #4071

alexander-akait commented Mar 17, 2022 •

edited

kdy1 Mar 17, 2022

kdy1 Mar 20, 2022

kdy1 left a comment

kdy1 Mar 20, 2022

kdy1 Mar 20, 2022

kdy1 Mar 21, 2022

kdy1 Mar 21, 2022

kdy1 Mar 21, 2022

kdy1 Mar 21, 2022

alexander-akait Mar 21, 2022

alexander-akait Mar 21, 2022

kdy1 Mar 21, 2022

kdy1 Mar 21, 2022

alexander-akait Mar 21, 2022 •

edited

kdy1 Mar 21, 2022

alexander-akait commented Mar 21, 2022

kdy1 Mar 21, 2022

alexander-akait Mar 21, 2022

kdy1 left a comment

		@@ -1 +1 @@
		console.log("\ud83d\ude00", "\ud83d@\ude00");
		console.log("\\\0ud83d\\\0ude00", "\\\0ud83d@\\\0ude00");

feat(es/ast): Add raw to Str #4071

feat(es/ast): Add raw to Str #4071

Conversation

alexander-akait commented Mar 17, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kdy1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexander-akait Mar 21, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexander-akait commented Mar 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kdy1 left a comment

Choose a reason for hiding this comment

feat(es/ast): Add `raw` to `Str` #4071

feat(es/ast): Add `raw` to `Str` #4071

alexander-akait commented Mar 17, 2022 •

edited

alexander-akait Mar 21, 2022 •

edited