Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(common): Fix handling of input source maps #6561

Merged
merged 25 commits into from
Dec 2, 2022
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
26 changes: 9 additions & 17 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/swc/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ swc_plugin_runner = { version = "0.78.1", path = "../swc_plugin_runner", optiona
swc_timer = { version = "0.17.17", path = "../swc_timer" }
swc_visit = { version = "0.5.2", path = "../swc_visit" }
tracing = "0.1.32"
url = "2.3.1"

[dependencies.napi-derive]
default-features = false
Expand Down
26 changes: 21 additions & 5 deletions crates/swc/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ use swc_ecma_visit::{noop_visit_type, FoldWith, Visit, VisitMutWith, VisitWith};
pub use swc_error_reporters::handler::{try_with_handler, HandlerOpts};
pub use swc_node_comments::SwcComments;
use swc_timer::timer;
use url::Url;

pub use crate::builder::PassBuilder;
use crate::config::{
Expand Down Expand Up @@ -328,20 +329,35 @@ impl Compiler {
}
InputSourceMap::Str(ref s) => {
if s == "inline" {
const NEEDLE: &str = "sourceMappingURL=";
// Load inline source map by simple string
// operations
let s = "sourceMappingURL=data:application/json;base64,";
let idx = fm.src.rfind(s);
let idx = fm.src.rfind(NEEDLE);
let idx = match idx {
None => bail!(
"failed to parse inline source map: `sourceMappingURL` not found"
),
Some(v) => v,
};
let encoded = &fm.src[idx + s.len()..];
let data_url = fm.src[idx + NEEDLE.len()..].trim();
let url = Url::parse(data_url).with_context(|| {
format!("failed to parse inline source map url\n{}", data_url)
})?;

let res = base64::decode(encoded.as_bytes())
.context("failed to decode base64-encoded source map")?;
let idx = match url.path().find("base64,") {
Some(v) => v,
None => {
bail!("failed to parse inline source map: not base64: {:?}", url)
}
};

let content = url.path()[idx + "base64,".len()..].trim();

let res = base64::decode_config(
content.as_bytes(),
base64::Config::new(base64::CharacterSet::Standard, true),
)
.context("failed to decode base64-encoded source map")?;

Ok(Some(sourcemap::SourceMap::from_slice(&res).context(
"failed to read input source map from inlined base64 encoded string",
Expand Down
14 changes: 14 additions & 0 deletions crates/swc/tests/fixture/next-39878/input/.swcrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"env": {
"targets": {
"chrome": "95"
}
},
"jsc": {
"parser": {
"syntax": "typescript",
"tsx": true
}
},
"sourceMaps": true
}
63 changes: 63 additions & 0 deletions crates/swc/tests/fixture/next-39878/input/box-model.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import type { UIStore, UIThunkAction } from "ui/actions";
import { isInspectorSelected } from "ui/reducers/app";
import { AppStartListening } from "ui/setup/listenerMiddleware";
import { getBoundingRectAsync, getComputedStyleAsync } from "ui/suspense/styleCaches";

import { nodeSelected } from "../../markup/reducers/markup";
import { LAYOUT_NUMERIC_FIELDS, Layout, layoutUpdated } from "../reducers/box-model";

export function setupBoxModel(store: UIStore, startAppListening: AppStartListening) {
// Any time a new node is selected in the "Markup" panel,
// try to update the box model layout data
startAppListening({
actionCreator: nodeSelected,
effect: async (action, listenerApi) => {
const { extra, getState, dispatch } = listenerApi;
const { ThreadFront, protocolClient, replayClient } = extra;
const state = getState();
const { selectedNode, tree } = state.markup;

if (!isInspectorSelected(state) || !selectedNode || !ThreadFront.currentPause?.pauseId) {
return;
}

const nodeInfo = tree.entities[selectedNode];

if (!nodeInfo) {
return;
}

const [bounds, style] = await Promise.all([
getBoundingRectAsync(
protocolClient,
ThreadFront.sessionId!,
ThreadFront.currentPause.pauseId,
selectedNode
),
getComputedStyleAsync(
protocolClient,
ThreadFront.sessionId!,
ThreadFront.currentPause.pauseId,
selectedNode
),
]);

if (!bounds || !style) {
return;
}

const layout = {
width: parseFloat(bounds.width.toPrecision(6)),
height: parseFloat(bounds.height.toPrecision(6)),
autoMargins: {},
} as Layout;

for (const prop of LAYOUT_NUMERIC_FIELDS) {
layout[prop] = style.get(prop)!;
}

// Update the redux store with the latest layout properties and update the box model view.
dispatch(layoutUpdated(layout));
},
});
}
52 changes: 52 additions & 0 deletions crates/swc/tests/fixture/next-39878/output/box-model.map
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
{
"mappings": "AACA,SAASA,mBAAmB,QAAQ,kBAAkB;AAEtD,SAASC,oBAAoB,EAAEC,qBAAqB,QAAQ,0BAA0B;AAEtF,SAASC,YAAY,QAAQ,+BAA+B;AAC5D,SAASC,qBAAqB,EAAUC,aAAa,QAAQ,wBAAwB;AAErF,OAAO,SAASC,cAAcC,KAAc,EAAEC,iBAAoC,EAAE;IAClF,yDAAyD;IACzD,0CAA0C;IAC1CA,kBAAkB;QAChBC,eAAeN;QACfO,QAAQ,OAAOC,QAAQC,cAAgB;YACrC,MAAM,EAAEC,MAAK,EAAEC,SAAQ,EAAEC,SAAQ,EAAE,GAAGH;YACtC,MAAM,EAAEI,YAAW,EAAEC,eAAc,EAAEC,aAAY,EAAE,GAAGL;YACtD,MAAMM,QAAQL;YACd,MAAM,EAAEM,aAAY,EAAEC,KAAI,EAAE,GAAGF,MAAMG,MAAM;YAE3C,IAAI,CAACtB,oBAAoBmB,UAAU,CAACC,gBAAgB,CAACJ,YAAYO,YAAY,EAAEC,SAAS;gBACtF;YACF,CAAC;YAED,MAAMC,WAAWJ,KAAKK,QAAQ,CAACN,aAAa;YAE5C,IAAI,CAACK,UAAU;gBACb;YACF,CAAC;YAED,MAAM,CAACE,QAAQC,MAAM,GAAG,MAAMC,QAAQC,GAAG,CAAC;gBACxC7B,qBACEgB,gBACAD,YAAYe,SAAS,EACrBf,YAAYO,YAAY,CAACC,OAAO,EAChCJ;gBAEFlB,sBACEe,gBACAD,YAAYe,SAAS,EACrBf,YAAYO,YAAY,CAACC,OAAO,EAChCJ;aAEH;YAED,IAAI,CAACO,UAAU,CAACC,OAAO;gBACrB;YACF,CAAC;YAED,MAAMI,SAAS;gBACbC,OAAOC,WAAWP,OAAOM,KAAK,CAACE,WAAW,CAAC;gBAC3CC,QAAQF,WAAWP,OAAOS,MAAM,CAACD,WAAW,CAAC;gBAC7CE,aAAa,CAAC;YAChB;YAEA,KAAK,MAAMC,QAAQlC,sBAAuB;gBACxC4B,MAAM,CAACM,KAAK,GAAGV,MAAMW,GAAG,CAACD;YAC3B;YAEA,0FAA0F;YAC1FvB,SAASV,cAAc2B;QACzB;IACF;AACF,CAAC",
"names": [
"isInspectorSelected",
"getBoundingRectAsync",
"getComputedStyleAsync",
"nodeSelected",
"LAYOUT_NUMERIC_FIELDS",
"layoutUpdated",
"setupBoxModel",
"store",
"startAppListening",
"actionCreator",
"effect",
"action",
"listenerApi",
"extra",
"getState",
"dispatch",
"ThreadFront",
"protocolClient",
"replayClient",
"state",
"selectedNode",
"tree",
"markup",
"currentPause",
"pauseId",
"nodeInfo",
"entities",
"bounds",
"style",
"Promise",
"all",
"sessionId",
"layout",
"width",
"parseFloat",
"toPrecision",
"height",
"autoMargins",
"prop",
"get"
],
"sources": [
"../../input/box-model.ts"
],
"sourcesContent": [
"import type { UIStore, UIThunkAction } from \"ui/actions\";\nimport { isInspectorSelected } from \"ui/reducers/app\";\nimport { AppStartListening } from \"ui/setup/listenerMiddleware\";\nimport { getBoundingRectAsync, getComputedStyleAsync } from \"ui/suspense/styleCaches\";\n\nimport { nodeSelected } from \"../../markup/reducers/markup\";\nimport { LAYOUT_NUMERIC_FIELDS, Layout, layoutUpdated } from \"../reducers/box-model\";\n\nexport function setupBoxModel(store: UIStore, startAppListening: AppStartListening) {\n // Any time a new node is selected in the \"Markup\" panel,\n // try to update the box model layout data\n startAppListening({\n actionCreator: nodeSelected,\n effect: async (action, listenerApi) => {\n const { extra, getState, dispatch } = listenerApi;\n const { ThreadFront, protocolClient, replayClient } = extra;\n const state = getState();\n const { selectedNode, tree } = state.markup;\n\n if (!isInspectorSelected(state) || !selectedNode || !ThreadFront.currentPause?.pauseId) {\n return;\n }\n\n const nodeInfo = tree.entities[selectedNode];\n\n if (!nodeInfo) {\n return;\n }\n\n const [bounds, style] = await Promise.all([\n getBoundingRectAsync(\n protocolClient,\n ThreadFront.sessionId!,\n ThreadFront.currentPause.pauseId,\n selectedNode\n ),\n getComputedStyleAsync(\n protocolClient,\n ThreadFront.sessionId!,\n ThreadFront.currentPause.pauseId,\n selectedNode\n ),\n ]);\n\n if (!bounds || !style) {\n return;\n }\n\n const layout = {\n width: parseFloat(bounds.width.toPrecision(6)),\n height: parseFloat(bounds.height.toPrecision(6)),\n autoMargins: {},\n } as Layout;\n\n for (const prop of LAYOUT_NUMERIC_FIELDS) {\n layout[prop] = style.get(prop)!;\n }\n\n // Update the redux store with the latest layout properties and update the box model view.\n dispatch(layoutUpdated(layout));\n },\n });\n}\n"
],
"version": 3
}
41 changes: 41 additions & 0 deletions crates/swc/tests/fixture/next-39878/output/box-model.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import { isInspectorSelected } from "ui/reducers/app";
import { getBoundingRectAsync, getComputedStyleAsync } from "ui/suspense/styleCaches";
import { nodeSelected } from "../../markup/reducers/markup";
import { LAYOUT_NUMERIC_FIELDS, layoutUpdated } from "../reducers/box-model";
export function setupBoxModel(store, startAppListening) {
// Any time a new node is selected in the "Markup" panel,
// try to update the box model layout data
startAppListening({
actionCreator: nodeSelected,
effect: async (action, listenerApi)=>{
const { extra , getState , dispatch } = listenerApi;
const { ThreadFront , protocolClient , replayClient } = extra;
const state = getState();
const { selectedNode , tree } = state.markup;
if (!isInspectorSelected(state) || !selectedNode || !ThreadFront.currentPause?.pauseId) {
return;
}
const nodeInfo = tree.entities[selectedNode];
if (!nodeInfo) {
return;
}
const [bounds, style] = await Promise.all([
getBoundingRectAsync(protocolClient, ThreadFront.sessionId, ThreadFront.currentPause.pauseId, selectedNode),
getComputedStyleAsync(protocolClient, ThreadFront.sessionId, ThreadFront.currentPause.pauseId, selectedNode)
]);
if (!bounds || !style) {
return;
}
const layout = {
width: parseFloat(bounds.width.toPrecision(6)),
height: parseFloat(bounds.height.toPrecision(6)),
autoMargins: {}
};
for (const prop of LAYOUT_NUMERIC_FIELDS){
layout[prop] = style.get(prop);
}
// Update the redux store with the latest layout properties and update the box model view.
dispatch(layoutUpdated(layout));
}
});
}
20 changes: 16 additions & 4 deletions crates/swc/tests/source_map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ use std::{

use anyhow::{Context, Error};
use swc::{
config::{Config, IsModule, ModuleConfig, Options, SourceMapsConfig},
config::{Config, InputSourceMap, IsModule, ModuleConfig, Options, SourceMapsConfig},
Compiler,
};
use testing::{assert_eq, NormalizedOutput, StdErr, Tester};
use walkdir::WalkDir;

fn file(f: &str) -> Result<(), StdErr> {
fn file(f: &str, config: Config) -> Result<(), StdErr> {
Tester::new().print_errors(|cm, handler| {
let path = canonicalize(f).expect("failed to canonicalize");

Expand All @@ -32,7 +32,7 @@ fn file(f: &str) -> Result<(), StdErr> {
config: Config {
is_module: IsModule::Bool(true),
inline_sources_content: true.into(),
..Default::default()
..config
},
swcrc: true,
source_maps: Some(SourceMapsConfig::Bool(true)),
Expand Down Expand Up @@ -69,7 +69,7 @@ fn file(f: &str) -> Result<(), StdErr> {

#[test]
fn issue_622() {
file("tests/srcmap/issue-622/index.js").unwrap();
file("tests/srcmap/issue-622/index.js", Default::default()).unwrap();
}

fn inline(f: &str) -> Result<(), StdErr> {
Expand Down Expand Up @@ -412,3 +412,15 @@ fn should_work_with_emit_source_map_columns() {
Ok(())
});
}

#[test]
fn issue_4578() {
file(
"tests/srcmap/issue-4578/after-babel.js",
Config {
input_source_map: Some(InputSourceMap::Str("inline".into())),
..Default::default()
},
)
.unwrap();
}
kdy1 marked this conversation as resolved.
Show resolved Hide resolved
6 changes: 6 additions & 0 deletions crates/swc/tests/srcmap/issue-4578/after-babel.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion crates/swc_common/src/source_map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1281,12 +1281,17 @@ impl SourceMap {
let mut col = max(chpos, linechpos) - min(chpos, linechpos);

if let Some(orig) = &orig {
if let Some(token) = orig.lookup_token(line, col) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's actually (what I consider to be) a bug in lookup_token, where it'll return an a mapping span on a previous line for spans that:

  • extend beyond the last mapped line
  • lines that don't have a mapping
  • unmapped columns on lines with a mapping

I tried to get them to fix it in getsentry/rust-sourcemap#53, but they rejected it. This breaks with Mozilla's source map behavior, because lines endings are supposed to split the mapping from any previous one.

Anyways, the "fix" is to do:

orig.lookup_token(line, col).filter(|t| t.get_dst_line() == line)

Is this the bug you're trying to fix with the find?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly. Thank you!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the reason, but it didn't work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh, we probably have an off-by-one. The visualized link you posted is definitely doing the lookup on the wrong line. The sourcemap crate uses 0-based lines, based on the readme example.

if let Some(token) = orig
.tokens()
.find(|token| token.get_src_line() == line - 1 && token.get_src_col() == col)
kdy1 marked this conversation as resolved.
Show resolved Hide resolved
{
line = token.get_src_line() + 1;
col = token.get_src_col();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice we're not pulling the name from the found token, which means we'll always the name found in SWC's input. But, the name could have already been changed by whatever preceeded SWC.

if let Some(src) = token.get_source() {
src_id = builder.add_source(src);
}
} else {
continue;
}
}

Expand Down