fix: replace `source-map` with `@jridgewell/trace-mapping` #743

SimenB · 2023-09-09T07:10:52Z

Following up istanbuljs/v8-to-istanbul#186

I don't think this is visible to consumers?

SimenB · 2023-09-09T07:19:50Z

packages/istanbul-lib-source-maps/lib/map-store.js

-                                filePath
-                            );
-                            this.sourceStore.set(sourceFilePath, content);
+                        if (s) {


types says s is string | null

Correct. If sources content is enabled, some tools will use null if the file's source content isn't available. So some files in the array will have source content (a string), and some won't (null).

SimenB · 2023-09-09T09:19:07Z

packages/istanbul-lib-source-maps/test/map-store.test.js

@@ -90,7 +90,7 @@ describe('map store', () => {
                    },
                    end: {
                        line: 5,
-                        column: Infinity
+                        column: 1


not sure if good or bad?

What is the source file, output file, and source map? I don't return an Infinity value in trace-mapping, and I don't know what computes end here.

I don't really know this code base, so the best answer I have for your q is "this test" 😅

I've gotten admin access to this repo, so I'm able to merge this now. I'd like to verify this one, tho. So if you're able to take a look, I'd be grateful!

What is the source file, output file, and source map?

I believe there are present in a comment right below:

istanbuljs/packages/istanbul-lib-source-maps/test/map-store.test.js

Lines 136 to 302 in 5584b50

// Original source

// export class SimpleClass {

// hy() {

// console.log("Hy");

// }

// }

// Transpiled Source

// "use strict";

// var SimpleClass = (function () {

// function SimpleClass() {

// }

// SimpleClass.prototype.hy = function () {

// console.log("Hy");

// };

// return SimpleClass;

// }());

// exports.SimpleClass = SimpleClass;

const coverageData = {

'test.js': {

path: 'test.js',

statementMap: {

'1': {

start: {

line: 2,

column: 19

},

end: {

line: 9,

column: 3

}

},

'2': {

start: {

line: 5,

column: 4

},

end: {

line: 7,

column: 6

}

},

'3': {

start: {

line: 6,

column: 8

},

end: {

line: 6,

column: 26

}

},

'4': {

start: {

line: 8,

column: 4

},

end: {

line: 8,

column: 23

}

},

'5': {

start: {

line: 10,

column: 0

},

end: {

line: 10,

column: 34

}

}

},

fnMap: {

'1': {

name: '(anonymous_1)',

decl: {

start: {

line: 2,

column: 19

},

end: {

line: 2,

column: 20

}

},

loc: {

start: {

line: 2,

column: 31

},

end: {

line: 9,

column: 1

}

}

},

'2': {

name: 'SimpleClass',

decl: {

start: {

line: 3,

column: 13

},

end: {

line: 3,

column: 24

}

},

loc: {

start: {

line: 3,

column: 27

},

end: {

line: 4,

column: 5

}

}

},

'3': {

name: '(anonymous_3)',

decl: {

start: {

line: 5,

column: 31

},

end: {

line: 5,

column: 32

}

},

loc: {

start: {

line: 5,

column: 43

},

end: {

line: 7,

column: 5

}

}

}

},

branchMap: {},

s: {

'1': 1,

'2': 1,

'3': 1,

'4': 1,

'5': 1

},

f: {

'1': 1,

'2': 1,

'3': 1

},

b: {},

inputSourceMap: {

version: 3,

file: 'test.js',

sourceRoot: '',

sources: ['test.ts'],

names: [],

mappings:

';AAAA;IAAA;IAIA,CAAC;IAHG,wBAAE,GAAF;QACI,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IACtB,CAAC;IACL,kBAAC;AAAD,CAAC,AAJD,IAIC;AAJY,mBAAW,cAIvB,CAAA'

}

}

Input lines 136-141, output 142-152 and inputSourceMap is lines 293-301

The origin of Infinity I'm guessing is from:

istanbuljs/packages/istanbul-lib-source-maps/lib/get-mapping.js

Lines 57 to 72 in 5584b50

if (

// If this is null, it means that we've hit the end of the file,

// so we can use Infinity as the end column.

afterEndMapping.line === null ||

// If these don't match, it means that the call to

// 'generatedPositionFor' didn't find any other original mappings on

// the line we gave, so consider the binding to extend to infinity.

sourceMap.originalPositionFor(afterEndMapping).line !==

beforeEndMapping.line

) {

return {

source: beforeEndMapping.source,

line: beforeEndMapping.line,

column: Infinity

};

}

(with line 64 being originalPositionFor(sourceMap, afterEndMapping).line on this branch)

@jridgewell sorry to ping you again, but does the added context above help in evaluating if the change is correct (or at least not regressive) or not?

All good, I missed this during Thanksgiving week.

Based on this, the visualization we're tracing is this.

The column that's changed is the end location for the anonymous IIFE. The mappings on line 9 are (using 1-based line and 0-based column as source-map does):

{ genColumn: 0, sourcesIndex: 0, sourceLine: 5, sourceColumn: 0 }

{ genColumn: 1, sourcesIndex: 0, sourceLine: 5, sourceColumn: 1 }

{ genColumn: 1, sourcesIndex: 0, sourceLine: 1, sourceColumn: 0 }

{ genColumn: 5, sourcesIndex: 0, sourceLine: 5, sourceColumn: 1 }

Notice that genColumn 1 has 2 mappings. In trace-mapping when doing the originalPositionFor search with { line: 9, column: 1 }, the first specified is returned (due to GREATEST_LOWER_BOUND). So beforeEndMapping points to { line: 5, column: 1 } in the original code. We then run a generatedPositionFor search, returning { line: 10, column: 34 }. Note that the last seen generated position that points to that source location is used (because of the LEAST_UPPER_BOUND bias), and there are many in this example.

However source-map uses a first-match style binary search. So our originalPostitionFor returns any match for the generated position { line: 9, column: 1 }. In this case, it hits { genColumn: 1, sourcesIndex: 0, sourceLine: 1, sourceColumn: 0 } first. Doing the generatedPositionFor with that position gives { line: 2, column: 0 }.

I dislike the results that source-map gives when there are multiple matches. If there were a different number of mappings on that line, then a different original position could be hit by the binary search.

I know we've discussed originalEndPositionFor in other PRs, but I can't find it now. The way that you're doing this kinda buggy. You're returning a single generated position out of the mapping, and that generated position could be sourced any line (hence the line check). In a case like this, you could incorrectly return an Infinity end when there could be more code on that source line that would actually end that range. What you really want is to get the list of every generated position corresponding to that source location, and see if any of them are on the correct line. Ie, call allGeneratedPositionsFor

Oh hey, it was #685. Looks like my opinion hasn't changed since I wrote that, we should be using allGeneratedPositionsFor. 😄

#768 shows that this same result happens when using allGeneratedPositionsFor to test all positions matching our end location.

Oh, I completely missed #685! I'll land that then (after your new PR). Thank you!

AriPerkkio

I tested these changes on Vitest's internal coverage tests (V8 and Istanbul) by patching the package. All tests pass like usual and coverage reports are identical.

Anything missing from this PR? Happy to see source-map replaced with lighter package.

SimenB · 2023-10-19T16:42:28Z

Missing a review pretty much - I cannot self review 😅

All tests pass in Jest as well with this patch.

bcoe

@SimenB on vacation and am starting to look through a few old PRs.

This update looks innocent enough to me, but I'm worried about the change from Infinity to 1 too ... and admittedly don't know this part of the codebase too well 😅

I think this might be able to left open for a bit until someone has the time to dig in deeply.

See reasoning in istanbuljs#743 (comment) and istanbuljs#685 (comment). This will enable us to switch to `TraceMap` without any test failures.

…sforms (#768) See reasoning in #743 (comment) and #685 (comment). This will enable us to switch to `TraceMap` without any test failures.

SimenB · 2024-02-26T09:23:02Z

Duplicate of #685

SimenB · 2024-02-26T09:31:15Z

https://github.com/istanbuljs/istanbuljs/releases/tag/istanbul-lib-source-maps-v5.0.0

fix: replace source-map with @jridgewell/trace-mapping

75d0a63

SimenB requested review from bcoe and coreyfarrell September 9, 2023 07:10

SimenB commented Sep 9, 2023

View reviewed changes

AriPerkkio approved these changes Oct 19, 2023

View reviewed changes

SimenB and others added 2 commits November 20, 2023 08:01

Merge branch 'master' into replace-source-map

7196e32

Merge branch 'main' into replace-source-map

f419111

bcoe reviewed Feb 19, 2024

View reviewed changes

jridgewell mentioned this pull request Feb 22, 2024

Use allGeneratedPositionsFor for more accurate source map transforms #768

Merged

SimenB pushed a commit that referenced this pull request Feb 26, 2024

fix: use allGeneratedPositionsFor for more accurate source map tran…

c6d0982

…sforms (#768) See reasoning in #743 (comment) and #685 (comment). This will enable us to switch to `TraceMap` without any test failures.

SimenB closed this Feb 26, 2024

SimenB marked this as a duplicate of #685 Feb 26, 2024

SimenB deleted the replace-source-map branch February 26, 2024 09:23

AriPerkkio mentioned this pull request Feb 27, 2024

chore(deps): update istanbul-lib-source-maps to v5 vitest-dev/vitest#5304

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: replace `source-map` with `@jridgewell/trace-mapping` #743

fix: replace `source-map` with `@jridgewell/trace-mapping` #743

SimenB commented Sep 9, 2023

SimenB Sep 9, 2023

jridgewell Sep 19, 2023

SimenB Sep 9, 2023

jridgewell Sep 19, 2023

SimenB Sep 19, 2023

SimenB Nov 20, 2023

SimenB Nov 20, 2023 •

edited

SimenB Feb 20, 2024

jridgewell Feb 21, 2024

jridgewell Feb 22, 2024

jridgewell Feb 22, 2024

SimenB Feb 22, 2024

AriPerkkio left a comment

SimenB commented Oct 19, 2023 •

edited

bcoe left a comment

SimenB commented Feb 26, 2024

SimenB commented Feb 26, 2024

	// Original source
	// export class SimpleClass {
	// hy() {
	// console.log("Hy");
	// }
	// }
	// Transpiled Source
	// "use strict";
	// var SimpleClass = (function () {
	// function SimpleClass() {
	// }
	// SimpleClass.prototype.hy = function () {
	// console.log("Hy");
	// };
	// return SimpleClass;
	// }());
	// exports.SimpleClass = SimpleClass;
	const coverageData = {
	'test.js': {
	path: 'test.js',
	statementMap: {
	'1': {
	start: {
	line: 2,
	column: 19
	},
	end: {
	line: 9,
	column: 3
	}
	},
	'2': {
	start: {
	line: 5,
	column: 4
	},
	end: {
	line: 7,
	column: 6
	}
	},
	'3': {
	start: {
	line: 6,
	column: 8
	},
	end: {
	line: 6,
	column: 26
	}
	},
	'4': {
	start: {
	line: 8,
	column: 4
	},
	end: {
	line: 8,
	column: 23
	}
	},
	'5': {
	start: {
	line: 10,
	column: 0
	},
	end: {
	line: 10,
	column: 34
	}
	}
	},
	fnMap: {
	'1': {
	name: '(anonymous_1)',
	decl: {
	start: {
	line: 2,
	column: 19
	},
	end: {
	line: 2,
	column: 20
	}
	},
	loc: {
	start: {
	line: 2,
	column: 31
	},
	end: {
	line: 9,
	column: 1
	}
	}
	},
	'2': {
	name: 'SimpleClass',
	decl: {
	start: {
	line: 3,
	column: 13
	},
	end: {
	line: 3,
	column: 24
	}
	},
	loc: {
	start: {
	line: 3,
	column: 27
	},
	end: {
	line: 4,
	column: 5
	}
	}
	},
	'3': {
	name: '(anonymous_3)',
	decl: {
	start: {
	line: 5,
	column: 31
	},
	end: {
	line: 5,
	column: 32
	}
	},
	loc: {
	start: {
	line: 5,
	column: 43
	},
	end: {
	line: 7,
	column: 5
	}
	}
	}
	},
	branchMap: {},
	s: {
	'1': 1,
	'2': 1,
	'3': 1,
	'4': 1,
	'5': 1
	},
	f: {
	'1': 1,
	'2': 1,
	'3': 1
	},
	b: {},
	inputSourceMap: {
	version: 3,
	file: 'test.js',
	sourceRoot: '',
	sources: ['test.ts'],
	names: [],
	mappings:
	';AAAA;IAAA;IAIA,CAAC;IAHG,wBAAE,GAAF;QACI,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IACtB,CAAC;IACL,kBAAC;AAAD,CAAC,AAJD,IAIC;AAJY,mBAAW,cAIvB,CAAA'
	}
	}

	if (
	// If this is null, it means that we've hit the end of the file,
	// so we can use Infinity as the end column.
	afterEndMapping.line === null \|\|
	// If these don't match, it means that the call to
	// 'generatedPositionFor' didn't find any other original mappings on
	// the line we gave, so consider the binding to extend to infinity.
	sourceMap.originalPositionFor(afterEndMapping).line !==
	beforeEndMapping.line
	) {
	return {
	source: beforeEndMapping.source,
	line: beforeEndMapping.line,
	column: Infinity
	};
	}

fix: replace source-map with @jridgewell/trace-mapping #743

fix: replace source-map with @jridgewell/trace-mapping #743

Conversation

SimenB commented Sep 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimenB Nov 20, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AriPerkkio left a comment

Choose a reason for hiding this comment

SimenB commented Oct 19, 2023 • edited

bcoe left a comment

Choose a reason for hiding this comment

SimenB commented Feb 26, 2024

SimenB commented Feb 26, 2024

fix: replace `source-map` with `@jridgewell/trace-mapping` #743

fix: replace `source-map` with `@jridgewell/trace-mapping` #743

SimenB Nov 20, 2023 •

edited

SimenB commented Oct 19, 2023 •

edited