Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add path mapping support to ESM and CJS loaders #1585

Open
wants to merge 79 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
3be1fde
Add path mapping support to ESM loader
geigerzaehler Dec 29, 2021
c542d98
fixup! Add path mapping support to ESM loader
geigerzaehler Dec 29, 2021
0cbfa6c
fixup! fixup! Add path mapping support to ESM loader
geigerzaehler Dec 30, 2021
e7082b1
fixup! fixup! fixup! Add path mapping support to ESM loader
geigerzaehler Dec 30, 2021
69a397b
Merge remote-tracking branch 'origin/main' into path-mapping
cspotcode Jan 24, 2022
2fcb982
address code review comments
geigerzaehler Jan 24, 2022
48f5262
map esm paths in all included files
geigerzaehler Jan 24, 2022
a19454b
improve error message when mapped module is not found
geigerzaehler Jan 24, 2022
32e26e8
Review changes; add CommonJS path mapping
cspotcode Jan 25, 2022
362935b
fix failing tests
cspotcode Jan 25, 2022
e573fd7
add path mapping to docs
cspotcode Jan 25, 2022
0012b22
add flag to enable/disable path mapping in the two loaders
cspotcode Jan 25, 2022
b6352e3
fix windows tests?
cspotcode Jan 25, 2022
85643fd
fix windows tests?
cspotcode Jan 25, 2022
9c46688
Merge remote-tracking branch 'origin/main' into path-mapping
cspotcode Jan 31, 2022
bede1b6
add path mapping option to docs
cspotcode Jan 31, 2022
80746c2
changes
cspotcode Jan 31, 2022
885b7b1
replace equal (deprecated) with strictEqual
charles-allen Feb 25, 2022
c3dbe73
extract shared tsconfig
charles-allen Feb 25, 2022
8c5fd23
move import targets 2-deep (to support combined baseUrl + * path)
charles-allen Feb 25, 2022
e66d236
test: baseUrl + no paths
charles-allen Feb 25, 2022
23f40b1
test: baseUrl + * path
charles-allen Feb 25, 2022
54fdbba
test: fallback to node_modules
charles-allen Feb 25, 2022
78652b2
clean up destructuring
charles-allen Feb 25, 2022
b8e6fb7
fix setting project
charles-allen Feb 25, 2022
2973399
test: fallback to built-in
charles-allen Feb 25, 2022
5ffd905
avoid space in command & add comment
charles-allen Feb 25, 2022
d200301
fix setting project and PATH
charles-allen Feb 27, 2022
c621af0
test: skip type-defs
charles-allen Feb 27, 2022
ec038c1
test: external imports ignore paths
charles-allen Feb 27, 2022
d471d1d
tests: relative/base-relative imports ignore paths
charles-allen Feb 27, 2022
d5728dc
Merge remote-tracking branch 'origin/main' into path-mapping
cspotcode Mar 1, 2022
5b686e1
Updates
cspotcode Mar 1, 2022
cb3706e
add missing package.json from node_modules
cspotcode Mar 1, 2022
d00bf73
rolling back a config change; I was wrong
cspotcode Mar 1, 2022
398db86
add helper execEsm(...)
charles-allen Mar 12, 2022
a91d5d7
test: decouple base-url-no-paths test (by extracting it)
charles-allen Mar 12, 2022
7829b54
test: decouple skip-type-definition test (by extracting it)
charles-allen Mar 12, 2022
9fd944e
test: refactor to apply tests across multiple module types & project …
charles-allen Mar 14, 2022
4e0d363
temporarily move old tests out of the way
charles-allen Mar 14, 2022
09cc514
test: refactor again to restore shared examples/node_modules (while m…
charles-allen Mar 14, 2022
70d9bf5
import `assert` async so we can assert pre-conditions (& so sut impor…
charles-allen Mar 14, 2022
e50abd4
sync tsconfigs
charles-allen Mar 14, 2022
73d6acc
fix error message in `import-node-built-in` (it's not a precondition)
charles-allen Mar 14, 2022
d02cbed
try to clean up cjs imports :/
charles-allen Mar 14, 2022
283309d
simplify tests
charles-allen Mar 14, 2022
f2e2f65
test: restore "ignore type definition" tests
charles-allen Mar 14, 2022
93ac2ac
delete obsolete examples
charles-allen Mar 14, 2022
bf7bfcd
fix star paths
charles-allen Mar 14, 2022
c26b9dd
move old tests a bit more out of the way
charles-allen Mar 14, 2022
09ebf63
Merge remote-tracking branch 'origin/main' into path-mapping
cspotcode Mar 15, 2022
e0af738
ensure ts-node is installed before running tests
cspotcode Mar 15, 2022
31acd50
disable typechecking in tests
cspotcode Mar 15, 2022
36377fc
test: add a case that imports an esm lib from node_modules
charles-allen Mar 28, 2022
56ef378
use require everywhere in cjs examples
charles-allen Mar 28, 2022
4dcff42
destructure import proxyLodash; align both depends-on-lodash deps as cjs
charles-allen Mar 28, 2022
ba176bf
fix baseUrl
charles-allen Mar 28, 2022
38ad3f5
revert assertions to use expect(err).toBeNull(); Clean up config cons…
charles-allen Mar 28, 2022
1687de9
prefer function declaration to arrow function
charles-allen Mar 28, 2022
8e9b117
move base 2 deep, so star path can be used in addition to base
charles-allen Mar 28, 2022
806ce74
tests: non-relative imports
charles-allen Mar 28, 2022
0fd8a5c
fix import extensions
charles-allen Mar 28, 2022
02818e3
tests: imports from js, jsx, tsx
charles-allen Mar 28, 2022
6faedd9
rename under-base to below-base (nicer lexical sort)
charles-allen Mar 28, 2022
64bb679
tests: relative imports
charles-allen Mar 28, 2022
058d2e2
tests: import invalid path
charles-allen Mar 28, 2022
4a3b6b4
test: should not use star-path to resolve relative import
charles-allen Mar 28, 2022
dc7e950
tests: basic path mapping
charles-allen Mar 28, 2022
c5ea9b0
fix import style
charles-allen Mar 28, 2022
7bfe31a
tests: map using first available candidate
charles-allen Mar 28, 2022
c8c35ca
tests: more specific path; static path
charles-allen Mar 28, 2022
99ca516
comment out file-system-base-relative import
charles-allen Mar 28, 2022
4fefc57
tests: mapping from js, jsx, tsx files
charles-allen Mar 28, 2022
3a0067e
clean up (delete all old tests)
charles-allen Mar 28, 2022
78e3eda
tweaks before pulling in the latest main branch
cspotcode May 18, 2022
4f5bc35
Merge remote-tracking branch 'origin/main' into path-mapping
cspotcode May 18, 2022
ef926b9
fix
cspotcode May 18, 2022
7552fc4
style tweak
cspotcode May 19, 2022
6632481
turn on experimental resolver; add required file extensions to esm tests
cspotcode May 19, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
45 changes: 22 additions & 23 deletions dist-raw/node-errors.js
@@ -1,30 +1,29 @@
exports.codes = {
ERR_INPUT_TYPE_NOT_ALLOWED: createErrorCtor(joinArgs('ERR_INPUT_TYPE_NOT_ALLOWED')),
ERR_INVALID_ARG_VALUE: createErrorCtor(joinArgs('ERR_INVALID_ARG_VALUE')),
ERR_INVALID_MODULE_SPECIFIER: createErrorCtor(joinArgs('ERR_INVALID_MODULE_SPECIFIER')),
ERR_INVALID_PACKAGE_CONFIG: createErrorCtor(joinArgs('ERR_INVALID_PACKAGE_CONFIG')),
ERR_INVALID_PACKAGE_TARGET: createErrorCtor(joinArgs('ERR_INVALID_PACKAGE_TARGET')),
ERR_MANIFEST_DEPENDENCY_MISSING: createErrorCtor(joinArgs('ERR_MANIFEST_DEPENDENCY_MISSING')),
ERR_MODULE_NOT_FOUND: createErrorCtor((path, base, type = 'package') => {
return `Cannot find ${type} '${path}' imported from ${base}`
}),
ERR_PACKAGE_IMPORT_NOT_DEFINED: createErrorCtor(joinArgs('ERR_PACKAGE_IMPORT_NOT_DEFINED')),
ERR_PACKAGE_PATH_NOT_EXPORTED: createErrorCtor(joinArgs('ERR_PACKAGE_PATH_NOT_EXPORTED')),
ERR_UNSUPPORTED_DIR_IMPORT: createErrorCtor(joinArgs('ERR_UNSUPPORTED_DIR_IMPORT')),
ERR_UNSUPPORTED_ESM_URL_SCHEME: createErrorCtor(joinArgs('ERR_UNSUPPORTED_ESM_URL_SCHEME')),
ERR_UNKNOWN_FILE_EXTENSION: createErrorCtor(joinArgs('ERR_UNKNOWN_FILE_EXTENSION')),
}
exports.codes = {}

function joinArgs(name) {
return (...args) => {
return [name, ...args].join(' ')
function defineError(code, buildMessage) {
if (!buildMessage) {
buildMessage = (...args) => args.join(' ')
}
}

function createErrorCtor(errorMessageCreator) {
return class CustomError extends Error {
exports.codes[code] = class CustomError extends Error {
constructor(...args) {
super(errorMessageCreator(...args))
super(`${code}: ${buildMessage(...args)}`)
this.code = code
}
}
}

defineError("ERR_INPUT_TYPE_NOT_ALLOWED")
defineError("ERR_INVALID_ARG_VALUE")
defineError("ERR_INVALID_MODULE_SPECIFIER")
defineError("ERR_INVALID_PACKAGE_CONFIG")
defineError("ERR_INVALID_PACKAGE_TARGET")
defineError("ERR_MANIFEST_DEPENDENCY_MISSING")
defineError("ERR_MODULE_NOT_FOUND", (path, base, type = 'package') => {
return `Cannot find ${type} '${path}' imported from ${base}`
})
defineError("ERR_PACKAGE_IMPORT_NOT_DEFINED")
defineError("ERR_PACKAGE_PATH_NOT_EXPORTED")
defineError("ERR_UNSUPPORTED_DIR_IMPORT")
defineError("ERR_UNSUPPORTED_ESM_URL_SCHEME")
defineError("ERR_UNKNOWN_FILE_EXTENSION")
87 changes: 87 additions & 0 deletions src/cjs-resolve-filename-hook.ts
Expand Up @@ -45,6 +45,54 @@ export function installCommonjsResolveHookIfNecessary(tsNodeService: Service) {
...rest
);

// #region path-mapping
// Note: [SYNC-PATH-MAPPING] keep this logic synced with the corresponding ESM implementation.
let candidateSpecifiers: string[] = [request];
const attemptPathMapping =
tsNodeService.commonjsPathMapping &&
parent?.filename &&
!tsNodeService.ignored(parent.filename);
if (attemptPathMapping) {
const mappedSpecifiers = tsNodeService.mapPath(request);
if (mappedSpecifiers) {
candidateSpecifiers = mappedSpecifiers;
}
}
// Attempt all resolutions. Collect resolution failures and throw an
// aggregated error if they all fail.
const moduleNotFoundErrors = [];
for (let i = 0; i < candidateSpecifiers.length; i++) {
try {
// TODO does this break if `options.paths` is passed? Should we bail if
// we receive `options.paths`?
return originalResolveFilename.call(
this,
candidateSpecifiers[i],
parent,
isMain,
options
);
} catch (err: any) {
const isNotFoundError = err.code === 'MODULE_NOT_FOUND';
if (!isNotFoundError) {
throw err;
}
moduleNotFoundErrors.push(err);
}
}
// If only one candidate, no need to wrap it.
if (candidateSpecifiers.length === 1) {
throw moduleNotFoundErrors[0];
} else {
throw new MappedCommonJSModuleNotFoundError(
request,
parent!.filename,
candidateSpecifiers,
moduleNotFoundErrors
);
}
// #endregion

// This is a stub to support other pull requests that will be merged in the near future
// Right now, it does nothing.
return originalResolveFilename.call(
Expand All @@ -57,3 +105,42 @@ export function installCommonjsResolveHookIfNecessary(tsNodeService: Service) {
);
}
}

interface NodeCommonJSModuleNotFoundError extends Error {
requireStack?: string[];
}

class MappedCommonJSModuleNotFoundError extends Error {
// Same code as other module not found errors.
readonly code = 'MODULE_NOT_FOUND' as const;
readonly errors!: ReadonlyArray<Error>;
readonly requireStack?: string[];

constructor(
specifier: string,
parentFilename: string,
candidates: string[],
moduleNotFoundErrors: Error[]
) {
super(
[
`Cannot find '${specifier}' imported from ${parentFilename} using TypeScript path mapping`,
'Candidates attempted:',
...candidates.map((candidate) => `- ${candidate}`),
].join('\n')
);
// TODO this differs slightly from nodejs errors; see if we can match them
this.name = `Error [${this.code}]`;
// Match shape of `AggregateError`
Object.defineProperty(this, 'errors', {
value: moduleNotFoundErrors,
configurable: true,
writable: true,
});
// Assume every `requireStack` is identical, and maybe downstream code is doing
// something with it
this.requireStack = (
moduleNotFoundErrors[0] as NodeCommonJSModuleNotFoundError | undefined
)?.requireStack;
}
}
2 changes: 2 additions & 0 deletions src/configuration.ts
Expand Up @@ -277,6 +277,7 @@ function filterRecognizedTsConfigTsNodeOptions(jsonObject: any): {
moduleTypes,
experimentalReplAwait,
swc,
experimentalPathMapping,
experimentalResolverFeatures,
...unrecognized
} = jsonObject as TsConfigOptions;
Expand All @@ -301,6 +302,7 @@ function filterRecognizedTsConfigTsNodeOptions(jsonObject: any): {
scopeDir,
moduleTypes,
swc,
experimentalPathMapping,
experimentalResolverFeatures,
};
// Use the typechecker to make sure this implementation has the correct set of properties
Expand Down
81 changes: 75 additions & 6 deletions src/esm.ts
Expand Up @@ -11,6 +11,7 @@ import {
UrlWithStringQuery,
fileURLToPath,
pathToFileURL,
URL,
} from 'url';
import { extname } from 'path';
import * as assert from 'assert';
Expand Down Expand Up @@ -159,13 +160,52 @@ export function createEsmHooks(tsNodeService: Service) {
return defer();
}

// pathname is the path to be resolved
// Note: [SYNC-PATH-MAPPING] keep this logic synced with the corresponding CJS implementation.
let candidateSpecifiers: string[] = [specifier];

if (tsNodeService.esmPathMapping && context.parentURL) {
const parentUrl = new URL(context.parentURL);
const parentPath =
parentUrl.protocol === 'file:' && fileURLToPath(parentUrl);
if (parentPath && !tsNodeService.ignored(parentPath)) {
const mappedSpecifiers = tsNodeService.mapPath(specifier);
if (mappedSpecifiers) {
candidateSpecifiers = mappedSpecifiers.map((path) =>
pathToFileURL(path).toString()
);
}
}
}

return nodeResolveImplementation.defaultResolve(
specifier,
context,
defaultResolve
);
// Attempt all resolutions. Collect resolution failures and throw an
// aggregated error if they all fail.
const moduleNotFoundErrors = [];
for (let i = 0; i < candidateSpecifiers.length; i++) {
try {
return await nodeResolveImplementation.defaultResolve(
candidateSpecifiers[i],
context,
defaultResolve
);
} catch (err: any) {
const isNotFoundError = err.code === 'ERR_MODULE_NOT_FOUND';
if (!isNotFoundError) {
throw err;
cspotcode marked this conversation as resolved.
Show resolved Hide resolved
}
moduleNotFoundErrors.push(err);
}
}
// If only one candidate, no need to wrap it.
if (candidateSpecifiers.length === 1) {
throw moduleNotFoundErrors[0];
} else {
throw new MappedModuleNotFoundError(
specifier,
context.parentURL,
candidateSpecifiers,
moduleNotFoundErrors
);
}
}

// `load` from new loader hook API (See description at the top of this file)
Expand Down Expand Up @@ -302,3 +342,32 @@ export function createEsmHooks(tsNodeService: Service) {
return { source: emittedJs };
}
}

class MappedModuleNotFoundError extends Error {
// Same code as other module not found errors.
readonly code = 'ERR_MODULE_NOT_FOUND' as const;
readonly errors!: ReadonlyArray<Error>;

constructor(
specifier: string,
base: string,
candidates: string[],
moduleNotFoundErrors: Error[]
) {
super(
[
`Cannot find '${specifier}' imported from ${base} using TypeScript path mapping`,
'Candidates attempted:',
...candidates.map((candidate) => `- ${candidate}`),
].join('\n')
);
// TODO this differs slightly from nodejs errors; see if we can match them
this.name = `Error [${this.code}]`;
// Match shape of `AggregateError`
Object.defineProperty(this, 'errors', {
value: moduleNotFoundErrors,
configurable: true,
writable: true,
});
}
}
cspotcode marked this conversation as resolved.
Show resolved Hide resolved
44 changes: 44 additions & 0 deletions src/index.ts
Expand Up @@ -25,6 +25,7 @@ import {
} from './module-type-classifier';
import { createResolverFunctions } from './resolver-functions';
import type { createEsmHooks as createEsmHooksFn } from './esm';
import { createPathMapper } from './path-mapping';
import {
installCommonjsResolveHookIfNecessary,
ModuleConstructorWithInternals,
Expand Down Expand Up @@ -364,6 +365,17 @@ export interface CreateOptions {
* @default console.log
*/
tsTrace?: (str: string) => void;
/**
* Enable TypeScript path mapping in the ESM loader, CommonJS loader, or both.
* Today, the default is 'esm' to map paths in the experimental ESM loader but not
* CommonJS. In the next major release, the default will become 'both'.
*
* Note: If you use tsconfig-paths, be sure to disable it before enabling ts-node's CommonJS path mapper.
* tsconfig-paths already maps paths in CommonJS but not ESM, and it may conflict with ts-node's mapper.
*
* See: https://www.typescriptlang.org/docs/handbook/module-resolution.html#path-mapping
*/
experimentalPathMapping?: 'both' | 'esm' | 'cjs' | 'none';
}

/** @internal */
Expand Down Expand Up @@ -495,6 +507,18 @@ export interface Service {
enableExperimentalEsmLoaderInterop(): void;
/** @internal */
transpileOnly: boolean;
/**
* @internal
*
* Map import paths to candidates according to the `paths` compiler
* option. Returns `null` if the specifier did not match and was not
* mapped.
*/
mapPath(specifier: string): string[] | null;
cspotcode marked this conversation as resolved.
Show resolved Hide resolved
/** @internal */
commonjsPathMapping: boolean;
/** @internal */
esmPathMapping: boolean;
}

/**
Expand Down Expand Up @@ -741,6 +765,21 @@ export function create(rawOptions: CreateOptions = {}): Service {
});
}

if (
![undefined, null, 'both', 'esm', 'cjs', 'none'].includes(
options.experimentalPathMapping
)
) {
throw new Error(
`experimentalPathMapping must be one of: "both", "esm", "cjs", "none"`
);
}
const experimentalPathMapping = options.experimentalPathMapping ?? 'esm';
const commonjsPathMapping =
experimentalPathMapping === 'both' || experimentalPathMapping === 'cjs';
const esmPathMapping =
experimentalPathMapping === 'both' || experimentalPathMapping === 'esm';

/**
* True if require() hooks should interop with experimental ESM loader.
* Enabled explicitly via a flag since it is a breaking change.
Expand Down Expand Up @@ -1335,6 +1374,8 @@ export function create(rawOptions: CreateOptions = {}): Service {
});
}

const mapPath = createPathMapper(config.options);

return {
[TS_NODE_SERVICE_BRAND]: true,
ts,
Expand All @@ -1351,6 +1392,9 @@ export function create(rawOptions: CreateOptions = {}): Service {
installSourceMapSupport,
enableExperimentalEsmLoaderInterop,
transpileOnly,
mapPath,
commonjsPathMapping,
esmPathMapping,
};
}

Expand Down