Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What’s confusing about modules? #51876

Closed
andrewbranch opened this issue Dec 13, 2022 · 60 comments
Closed

What’s confusing about modules? #51876

andrewbranch opened this issue Dec 13, 2022 · 60 comments
Labels
Discussion Issues which may not have code impact

Comments

@andrewbranch
Copy link
Member

After #51669 is merged, I plan to write documentation for it, and try to update/rewrite a bunch of our existing module-related documentation. While there are a lot of good examples in this issue tracker of specific questions and misconceptions about modules and TypeScript’s module-related options, I thought I’d ask explicitly what questions you have and what aspects of the module landscape or our configuration specifically are the most confusing.

@andrewbranch andrewbranch added the Discussion Issues which may not have code impact label Dec 13, 2022
@GabenGar
Copy link

Is there a difference between import moduleName from "module-path"; and import { default as moduleName } from "module-path";?

@andrewbranch
Copy link
Member Author

(I don’t plan to answer all or even most questions here, but if the answer is quick and contains any context that may not make it into the docs, I might.)

@GabenGar no, except there accidentally was a difference in type resolution for a while, but it’s fixed now: #49567 fixed by #49814

@treybrisbane
Copy link

treybrisbane commented Dec 13, 2022

Any chance you could include a dedicated section or two on why it's not practical to have the compiler transform imports ending in .ts to ones ending in .js? 🙂

I've seen a few of the TypeScript team posting some pretty solid rationale in various GitHub comments over the years, but I can never remember all the reasons, and finding those comments again is quite difficult. It would be amazing to get one clear, objective, well-written explanation/rationale that could be easily linked to by anyone whenever this comes up.

@IanVS
Copy link

IanVS commented Dec 13, 2022

One thing that tripped me up was how .mts and .cts files interact with package.json exports fields. Especially in the case of dynamic import() (which always uses the import condition, even in .cts files).

@Andarist
Copy link
Contributor

What are the exact requirements for publishable packages? And how do they relate to: package.json#type, package.json#exports, moduleResolution values? When do extensions in .d.ts matter? And what they should be?

@khmseu
Copy link

khmseu commented Dec 24, 2022

An import path cannot end with a '.mts' extension. Consider importing './XXX.mjs' instead.

Why does this error message even exist? It seems insane to direct tsc to import a .mjs when we have a .mts with types and everything. What am I missing here? Is there a way to direct tsc to the real type info? That suggestion certainly doesn't sound like something I want to do.

@unional
Copy link
Contributor

unional commented Jan 7, 2023

I'm working on a demo about many issues with the current module, moduleResolution, allowSyntheticDefaultImports, and esModuleInterop combinations.

It's still a work in progress. I can probably finish it by this weekend or next week.

Feel free to check it out or cooperate about this topic.

https://github.com/cyberuni/typescript-module-resolutions-demo

@bobaaaaa
Copy link

bobaaaaa commented Feb 18, 2023

My team is building a non-fancy web-app (not a library/framework) in a monorepo setup. Are there any different configs needed for app vs. library authors?

I would love to see some kind of cli check -> "Is my setup ok?" like brew doctor or https://github.com/bluwy/publint

@unional
Copy link
Contributor

unional commented Feb 18, 2023

Cross linking the great post made by Ryan: #49083 (comment)

There are still many things worth to discuss and figure out, for example:

  • How to actually consume packages in today's CJS/ESM world
  • How to support both as a library? Maintaining multiple versions? Duplicate code?
  • How to make the life of TypeScript library authors easier today. What can be done by TypeScript, community tools, library authors?
  • What's the TypeScript and JavaScript community direction

But in general, that is a good read and should check it out.

@GabenGar

This comment was marked as off-topic.

@unional

This comment was marked as off-topic.

@GabenGar

This comment was marked as off-topic.

@IanVS
Copy link

IanVS commented Feb 21, 2023

Seems like this isn't really the place to argue over ESM-only vs CJS compat...

@GabenGar

This comment was marked as off-topic.

@andrewbranch
Copy link
Member Author

Let’s keep this an uncluttered place for people to ask questions, please. Because GitHub issues don’t have a reasonable means of threading or replies, that means not answering other people’s questions, even if you have an answer or opinion. Thanks for understanding!

@RichiCoder1
Copy link

RichiCoder1 commented Mar 11, 2023

  • How to support both as a library? Maintaining multiple versions? Duplicate code?
  • How to make the life of TypeScript library authors easier today. What can be done by TypeScript, community tools, library authors?

I wanted to emphasize these two. While being either fully CJS and fully ESM w/ extensions is straight forward, any package trying to bridge these two worlds falls into numerous quietly failing pitfalls.

Specifically for dual packaging lib authors, things that would be awesome:

  • Specific recommendations around the structuring of package.json type and exports.
  • What file extensions matter and what don't in a packaged library, with and without exports.
  • A demo, demos, or living document with concrete steps lib authors can use to correctly emit dual type packages.

I think the above would be tremendously helpful as those seem to be were many authors on twitter (and myself) trip over all the steps surrounding dual packaging, Node16, and now bundler.

@me4502
Copy link
Member

me4502 commented Mar 17, 2023

One thing that would be nice to document, is how to correctly setup modules where you're using a bundler that takes in ESM code, but also tooling like Jest that operates on CJS code.

As moduleResolution: node appears to not allow resolvePackageJsonExports etc to be enabled in TypeScript 5.0, a setup like this limits the codebase from using the new customisation functionality.

An ideal setup would allow use of package.json exports/imports fields, the app code to ouput ESM to the bundler (webpack, etc), and Jest to recieve CJS output (eg, using a tsconfig.test.json file). From my testing there doesn't seem to be a "perfect" setup here, so documentation around the best way to solve something like this / best practices IMO would be ideal

@beorn
Copy link

beorn commented Mar 18, 2023

I'm taking this to mean module and module resolution. If I had to guess, I think what most people want is:

  1. A great developer experience

    • seamless compatibility between bundlers (such as vite) and tsc / vscode
    • not being pushed to deal with idiosyncrasies such as adding .js extensions to typescript module imports (extensionless)
  2. A module & resolution system they can understand

    • A path towards clearer and more standardized module & resolution systems that's actually possible to learn and which by default (with little/no config) just works for the most common cases
    • Guidelines around resolution methods in monorepos: tsconfig references vs workspace packages vs path aliases — too many ways to skin a cat
    • Tooling to give great observability into module & resolution problems — configuration linting, help pinpoint blame with 3rd party modules vs your own configuration or own code
    • Updated documentation that gives a great overview
  3. Future-proofing

    • A path to an ESM-only future, and tooling that gives a gentle hand pushing in that direction (for your own code, and a "manifesto" of sorts for the FOSS community)
    • Prepare for a typescript-first/only world — for most people that work in typescript having to even deal with dist/.js files is just a complication they'd rather do without. It seems the world is moving towards a typescript-first/only world, and i would be great to support and document how to do this better — e.g., how can packageA import from packageB through package.json exports/imports without dealing with .d.ts or .js files or configuration. This kind-of works in some situations, but it would be nice to see it improved and documented/promoted.

I'm sure thousands of man-years have been spent (in anger!) debugging JS/TS module and resolution issues. Most people have no clue what's going on, they just muddle through, turning on and off config flags until things break less. If you can make this more seamless you'd literally be saving thousands of lives — and bringing big smiles to all JS/TS devs 😀

A tangent: I have high hopes for the bundler mode, and with traceResolution there is some nice observability, but I was a bit disappointed to see that it seems to default to resolve in CJS mode with no apparent way to override?

@connorjs
Copy link

connorjs commented Mar 20, 2023

Echoing @beorn’s eloquent "path forward" and "future proofing" points in a more "Hey, I'm new" way.

  1. I come to JS/TS world hearing that JS rocks b/c you write the same thing on server and client/web
  2. But these are not the same when it comes to module resolution (I think, and other stuff)
    1. Tell me best practice ← This is my goal and what I came here to say: There should be strong recommendations 🤞🏻
    2. Set me up for success in the future
    3. Link to "in depth"* guides

*And maybe in depth guides come later, not now, or maybe just include a collection of resources.

Also, I think recommendations differ from "library" vs. "application" authoring, and I still find that odd X years later. If they are different, maybe this clarifies that too.


Awesome to see this issue! I was looking for bundler documentation on the TS website. Glad to see it's coming (soon).

Edit: I just re-read #50152 in depth, and it has a lot of great context. Unsure how I missed some of that info before. Pulling some of that into permanent documentation (or linking to it for context) would be great!

@Andarist
Copy link
Contributor

Andarist commented Apr 2, 2023

this part from --traceResolution is pretty confusing:

======== Resolving module 'react/jsx-runtime' from '~/webstudio-designer/packages/authorization-token/src/index.server.ts'. ========
Explicitly specified module resolution kind: 'Bundler'.
Resolving in CJS mode with conditions 'import', 'types', 'source'.

How does it run in "CJS mode" and uses an import condition at the same time?

@andrewbranch
Copy link
Member Author

“CJS mode” just refers to the flavor of module resolution and doesn’t actually indicate anything about the kind of the importing or imported modules here. It means index files and extensionless shenanigans are supported. Do you have a suggestion for an alternative nomenclature?

@Andarist
Copy link
Contributor

Andarist commented Apr 3, 2023

What I find more confusing about this is not the "CJS mode" but the mention of the import condition here.

@andrewbranch
Copy link
Member Author

You wrote an import statement, so the import condition gets used. That’s how bundlers do it.

@beorn
Copy link

beorn commented Apr 3, 2023

“CJS mode” just refers to the flavor of module resolution and doesn’t actually indicate anything about the kind of the importing or imported modules here. It means index files and extensionless shenanigans are supported. Do you have a suggestion for an alternative nomenclature?

At least I misunderstood what "resolving in CJS mode" meant — I thought it meant modules that were resolved would be assumed to be CJS. Is there another name that could be used to refer to the module resolution without involving ESM/CJS names (which I think will make most people think about the module format, not just resolution algorithm)?

@jedwards1211
Copy link

jedwards1211 commented Apr 3, 2023

One comment in a documentation example says that package.json "types" is a fallback for older versions of TypeScript, but it seems like TS is still resolving with "types" when "nodenext" resolution is used and the package doesn't have an export map?

Basically https://www.typescriptlang.org/docs/handbook/esm-node.html doesn't list all of the resolutions that "nodenext" will try for resolving type declarations so I'm not 100% sure how it behaves.

@Andarist
Copy link
Contributor

Andarist commented Apr 3, 2023

One comment in a documentation example says that package.json "types" is a fallback for older versions of TypeScript, but it seems like TS is still resolving with "types" when "nodenext" resolution is used and the package doesn't have an export map?

If that wouldn't be the case then we probably wouldn't be able to use those new moduleResolutions for a long time.

@andrewbranch
Copy link
Member Author

One comment in a documentation example says that package.json "types" is a fallback for older versions of TypeScript

That comment is specific to the example it’s contained in. It means to say that because there are exports, only old versions of TS would ever look at the top-level types. I agree that wording is pretty confusing. Thanks 👍

@Mahi
Copy link

Mahi commented May 25, 2023

Yes that's fair enough, I have been fighting with this issue many times before and surely there was some repressed frustration deep down, also I am not a native so sorry if I wrote something horribly when I didn't mean bad towards you, but in the end it was genuine questions on the .js extension.

anything TS can understand should arguably be legal.

Yes that makes sense, thank you.

It seems like your question was assuming that TS would make you refer to jsfile.py as "./jsfile.js" which is not and has never been the case.

No my question is: why does the TypeScript code I write need to be heavily coupled to the compiler I will use in the future? Importing a simple example.ts has to be done as import ... from './example.js', which binds my codebase to a compiler that only generates .js files with same names as the original files. I can't use a compiler with name mangling or custom extensions (see .jsx so not too far fetched).

In my mind this isn't how compiler should work, I should write import ... from 'example.ts' and essentially say "hey compiler, however and wherever you decide to compile this .ts file, I want to import the end result". I guess my question is, where is my logic going wrong and why isn't this the case with TypeScript?

@andrewbranch
Copy link
Member Author

Got it, I misunderstood your question with the .py example. You’re correct that there are some fairly deep assumptions baked in here. The original assumption was that tsc is the only TypeScript compiler, and that was true not so long ago! Even as other transpilers came around, we’ve found that continuing the model of assuming runtime behavior based on what tsc would do, and then getting your other compiler and tsc to agree as closely as possible by modifying both of their options, still works pretty well. The place where this has really started to break down most significantly is with runtimes and bundlers that understand TS natively, so we introduced allowImportingTsExtensions that can be used with noEmit, and I think that solves your example use case. That tells the compiler that it’s not going to do emit, and another tool that understands how to resolve to and process .ts files directly is going to handle it.

@not-my-profile
Copy link

not-my-profile commented Jul 3, 2023

I am really confused about why you need both --module node16 for tsc as well as "type": "module" in package.json ... isn't there some way to get the same type checking behavior without having to create a package.json file? That would certainly be convenient for quickly debugging packages that provide incorrect types under node16 module resolution.

Edit: opened #54876

@trusktr
Copy link

trusktr commented Nov 19, 2023

Is there a difference between import moduleName from "module-path"; and import { default as moduleName } from "module-path";?

off topic but there's a difference between
export default foo

and

export { foo as default }

The latter creates a live binding, while the former does not.

Solid playground without live binding

Solid playground with live binding

Type-wise no difference in TypeScript.

@trusktr
Copy link

trusktr commented Nov 19, 2023

In my mind this isn't how compiler should work, I should write import ... from 'example.ts' and essentially say "hey compiler, however and wherever you decide to compile this .ts file, I want to import the end result". I guess my question is, where is my logic going wrong and why isn't this the case with TypeScript?

@Mahi The problem is there are various different ES Module and non-ESM systems, and TypeScript needs to be configurable for all of them:

  • browser ESM
  • Node ESM
  • Deno ESM
  • Webpack ESM (very close to Node modules)
  • Rollup ESM
  • Bun ESM
  • Meteor ESM
  • other systems that consume a non-ESM format
  • other systems with no module format at all
  • etc

TypeScript cannot simply build code a single way, because then it would work in only one, maybe two, of those environments.

When you write TypeScript code, what is going to be importing your TypeScript code? Is it gonna be a browser ES module that will natively import your code? Is it gonna be a Deno ES module? Etc?

How will TypeScript know the output format that your code needs to be in, for it to work properly in some target environment, maybe even a non-ESM environment?

The answer is that in reality, there are currently too many ESM and non-ESM consumers (build tools or runtimes or both) with varying rules. Bun recently threw a wrench into the mix by mixing CommonJS with ESM in the same file (❔, don't do that unless you want to write very non-portable code).

why does the TypeScript code I write need to be heavily coupled to the compiler I will use in the future?

TLDR: because the compiler or environment you choose may have certain rules that others don't.

My advice:

Avoid moduleResolution:bundler if you can, and try to output the most vanilla ES Modules that you can so that they work with the least friction in systems that use import maps like browsers and Deno, because that format is closest to the ESM spec.

If you do this, then it'll be the simplest to get up and running with any alternative tools in the future with the least amount of config fiddling or code changes.

  • don't use package.json exports, which is a Node.js invention, not part of ES spec
  • don't use package.json imports, which is not aligned with import map spec
  • write .js extensions in all your import specifiers, learn to love it (stop caring you are in a .ts file and writing .js extensions) knowing the output simply works in the most places without config (f.e. in every browser, without having to write a ServiceWorker that auto-appends extensions before fetching)
  • don't import non-standard files (f.e. don't import a .jpg file) and instead await fetch them and export them from a JS module

Basically: when you write code as close as possible to what a browser can consume, only having to set up an import map for bare specifiers referring to library names, then your code will be as portable as possible.

The bundler option may help you write less-portable code for a specific case, but that's something to avoid.

@nlwillia
Copy link

The new theory document is referenced from the tsconfig reference for paths, but the "There is a larger coverage of paths in the handbook" statement references a #path-mapping fragment that doesn't exist, and there doesn't appear to be any discussion of paths in the page. It's particularly confusing to try to understand paths in the context of the new bundler mode which seems to defer similar mapping configurability to the package.

@andrewbranch
Copy link
Member Author

Thank you. You got linked there via a redirect from the old content. I need to update the link to point here: https://www.typescriptlang.org/docs/handbook/modules/reference.html#paths

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 2, 2024

  1. Usage of package.json#imports. From the release notes I got the impression they are supported in the same way as package.json#exports, meaning you can create a mapping between a path and an object containing "types"/"default"/etc., so that Node will use the ./dist/foobar/*.js defined in "default", but TypeScript will use ./dist/foobar/*.d.ts defined in "types". But it's not the case, because package.json#imports only supports a string on the right side of the mapping (at least that's what I understood from the Node docs, however they are not too explicit here). So what's the expected setup for the "native" #imports usage in Node16? I apologize, but I did not understand the response here.

    Is it that you need to emit the .js into the same directory as the source files? Or that you need to use path aliases to translate #foobar on the TypeScript layer (but if that was the case, then there's no particular "support" for imports, this should've been always possible).

    EDIT: It turns out that I was wrong here, see next comments. Thank you, @Andarist, for clearing this out. But the conversation we had below only proves that this could've been explained better (e.g. point out that the feature is independent of path aliases and mention how to declare those imports properly using the "conditional" setup).

  2. Creating libraries, with the emphasis on internal libraries (in monorepos). How should the setup look like, assuming Yarn/pnpm workspaces (so @acme/app properly imports from @acme/package-a thanks to symlinks in node_modules, not path aliases nor relative imports). Having this info would be even more beneficial to TypeScript itself to further lead people away from (ab)using path aliases (which I learned very recently was meant to be used with RequireJS, can't find a link to the comment on GH though). Because if we already advice people to properly use the workspaces instead of abusing path aliases (source), then we could also add more info on how to declare the manifest for these internal packages properly. And to this day I have seen only monorepo guides on the Internet that set up path aliases to reference one package from another.
    Assuming that our best-practice TS monorepo setup does not use path aliases:

    1. Is it required for a monorepo to be using project references, or are those only to support build orchestration from tsc? Especially important with the popularity of the new tools that help with the orchestration (Nx, Turbo, Moon, etc.) -- if the tools are running compilation of the affected (modified) libraries in a topological order, then tsc -b will double the work.

      1. Is the answer different if we're using tsc only for typechecking?
    2. Is it possible for a monorepo to skip emitting declaration files for internal libraries, have package.json#exports#.#types point to the source .ts? Was this ever considered to be a proper setup? (I tried that but skipLibCheck was not supported -- tsc continued typechecking the internal dependency with parent's stricter compiler settings, which ended up in errors). This would bring DX closer to the one where people use path aliases, otherwise you have to regenerate declaration files for your internal dependencies before you can work on your project in the IDE.

    3. What is the best way of self-referencing root directory of a project in imports (other than path aliases)? This relies on the answer to question 1. If not package.json#imports, then I believe the only way are just ../../relative/paths.

@GabenGar
Copy link

GabenGar commented Feb 2, 2024

  1. package.json#imports is consumed by NodeJS at runtime, so it has to point to the the output .js files in the end. Typescript resolves module paths by the rules in tsconfig.json#compilerOptions.paths instead. And yes, that means you have to almost duplicate path mappings in both files in order for typescript and nodejs to resolve module aliases without problems. In the best case scenario, aka you don't rely on syntax specifics of neither #imports nor paths and just map one module path to a single file, it amounts to prepending dist/ to your #imports paths. And at worst it might not be resolvable.

  2. i.a. "Typechecking" is just running tsc without output files. The compiler still has to statically analyze all module dependencies and therefore must be able to resolve all module paths.

  3. ii. This entirely depends on the monorepo structure. Turborepo specifically separates between applications which build output files and do not get dependent on and packages which declare their source files directly in package.json#exports and are used strictly as dependencies, therefore it does not rely on fancy typescript monorepo magic of partial builds. As far as package.json#imports in a monorepo setting concerned, all other projects with package.json are "external" and therefore do not belong in the #imports mapping.

@Andarist
Copy link
Contributor

Andarist commented Feb 2, 2024

And yes, that means you have to almost duplicate path mappings in both files in order for typescript and nodejs to resolve module aliases without problems.

That's not completely true. You are using output paths in package.json#imports and as long as those are not different from what TS would emit then you can rely on TS to resolve the source files by mapping patterns defined by package.json#imports to their respective sources.

An example test case for this can be found here. Notice how package.json#imports refer to .js files in dist directory but yet TS is able to offer auto-completions etc within src directory. That happens by matching package.json#imports against tsconfig.json#compilerOptions.rootDir and tsconfig.json#compilerOptions.outDir (don't quote this is an exhaustive description of the algorithm 😉 )

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 2, 2024

An example test case for this can be found here.

@Andarist , wait so you say that it is possible to do the conditional import?

If that's true, then I have to try once more myself and cross out the paragraph in my previous message.

Turborepo specifically separates between applications which build output files and do not get dependent on and packages which declare their source files directly in package.json#exports and are used strictly as dependencies, therefore it does not rely on fancy typescript monorepo magic of partial builds.

Thank you for mentioning this, @GabenGar. Actually, I first saw the idea of using the source files as the type source in a blog post by Turbo. But they also declared main as *.ts files, which made no sense to me, as the .ts files are not understandable by Node. I assumed they are relying heavily on some bundler.

And like you said, they rely on applications being the "leaves of the topological tree", being the projects that actually do the building. This means that the compiler options in apps cannot be "stricter" than the compiler options for the libraries, because the library is ultimately compiled with the dependent applications' compiler options. This might seem like a tiny detail, but it matters when we want to gradually introduce some stricter compiler options to a monorepo, project by project, where each project/lib can be developed by a separate team. That was one of the pitfalls I discovered when using .ts files as package.json#types/package.json#exports['.*'].types.

Typescript resolves module paths by the rules in tsconfig.json#compilerOptions.paths instead. And yes, that means you have to almost duplicate path mappings in both files in order for typescript and nodejs to resolve module aliases without problems.

@GabenGar , that was the only way before it was announced that the package.json#import is now supported by TS.

@Andarist
Copy link
Contributor

Andarist commented Feb 2, 2024

@Andarist , wait so you say that it is possible to do the conditional import?

Sure thing. Conditions are supported both in exports and imports.

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 2, 2024

Supported since version 4.7? It's only the auto-completion that will be added in 5.4, right?

@Andarist
Copy link
Contributor

Andarist commented Feb 2, 2024

Yes, the module resolution part of things was released in 4.7 (release note here). 5.4 just offers new auto-completions in this area

@GabenGar
Copy link

GabenGar commented Feb 2, 2024

@Andarist

An example test case for this can be found here.

The key word in the underlying PR is "roughly". Since paths and #imports use different ways to map several files to a path, it will only "just work" if the mapping is one-to-one, which can easily not be the case in situations like a bundled NodeJS package (and I bet the errors will be as confusing as always). So the point still stands, #imports field is still a footgun for typescript codebases, with ugly workarounds.

@akwodkiewicz

I assumed they are relying heavily on some bundler.

While technically true, it's very unlikely you'll end up in a monorepo situation without all js code lathed in typescript (which might or might not be a bundler) and without a build step.

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 2, 2024

So the point still stands, #imports field is still a footgun for typescript codebases, with ugly workarounds.

It seems we happen to work on projects with entirely different DX -- hence the disagreement.

We usually seem to be stuck between choosing solutions that either require lots of configuration or those that are super magical and in case of a non-standard issue require reading their source code to understand what they are doing.

I'm searching for a minimal, but a non-magical approach, where we are as compliant with various tools as we can get. And I'd like the TS docs to help other people get there as well.

And I believe that #imports are a "native" alternative to the path aliases, at least for Node, because they just provide the "type layer" on the stuff that is already supported by the runtime. So no bundler magic, no unnecessary configuration -- it's the sweet spot.

Let me go back to the original issue, which is about the docs themselves. The aforementioned path aliases, that 5 years ago were not explained well enough and made people (ab)use them as import syntax-sugar, today they have a single essential paragraph explaining they should not be used to reference other packages in a monorepo. It's great that this particular piece of information is now in the docs, and it's great that Yarn/pnpm workspaces are mentioned as the alternative.

What I don't get though, is that the self-referencing path aliases seem to be endorsed in the next subsection of the same document. The docs could be more specific that this is probably a good idea only if you work with an additional build step other than just tsc. And here we could be mentioning #imports as the alternative, in the same way we are mentioning Yarn workspaces as the alternative for aliases to internal libs.

Moreover, I know that the information about paths not being emitted is 2 sections above, but since the documentation is not separated into "vanilla tsc + Node/browser" and "things that make sense if your bundler allows it", reading the "wildcard pattern" section alone can still mislead developers into using the path aliases without understanding their consequences.

EDIT: what's also confusing is the part where baseUrl stopped being necessary for the path aliases to work. As a user of path aliases I obviously thought it's a good idea to reduce unnecessary "baseUrl: '.'" entry in my config. But today, knowing about the original idea behind the feature, I'm wondering if it was a good idea that we simplified the usage of the feature that at the same time we try to warn people about (did it make sense to drop baseUrl in the context of usage with RequireJS? I assume not)

@andrewbranch
Copy link
Member Author

andrewbranch commented Feb 2, 2024

I don’t know how anything in that document can be read as an endorsement. It’s a reference page that has to spell out how features work. I prefixed the section with a big disclaimer about what not to do, but eventually I had to put some examples on the page.

I'm wondering if it was a good idea that we simplified the usage of the feature that at the same time we try to warn people about (did it make sense to drop baseUrl in the context of usage with RequireJS? I assume not)

When I removed the need for baseUrl to use paths in TypeScript 4.1, we were not yet as anti-paths as we have grown to be. However, baseUrl has absolutely no use in bundler code or Node.js, while paths remains useful in some cases. I have literally never seen a project use baseUrl for any reason besides to use paths, and yet it creates a broken way to refer to every file that people had to be careful to avoid. They fundamentally never needed to be coupled, so yes, I still think it was absolutely the right choice to untangle them. I will advocate for deprecating baseUrl in 6.0, while paths will likely live forever, however many caveats we attach to it.

@akwodkiewicz
Copy link

You are right, I went too far with endorsement. What I was trying to say is that I thought that the usage of paths for imports convenience was not meant to be officially recognized while the docs show it. But reading your reply I understand that my assumptions were wrong and the de facto usage of paths was embraced by the maintaining team (hence the decoupling of baseUrl from paths).

Thank you, @andrewbranch, for sharing your thoughts on this matter.

I have literally never seen a project use baseUrl for any reason besides to use paths, and yet it creates a broken way to refer to every file that people had to be careful to avoid.

Do you mean some specific issue that can happen when using baseUrl? What is this way you're mentioning if I may ask?

@andrewbranch
Copy link
Member Author

andrewbranch commented Feb 2, 2024

{
  "compilerOptions": {
    "baseUrl": "./src"
  }
}

This means that a file ./src/foo.ts can be imported like this:

import foo from "foo";

from any file in the program, no matter its location. All “bare specifiers” like this are looked up in the baseUrl directory before trying node_modules or whatever the moduleResolution setting implies should happen.

Basically, you’re saying that an AMD loader is going to make an HTTP request with URL fragment foo, and there’s an HTTP server serving the built output of the src directory. But I’m not sure how fully that was even thought through when it was a plausible way you’d ship JS to the browser, because the resolved URL of the request is going to be dependent on the URL of the current page the script is executing from. For example, I just opened up the console and did fetch("foo") and it made a request to https://github.com/microsoft/TypeScript/issues/foo, not https://github.com/foo. If I wanted my imports to work as HTTP requests from any page, it seems like I ought to use a leading /, but baseUrl doesn’t allow that. Maybe the AMD loader accounts for this somehow, in a way that modern ESM imports from the browser wouldn’t. Whatever the reason, it’s completely irrelevant to Node.js and bundlers, and is very poorly applicable to browser-based ESM too.

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 8, 2024

@Andarist , coming back to the "imports" thing -- I can't make them work in my project.

Cannot find module '#lib/something' or its corresponding type declarations. ts(2307)

But I also don't understand how TS is supposed to work here. Could I ask for your help here? Please, take a look at the following example.


Having these compiler options in tsconfig.json:

"module": "NodeNext",
"moduleResolution": "NodeNext",
"rootDir": ".",
"outDir": "dist",

and these entries in package.json:

"type": "commonjs",
"imports": {
    "#lib/*": {
        "default": "./dist/src/*.js"
    }
}

I'm trying to import anything from the top-level of the package with"

import { something } from '#lib/something'

but it does not work. So here are the questions that could help me understand the resolution algorithm better (and at the same time fix my project configuration 😅):

  1. Where should tsc look for the types?

    "exports" define how the package should work for the users of the package, that's why it's obvious paths to .js files in exports are substituted to .d.ts files in the same directory by default. But "imports" define how the package should work for... maintainers of the package? Both users and maintainers? We cannot tell tsc to resolve the imports using the d.ts files, because they first have to be generated by tsc, and it won't be able to generate them if it cannot resolve the imports, because .d.ts files do not exist yet 😵‍💫

    So I suppose that tsc should look at the source code (.ts) files when resolving the imports. In my case ./src/*.ts

  2. In my case, where does tsc look for the types for `"default": "./dist/src/*.js"?

    Does it know it should be ./src/*.ts? Or does it look in ./dist/src/*.ts? Or maybe it looks at some other location?

    Is it possible to print this information somehow on my computer to learn where it tries to resolve #lib/something?

  3. Should I add a types conditional import?

    If the answer to 2. suggests that tsc looks at a different place than I want it to look, do I have to tell it to look elsewhere using conditionals (for example "types": "./src/*.ts")?

  4. Is my imports entry wrong?

    Am I using the wildcard properly? Should the path end with .js?

  5. Does the import in source code refer to some specific file that I don't have in my project?

    Maybe import {something} from '#lib/something' is resolved to some specific file like ./src/something.ts where I expect it to resolve to ./src/something/index.ts

  6. Are there any compiler options (or other tsconfig options) that have the effect on the feature?

    Maybe I have some more options set with some particular values that break the resolution? I did not paste all of my tsconfig options here on purpose.

EDIT: 7 [META]: Is there a specific place where I could read all about this without using your help and time and/or referring to comments under GH issues 😅

@Andarist
Copy link
Contributor

Andarist commented Feb 8, 2024

@akwodkiewicz i'll try to respond to questions sometime later - but when it comes to the given example, could you wrap it up in a repository that I could clone? that would save me some time

@Andarist
Copy link
Contributor

Andarist commented Feb 8, 2024

Btw. you can use --traceResolution to see how TS tries to resolve all of this. You might be able to spot the problem in the output.

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 8, 2024

@Andarist, I was preparing the reproduction repository and was able to reproduce the issue, but I started messing with the tsconfig.json + package.json + import statement and I finally made the imports work. I'll post a bigger comment in a moment (currently writing it), just letting you know that you don't have to go all the way with the explanation.

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 8, 2024

It all boiled down to me not understanding how Node works.

Pointing to folders will not work for subpath imports. Your imports have to point to particular files. Or more exactly, the sum of the import mapping and the thing in your import statement has to result in a particular file. I'll explain in the examples below in a while.

"Folders as modules" seem not to work for subpath imports. It's not mentioned explicitly that it won't work, there's just a following line in the docs:

Using package subpath exports or subpath imports can provide the same containment organization benefits as folders as modules, and work for both require and import.

which suggests that "subpath imports" replace "folders as modules". And the "folders as modules" feature itself is marked as legacy anyway.

So if you have a tree like this:

.
|- dist
|  |- foo
|  |  |- index.js
|  |  
|  |- bar
|  |  |-  index.js
|  |
|  |- baz
|     |-  index.js
|
|- package.json

and you want to import from foo in sibling folders bar or baz, then the subpath import #lib/foo has to point exactly to your ./dist/foo/index.js file, like so:

"imports" {
  "#lib/foo": "./dist/foo/index.js"
}

If you don't want to specify a separate subpath for each directory, you can try using wildcards, like this:

"imports" {
  "#lib/*": "./dist/*/index.js"
}

The above will work even for nested directories, because the asterisk is not a proper glob pattern, it's just string substitution.

Now, if you make the mistake of specifying the path mapping like this:

"imports" {
  "#lib/*": "./dist/*"
}

assuming that Node will resolve the folder as a module, then it won't work for '#lib/foo'. You'll need to import from '#lib/foo/index.js'.

Or if you decide to use:

"imports" {
  "#lib/*": "./dist/*.js"
}

then the correct import statement will be even weirder: '#lib/foo/index'.

———-

When I wrote the import statement including the word “index” in my project, the type acquisition worked out of the box, even without adding any conditional “types” entries to “imports”.

Now the answers to my questions are not that important, I guess, but it still would be useful to know how TS manages to find type information for the subpath imports. I believe it relies on the information from rootDir in tsconfig.json. But if I knew how it works exactly, then I'd know if it's even necessary for me to write a nested "types" entry, and what are the cases that require it.

@andrewbranch
Copy link
Member Author

Yeah, that is a tricky subtlety. Note that it applies to "exports" too.

{
  "name": "foo",
  "exports": {
    "./*": "./dist/*"
  }
}
// main.mjs
import "foo/bar.js";
// main.cjs
require("foo/bar")

Given a file node_modules/foo/dist/bar.js, you might expect each of these to work—the ESM import has to supply the file extension while the CJS require is allowed to drop it as usual. But the normal rules don’t apply once you go through an imports or exports mapping. Like you said, the result of the wildcard substitution has to be a full filename (relative to the package.json). Node.js really wanted the imports/exports algorithm to produce exactly zero or one file lookup location per request for performance reasons (file system access is expensive, to say nothing of HTTP requests in a hypothetical world where these resolutions might take place across a network).

This is also one way the imports/exports resolution algorithm diverges from the tsconfig paths algorithm. With paths, extension searching and directory modules still work on the result of the wildcard substitution, if it would happen for an equivalent request that didn’t map through paths.

@akwodkiewicz
Copy link

akwodkiewicz commented Feb 8, 2024

But the normal rules don’t apply once you go through an imports or exports mapping

I have just said jokingly to a colleague that “Node forgot to backport the folders as modules feature to imports”.

I understand the decision, and seeing the feature marked as legacy helped me tie the pieces together. That was the similarity of exports and imports I failed to recognise.

I thought the similarity is the support of the conditional “types” entry. I got too fixated on trying to point the type sources to TypeScript with the conditional entry, that I just did not realize that I’m writing improper code, resulting in Node runtime errors after transpilation.

This is also one way the imports/exports resolution algorithm diverges from the tsconfig paths algorithm. With paths, extension searching and directory modules still work on the result of the wildcard substitution, if it would happen for an equivalent request that didn’t map through paths.

And the reason it will still “work in runtime” (when translated with the help of bundlers or tools like tsconfig-paths) is that these alias paths end up being relative paths. And “folders as modules” is a feature working exactly just with the relative paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Discussion Issues which may not have code impact
Projects
None yet
Development

No branches or pull requests