Community Feedback: Project Service APIs

[JoshuaKGoldberg]

Back in Relative TSConfig Projects with parserOptions.project = true > Project Services, we'd mentioned we're working on a replacement for parserOptions.project:

The downside of having users specify parserOptions.project at all is that @typescript-eslint/parser needs manual logic to create TypeScript Programs and associate them with linted files.
...
We're working on an option to instead call the same TypeScript "Project Service" APIs that editors such as VS Code use to create Programs for us instead. Project Services will automatically detect the TSConfig for each file (like project: true), and will also allow type information to be computed for JavaScript files without the allowJs compiler option (unlike project: true).

The EXPERIMENTAL_useProjectService parser API is that experimental alternative to parserOptions.project. We hope it brings simpler ESLint configurations, mildly faster lint times, and more closely aligned type information for lint rules to what editors produce.

module.exports = {
  // ...
  parser: '@typescript-eslint/parser',
  parserOptions: {
-   project: true,
+   EXPERIMENTAL_useProjectService: true,
    tsconfigRootDir: __dirname,
  },
  // ...
}

Note: simpler configurations are blocked on #7752. We'll update this pinned discussion once that PR lands.

As with any big new API, there's only so much testing we can do on our own without the community. Please try out the project service API and post back here with how it goes for you! We'd love to know: does it indeed solve those configuration and/or performance problems for you? Are there any bugs? Was it a smash success? Any and all posts about how it goes are welcome! ♥️

[liuxingbaoyu]

FYI: I tested it in babel/babel and it worked fine, the overall lint time was reduced from 38 seconds to 36 seconds.

[JoshuaKGoldberg]

Linking for reference: babel/babel#16192 - thanks for posting!

[dcecile]

I tried it out on a personal project (.eslintrc.json, tsconfig.json) and the lint time increased from 31 seconds to 34 seconds. (Using @typescript-eslint/eslint-plugin 6.17.0.)

Is there any logging I can enable? For example, to see if the same service is used for all files?

[JoshuaKGoldberg]

Giving this the ❤️ reaction as I'm enthused you're trying it out... and the 😕 reaction because I'm sad it's slower on your end. Darn.

Though, I tried a direct clone -> npx eslint . on that project, and saw it get slightly faster when I switched to either "project": true or "EXPERIMENTAL_useProjectService": true. Was that 31 -> 34 seconds measurement done as an average / through hyperfine or an equivalent?

Measurements from hyperfine "npx eslint ." -i on Node v20.2.0 on my M1 Max:

Baseline	"project": true	"EXPERIMENTAL_useProjectService": true
6.597 s ± 0.119 s	6.224 s ± 0.041 s	6.227 s ± 0.080 s

Note that EXPERIMENTAL_useProjectService had no lint complaints whereas the other two had identical ones. I didn't bother building fully to get a true comparison from 0 lint failures. Is there a quick set of build commands I can run?

[JoshuaKGoldberg]

Oh, and I filed #8189 to track adding docs for logging info. That issue has some more info in it on the DEBUG flags you can provide.

[dcecile]

Thanks Josh! I've tried again with this hyperfine invocation:

hyperfine "./node_modules/.bin/eslint --ignore-path .gitignore --max-warnings 0 --ext ts,js,cjs ./dev-esbuild ./dev-node-loader ./dev-node-suppress ./dev-pnpm-test ./lib-collection ./lib-concurrency ./lib-error ./lib-hex-ts ./lib-payload ./lib-random ./lib-stream ./lib-style ./lib-test ./lib-time ./lib-widget ./svc-auth-guest-read ./svc-auth-guest-view ./svc-gateway-guest-run"

Setup:

• git checkout ec6d173f390e02762145a762b97acebe5eaa0b9e
• pnpm i
• ./node_modules/.bin/tsc --build

Baseline	"project": true	"EXPERIMENTAL_useProjectService": true
16.655 s ± 0.760 s	16.232 s ± 0.196 s	15.897 s ± 0.203 s

This is a more positive result. Switched my repo over to EXPERIMENTAL_useProjectService now ✔️

What I previously saw as "34 seconds" was definitely user CPU time (nice that hyperfine reports this too), not total time, sorry for the confusion!

I got lots of logs with --debug, but didn't see any obvious differences with parserOptions.debugLevel or any obvious TypeScript problems. Is there anything I could look for that would relate to the new project service?

[JoshuaKGoldberg]

Is there anything I could look for that would relate to the new project service?

🙌 Love that you're asking this! At this point your best bet is probably to search for useProgramFromProjectService in the typescript-eslint source and see where that gets called. Logs generally come from a const log = debug(. If there are points you'd want logged but is not yet, Local Development -> Local Linting can get you to a place where you can add those locally and try them out. If you find anything particularly useful please do file an issue suggesting we add them. We don't have complete coverage of those and so can always look into adding more.

[Zamiell]

I tried this out on my monorepo via this commit, but I got this error:

Error: Error while loading rule '@typescript-eslint/await-thenable': You have used a rule which requires parserServices to be generated. You must therefore provide a value for the "parserOptions.project" property for @typescript-eslint/parser.
Occurred while linting D:\Repositories\isaacscript\scripts\getMonorepoPackageNames.ts
    at getParserServices (D:\Repositories\isaacscript\node_modules\@typescript-eslint\utils\dist\eslint-utils\getParserServices.js:29:15)
    at create (D:\Repositories\isaacscript\node_modules\@typescript-eslint\eslint-plugin\dist\rules\await-thenable.js:47:55)
    at Object.create (D:\Repositories\isaacscript\node_modules\@typescript-eslint\utils\dist\eslint-utils\RuleCreator.js:38:20)
    at createRuleListeners (D:\Repositories\isaacscript\node_modules\eslint\lib\linter\linter.js:895:21)
    at D:\Repositories\isaacscript\node_modules\eslint\lib\linter\linter.js:1066:110
    at Array.forEach (<anonymous>)
    at runRules (D:\Repositories\isaacscript\node_modules\eslint\lib\linter\linter.js:1003:34)
    at Linter._verifyWithoutProcessors (D:\Repositories\isaacscript\node_modules\eslint\lib\linter\linter.js:1355:31)
    at Linter._verifyWithoutProcessors (D:\Repositories\isaacscript\node_modules\eslint-plugin-eslint-comments\lib\utils\patch.js:181:42)

For reference, the CI run is here: https://github.com/IsaacScript/isaacscript/actions/runs/7415844579/job/20179746969

[JoshuaKGoldberg]

Ah that's misconfigured: it's not projectService yet. For now it's EXPERIMENTAL_useProjectService. #6403 tracks adding a notice for typos like that.

[Zamiell]

Ok. Can you edit the OP to clarify the +- instructions? That's what I was going off of.

[Zamiell]

I tried this out via this commit, but I got this new error: https://github.com/IsaacScript/isaacscript/actions/runs/7431948579/job/20223198672

[JoshuaKGoldberg]

Aha! Thanks for mentioning. This is #8206: the project service doesn't (yet?) provide a tsconfig.json's compiler options for a neighboring file not included in that tsconfig.json.

I'm surprised I hadn't yet filed an issue for this. It's been on the back of my mind for a while... adding the issue to the v7 milestone too.

[mrazauskas]

Latest release (v7.8.0) gave me: "Too many files (>8) have matched the default project." See: https://github.com/tstyche/tstyche/actions/runs/8893774897/job/24420694811

This is GHA failure. I think it is worth taking a look at it, because the error is somehow strangely constructed (it grows and grows and grows ...)

Hard to read the message, but looks like all files in the repo are reported as belonging to the default project. Hm.. That sounds incorrect. There should be none. I was using EXPERIMENTAL_useProjectService in this repo for some time, it seemed that all files were included to one or another tsconfig.json.

[bradzacher]

@JoshuaKGoldberg it looks like the project service isnt using other tsconfigs?

I this repo the root tsconfig excludes the examples folder (which has its own tsconfig). The error in the log shows that all the examples are falling into the default project.

[JoshuaKGoldberg]

Yeah this is happening for a few users (@ehoogeveen-medweb, @me4502). Tracked in #9032. Sorry for the inconvenience all!

If you're seeing this, a temporary workaround can be to set parserOptions.EXPERIMENTAL_useProjectService.maximumDefaultProjectFileMatchCount_THIS_WILL_SLOW_DOWN_LINTING to some huge value like Infinity. I'd suggest adding a comment with a link to #9032 to explain why such a ridiculously named option is the right one to use for now. 😄