Bug: repeated lints in the same process run out of memory

[sandersn]

Before You File a Bug Report Please Confirm You Have Done The Following...

• I have tried restarting my IDE and the issue persists.
• I have updated to the latest version of the packages.
• I have searched for related issues and found none that matched my issue.
• I have read the FAQ and my problem is not listed.

Issue Description

Expected: Repro should finish, printing memory usage that stays roughly about the same.
Actual: Memory usage grows throughout the run until the VM runs out of memory.

This happens even with all .eslintrc.json rules turned off, as long as typescript-eslint is specified as eslint's parser.

Probably related to #1192

Reproduction Repository Link

https://github.com/sandersn/ts-eslint-oom

Repro Steps

1. clone the repo inside a folder like repros:

$ cd repros
$ git clone git@github.com:sandersn/ts-eslint-oom.git

2. clone the Definitely Typed repo two levels up from ts-eslint-oom:

$ cd ..
$ git clone git@github.com:DefinitelyTyped/DefinitelyTyped.git --depth 1
$ cd DefinitelyTyped
$ npm i --no-package-lock

At least, I'm pretty sure a shallow clone will work here.
Yes, there is no package-lock.

3. Back in ts-eslint-oom, run npm ci
4. Then node index.js

2a. Optionally, delete all the rules in .eslintrc.json, since that shows that the problem is from typescript-eslint and not the rules. You may also need to delete all the override .eslintrc.jsons in subfolders.

ts-eslint-oom loops over every folder in DefinitelyTyped/types and calls eslint on it. The config at DT root specifies typescript-eslint as the parser.

Versions

package	version
@typescript-eslint/eslint-plugin	5.52.0
@typescript-eslint/parser	5.52.0
@typescript-eslint/utils	5.52.0
ESLint	8.34.0
node	18 or 16

[bradzacher]

Okay - so I think I understand the problems.

First - the clearProgramCache function you're using is pretty useless. #6476 fixes that. You can work around this by also calling clearWatchCaches until the PR lands.

Second - you're using import { clearProgramCache } from '@typescript-eslint/typescript-estree', however the path you actually want is ../../DefinitelyTyped/node_modules/@typescript-eslint/typescript-estree/index.js.
ESLint resolves things relative to the file and config, not relative to the execution point. Your script was clearing the cache of the local module which was always empty anyways!

---

Fun little extra thing - you can add process.env.TSESTREE_SINGLE_RUN = 'true' to improve the performance of your script. This will put our parser into "single run" mode - which is what we'll (soon) be using as the default for ESLint CLI runs (but not for custom scripts!)

In "normal" mode we'll create a watch compiler host and builder program, which is super slow for various reasons. We do this so we can efficiently handle the IDE usecase, but it sucks for the CLI / "one and done" usecase.

In "single run" mode we'll directly create a ts.Program - circumventing the watch/builder overhead. Obv this wouldn't work in the IDE, but it works perfectly in the CLI usecase.
But sadly we can't detect custom scripts - only the ESLint binary - hence you can force it using the environment variable