Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue in TYPESCRIPT-ES when using mega-lint-runner #1572

Closed
MRDGH2821 opened this issue Jun 28, 2022 · 68 comments · Fixed by #2455
Closed

Issue in TYPESCRIPT-ES when using mega-lint-runner #1572

MRDGH2821 opened this issue Jun 28, 2022 · 68 comments · Fixed by #2455
Assignees
Labels
bug Something isn't working nostale This issue or pull request is not stale, keep it open

Comments

@MRDGH2821
Copy link

MRDGH2821 commented Jun 28, 2022

Describe the bug

When using mega-lint-runner, TYPESCRIPT_ES always fails to lint, because it is not able to access /tsconfig.json

mega-lint-runner on local system

Where as in Github Actions, there is no error for the same.

Mega Linter Github Action

Expected behavior
TYPESCRIPT_ES should have thrown error or success flag on both CI & local, but it is not happenning.

Screenshots
Here is the log file instead

ERROR-TYPESCRIPT_ES.log

Additional context
My repository where it is enabled - https://github.com/MRDGH2821/Perpetual-Mechanical-Array-Bot/tree/rewrite-detritus-bug-hunt
Branch - rewrite-detritus-bug-hunt
The configurations are at root of the directory.

.eslintrc.json -

{
  "extends": ["eslint-config-airbnb-base", "eslint-config-airbnb-typescript/base"],
  "parserOptions": {
    "project": ["tsconfig.json", "tsconfig.eslint.json"]
  },
  "rules": {
    "no-console": "off"
  }
}

tsconfig.json -

{
  "compilerOptions": {
    "allowJs": false,
    "baseUrl": "./src",
    "declaration": true,
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "inlineSources": true,
    "lib": ["ES2019", "DOM", "ESNext"],
    "module": "CommonJS",
    "moduleResolution": "node",
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "outDir": "out",
    "sourceMap": true,
    "strict": true,
    "strictNullChecks": true,
    "target": "ESNext",
    "resolveJsonModule": true,
    "watch": false
  },
  "ts-node": {
    "require": ["tsconfig-paths/register"],
    "swc": true
  },
  "tsc-alias": {
    "resolveFullPaths": true
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules", ".vscode", "./old-src", "./out/*"]
}
@MRDGH2821 MRDGH2821 added the bug Something isn't working label Jun 28, 2022
@Kurt-von-Laven
Copy link
Collaborator

This is a stab in the dark, but what happens if you remove the PRE_COMMANDS, POST_COMMANDS, and/or DEFAULT_WORKSPACE from your MegaLinter config?

I'm not sure whether or not this will be helpful to you, but perhaps it will be helpful to others. I haven't run into this issue in our TypeScript projects, because we use Prettier. See ScribeMD/docker-cache for an example of a working MegaLinter + TypeScript config.

@MRDGH2821
Copy link
Author

MRDGH2821 commented Jun 28, 2022

PRE_COMMANDS was to install the eslint plugins & Airbnb config
POST_COMMANDS was because I couldn't find how to make specific linters fix the code after a specific linter is done fixing (i.e. I want prettier to fix first, then eslint to fix next. Mega linter didn't give expected results, thus put npm script in post commmands)

Anyways, I did the following -

Case 1

ESlint config

{
  "extends": ["eslint-config-airbnb-base", "eslint-config-airbnb-typescript/base"],
  "parserOptions": {
    "project": ["./tsconfig.json", "./tsconfig.eslint.json"]
  },
  "rules": {
    "no-console": "off"
  }
}

Mega linter config

# Configuration file for MegaLinter
# See all available variables at https://megalinter.github.io/configuration/ and in linters documentation

APPLY_FIXES: all # all, none, or list of linter keys
# ENABLE: # If you use ENABLE variable, all other languages/formats/tooling-formats will be disabled by default
# ENABLE_LINTERS: # If you use ENABLE_LINTERS variable, all other linters will be disabled by default
DISABLE:
  - GIT
# - COPYPASTE # Uncomment to disable checks of excessive copy-pastes
# - SPELL # Uncomment to disable checks of spelling mistakes
DISABLE_LINTERS:
  - JAVASCRIPT_STANDARD
  - TYPESCRIPT_STANDARD
SHOW_ELAPSED_TIME: true
FILEIO_REPORTER: false
# DISABLE_ERRORS: true # Uncomment if you want MegaLinter to detect errors but not block CI to pass
JAVASCRIPT_DEFAULT_STYLE: prettier
TYPESCRIPT_DEFAULT_STYLE: prettier
LINTER_RULES_PATH: .
PRINT_ALPACA: false

PRE_COMMANDS:
  - command: 'npm install'
    cwd: 'workspace'
POST_COMMANDS:
  - command: 'npm run pretty'
    cwd: 'workspace'
IGNORE_GITIGNORED_FILES: true

# Excludes cspell file itself
SPELL_MISSPELL_FILTER_REGEX_EXCLUDE: cspell.

Result

TYPESCRIPT_ES fails on local, success on Github Action

Log file shows same error text as attached in the first comment of the issue.

Case 2:

ESlint config

Same as in Case 1

Mega linter

# Configuration file for MegaLinter
# See all available variables at https://megalinter.github.io/configuration/ and in linters documentation

APPLY_FIXES: all # all, none, or list of linter keys
# ENABLE: # If you use ENABLE variable, all other languages/formats/tooling-formats will be disabled by default
# ENABLE_LINTERS: # If you use ENABLE_LINTERS variable, all other linters will be disabled by default
DISABLE:
  - GIT
# - COPYPASTE # Uncomment to disable checks of excessive copy-pastes
# - SPELL # Uncomment to disable checks of spelling mistakes
DISABLE_LINTERS:
  - JAVASCRIPT_STANDARD
  - TYPESCRIPT_STANDARD
SHOW_ELAPSED_TIME: true
FILEIO_REPORTER: false
# DISABLE_ERRORS: true # Uncomment if you want MegaLinter to detect errors but not block CI to pass
JAVASCRIPT_DEFAULT_STYLE: prettier
TYPESCRIPT_DEFAULT_STYLE: prettier
LINTER_RULES_PATH: .
PRINT_ALPACA: false

#PRE_COMMANDS:
#  - command: 'npm install'
#    cwd: 'workspace'
#POST_COMMANDS:
#  - command: 'npm run pretty'
#    cwd: 'workspace'
IGNORE_GITIGNORED_FILES: true

# Excludes cspell file itself
SPELL_MISSPELL_FILTER_REGEX_EXCLUDE: cspell.

Result

Same as Case 1 + There are git diffs in some files, because ESlint couldn't "fix" them

Case 3

ESlint config

{
  "extends": ["eslint-config-airbnb-base", "eslint-config-airbnb-typescript/base"],
  "parserOptions": {
    "project": ["tsconfig.json", "tsconfig.eslint.json"]
  },
  "rules": {
    "no-console": "off"
  }
}

Mega linter config

Same as Case 2

Result

Same as Case 2

Verdict

Even after removing PRE_COMMANDS, POST_COMMANDS & DEFAULT_DIRECTORY ; TYPESCRIPT_ES fails because in eslint config it is supposed to extend tsconfig.json which actually exists in the root of project directory (same place as eslint config) but cannot find /tsconfig.json

I wonder if the issue is about that / (slash) in /tsconfig.json which logs say

@MRDGH2821
Copy link
Author

MRDGH2821 commented Jun 28, 2022

Forgot to mention this -

$ docker --version
Docker version 20.10.16, build aa7e414

$  docker-compose --version
docker-compose version 1.29.2, build 5becea4c

I'm using Windows 11 pro, Docker is using WSL 2.

The issue affects me locally only, so I'm fine if no definite solution is found as long as it works properly as Github Action

@Kurt-von-Laven Kurt-von-Laven self-assigned this Jun 30, 2022
@Kurt-von-Laven
Copy link
Collaborator

Kurt-von-Laven commented Jun 30, 2022

Now I'm doubly glad you filed this bug, because I just realized that we haven't been running ESLint at all, which explains why we didn't experience this issue. (We don't use the GitHub Action.) When I create a .eslintrc.yaml, ESLint no longer gets skipped, and I am able to reproduce this issue locally. Users who aren't using any rules that require parserServices to be generated can avoid this issue entirely by deleting parserOptions.project from their ESLint config. For those who are, prefixing each path in parserOptions.project with /tmp/lint appears to be a hacky, short-term workaround locally. I'm guessing you'll probably also have to add the following to your MegaLinter config:

TYPESCRIPT_ES_PRE_COMMANDS:
  - command: npm install <missing dependencies, such as your base tsconfig>

Setting DEFAULT_WORKSPACE in the MegaLinter config has no effect on the bug.

On a side note, if I understood correctly, you are saying that ESLint and Prettier are configured in such a manner that they potentially contradict each other, which sounds problematic. Please see Prettier's guidance on playing nicely with ESLint. You can also configure many editors to run many linters on save.

I do think this issue needs to be fixed, and I'm making it my current active work item because the present work around is so unacceptable long-term.

@Kurt-von-Laven
Copy link
Collaborator

@MRDGH2821, we still need to fix the underlying bug, but hopefully this improved workaround gets you unblocked for the time being. I recommend adding the following to your MegaLinter config:

TYPESCRIPT_ES_CLI_LINT_MODE: project

This causes ESLint to be run on the entire project rather than be passed each individual file, so if your project has a lot of files, this will unfortunately slow things down. You probably won't even need to use TYPESCRIPT_ES_PRE_COMMANDS.

When I run ESLint outside of MegaLinter in the same project, I don't experience the same issue, so I am pretty confident this is our bug. I debugged this issue by passing --env LOG_LEVEL=DEBUG to mega-linter-runner and noting the following output in the logs:

[eslint] command: ['eslint', '--no-ignore', '--fix', '/tmp/lint/file1.ts', '/tmp/lint/file2.ts', ...]
[eslint] CWD: /

With the lint mode overridden from the default of list_of_files to project, I instead see:

[eslint] command: ['eslint', '--no-ignore', '--fix']
[eslint] CWD: /tmp/lint
[eslint] result: 0 

This, in conjunction with the error you initially reported, implies that ESLint is not run with the correct working directory in list_of_files mode. @nvuillam may have some insights on how we can fix this issue for all our users after the v6 release. Also, bravo for including the current working directory in the debug logs as that greatly reduced my debugging time. What a forward-thinking decision :).

@MRDGH2821
Copy link
Author

MRDGH2821 commented Jul 1, 2022

On a side note, if I understood correctly, you are saying that ESLint and Prettier are configured in such a manner that they potentially contradict each other, which sounds problematic.

The code formatted by prettier triggers eslint errors.
It manages to touch all the code (which ESlint doesn't somehow) but even after multiple modifications to the config, I couldn't get any config which didn't trigger eslint errors.
So I run eslint fix after prettier. And the output from eslint also looks good. Hence the post commands (prettier fixes then eslint fixes)

Adding this line into the config worked for me in local & github action both.

TYPESCRIPT_ES_CLI_LINT_MODE: project

This causes ESLint to be run on the entire project rather than be passed each individual file, so if your project has a lot of files, this will unfortunately slow things down.

If there's a way to utilise multiple CPU cores then I would like to know. Else slow speed not much of an issue. On top of that I won't be linting the files when the code goes live in production 😂

Thanks for the temporary fix!
Looking forward to v6

@Kurt-von-Laven
Copy link
Collaborator

Sorry I overlooked your lingering question here. Unfortunately, beyond that you upvote eslint/eslint#3565, I don't have a great suggestion for you. @nvuillam may correct me if I am mistaken, but my understanding is that MegaLinter runs the linter once in project mode by definition and can only run a linter multiple times in files or list_of_files modes. Linters that themselves support parallelization can be run in parallel in project mode by MegaLinter provided any necessary additional arguments are passed to the underlying linter. Ha ha, I am glad the performance is not a major issue for you at the moment.

@nvuillam
Copy link
Member

That brings us back to #1544 :)
I'll try to make a doc page today :)

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

@github-actions github-actions bot added the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Aug 19, 2022
@github-actions github-actions bot closed this as completed Sep 3, 2022
@Kurt-von-Laven Kurt-von-Laven reopened this Oct 1, 2022
@Kurt-von-Laven Kurt-von-Laven added nostale This issue or pull request is not stale, keep it open and removed O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity labels Oct 1, 2022
@Kurt-von-Laven
Copy link
Collaborator

I think this issue : ESLint :: #440 : Swiftlint.

@Gainutdinov
Copy link

Gainutdinov commented Mar 3, 2023

Hello, face the same issue with TYPESCRIPT_ES above, the workaround doesn't help, well issue is gone and it seems that .ts_config is there any improvements?
I use gitlab and my config is pretty basic but TYPESCRIPT_ES doesn't catch any errors. Maybe I am doing something wrong.

linter:
  stage: pre-build
  image:
    name: oxsecurity/megalinter:v6
  script: [ "true" ]
  variables:
    DEFAULT_WORKSPACE: "/builds/project/project/src/frontend"
---
ENABLE: 
  - "TYPESCRIPT"
ENABLE_LINTERS:
  - "TYPESCRIPT_ES"
  - "TYPESCRIPT_PRETTIER"
JAVASCRIPT_DEFAULT_STYLE: "prettier"
TYPESCRIPT_DEFAULT_STYLE: "prettier"
TYPESCRIPT_ES_RULES_PATH: "/builds/project/project/src/frontend"
TYPESCRIPT_ES_CLI_LINT_MODE: project
TYPESCRIPT_ES_PRE_COMMANDS: 
  - command: npm install eslint-plugin-rxjs@latest eslint-plugin-rxjs-angular@latest
    cwd: "workspace"
SHOW_ELAPSED_TIME: true
...

@Kurt-von-Laven
Copy link
Collaborator

Do you have a .eslintrc.yaml? Some linters are configured only to run when certain files they rely on are present.

@Gainutdinov
Copy link

Yeah sure, initially we used .eslintrc as config file for eslint but this type of file wasn't recognized by linter even when I provided corresponding environment variable and TYPESCRIPT_ES was always inactive because of this I renamed file to .eslintrc.json and the linter successfully passed initialization and became active but failed with error mentioned above: /.tsconfig.json not found, so I added PRE_COMMANDS:.. which copies src/frontend/* files into /. Last step helped me to overcome issue with not founded .tsconfig.json but still it seems that it doesn't use the provided config file cause I see more errors in comparison with local linter.

@Kurt-von-Laven
Copy link
Collaborator

Thank you for confirming; in retrospect that should have been obvious, but I am doing a bit too much context switching right now. This doesn't solve your problem, but TYPESCRIPT_ES_CONFIG_FILE defaults to .eslintrc.json and can be overridden if you desire. I tend to set *_CONFIG_FILE to LINTER_DEFAULT personally as I find this behavior most intuitive. Are you at liberty to share the contents of your .eslintrc.json? Ours that is working at MegaLinter v6.19.0 is quite minimal, so maybe you will spot something in there that you can copy.

@Gainutdinov
Copy link

Thank you for the quick response, just want to mention that I tried to use TYPESCRIPT_ES_CONFIG_FILE but it didn't help and on initialization step I still saw that TYPESCRIPT_ES linter skipped. Regarding the config, I think I can send the config, I will send it a bit later when will be near the computer.

@Gainutdinov
Copy link

.eslintrc.json file:

{
  "root": true,
  "parser": "@typescript-eslint/parser",
  "ignorePatterns": ["**/jest.config.ts", "**/jest.preset.ts", "**/custom.d.ts"],
  "plugins": [
    "@typescript-eslint",
    "rxjs",
    "rxjs-angular"
  ],
  "parserOptions": {
    "project": "tsconfig.json"
  },
  "extends": [
  ],
  "rules": {
    // basic rules
    "prefer-const": "error",
    "rxjs-angular/prefer-takeuntil": [
      "error",
      {
        "alias": ["untilDestroyed"],
        "checkComplete": true,
        "checkDecorators": ["Component"]
      }
   ],
    "@typescript-eslint/no-useless-constructor": "error",
    "max-lines-per-function": ["error", { "max": 35 }],
    "no-param-reassign": "error",
    "indent": ["error", 2, {"SwitchCase": 1}],
    "semi": ["error", "always", { "omitLastInOneLineBlock": true}],
    "no-template-curly-in-string": "error",
    "prefer-template": "error",
    "@typescript-eslint/explicit-member-accessibility": [
      "error",
      {
        "accessibility": "no-public"
      }
    ],
    "@typescript-eslint/no-extraneous-class": ["error", {
      "allowWithDecorator": true
    }],
    "@typescript-eslint/no-this-alias": [
      "error",
      {
        "allowDestructuring": false
      }
    ],
    // typescript strict null check required
    "@typescript-eslint/strict-boolean-expressions": "error",
    "@typescript-eslint/explicit-function-return-type": ["error"],
    "@typescript-eslint/consistent-type-assertions": [
      "error",
      {
        "assertionStyle": "never"
      }
    ],
    "@typescript-eslint/no-non-null-assertion": "warn",
    "@typescript-eslint/naming-convention": [
      "error",
      {
        "selector": "default",
        "format": ["camelCase"],
        "leadingUnderscore": "allow",
        "trailingUnderscore": "forbid"
      },
      {
        "selector": "variable",
        "format": ["camelCase", "UPPER_CASE"],
        "leadingUnderscore": "allow",
        "trailingUnderscore": "forbid"
      },
      {
        "selector": "variable",
        "modifiers": ["destructured"],
        "format": null
      },
      {
        "selector": "typeProperty",
        "format": ["camelCase", "PascalCase", "UPPER_CASE"]
      },
      {
        "selector": "objectLiteralProperty",
        "format": null
      },
      {
        "selector": ["typeLike", "enumMember"],
        "format": ["PascalCase"]
      }
    ],

    // compact - better reading
    "max-len": [
      "error",
      {
        "code": 120,
        "ignoreUrls": true,
        "ignoreStrings": true
      }
    ],
    "max-lines": ["error", 500],
    "@typescript-eslint/type-annotation-spacing": "error",
    "@typescript-eslint/typedef": [
      "error",
      {
        "variableDeclaration": true,
        "objectDestructuring": true,
        "variableDeclarationIgnoreFunction": true
      }
    ],

    // clear trash
    "no-trailing-spaces": "error",
    "@typescript-eslint/no-unused-expressions": "error",
    "@typescript-eslint/no-extra-non-null-assertion": "error",
    "@typescript-eslint/no-empty-interface": [
      "error",
      {
        "allowSingleExtends": false
      }
    ],
    "@typescript-eslint/no-explicit-any": "error",
    "@typescript-eslint/no-confusing-non-null-assertion": "warn",
    "no-duplicate-case": "error",
    "no-duplicate-imports": "error",
    "no-console": "error",
    "no-empty": "error",
    "no-sequences": "error",
    "no-sparse-arrays": "error"
  }
}

@Kurt-von-Laven
Copy link
Collaborator

Thanks; that's helpful, but unfortunately I have more questions than answers for you. The original error was Parsing error: Cannot read file '/tsconfig.json', but can you confirm that yours is Parsing error: Cannot read file '/.tsconfig.json'? I ask because your parserOptions.parser is tsconfig.json without the leading .. Do you have a megalinter-reports/linters_logs/*-TYPESCRIPT_ES.log after running MegaLinter? Are you able to share your package.json?

On a side note, I suspect you should have env.browser and env.jest set to true.

@Gainutdinov
Copy link

Gainutdinov commented Mar 5, 2023

package.json

{
  "name": "project-frontend",
  "license": "MIT",
  "scripts": {
    "dep-graph": "nx dep-graph",
    "start:prod": "nx run project:serve:development --proxyConfig=tools/proxies/prod.json --live-reload",
    "start:dev1": "nx run project:serve:development --proxyConfig=tools/proxies/dev1.json --live-reload",
    "start:demo2": "nx run project:serve:development --proxyConfig=tools/proxies/demo2.json --live-reload",
    "postinstall": "node ./decorate-angular-cli.js && ngcc --properties es2015 browser module main",
    "build": "nx build --deploy-url=/tasks/ --base-href=/tasks/",
    "test": "nx test",
    "ng": "nx",
    "lint": "eslint . --ext .ts",
    "prettier:check-diff": "prettier --config .prettierrc.json -c $(git diff HEAD --relative --name-only --diff-filter d | grep -E './**/*.(html|ts)' | xargs) --ignore-unknown",
    "prettier:fix-diff": "prettier --config .prettierrc.json -w $(git diff HEAD --relative --name-only --diff-filter d | grep -E './**/*.(html|ts)' | xargs) --ignore-unknown",
    "prettier:check-all": "prettier --config .prettierrc.json -c ./**/*.ts ./**/*.html --ignore-unknown",
    "prettier:fix-all": "prettier --config .prettierrc.json -w ./**/*.ts ./**/*.html --ignore-unknown",
    "start:production": "nx run project:serve:production"
  },
  "private": true,
  "dependencies": {
    "@angular/animations": "13.3.8",
    "@angular/cdk": "13.3.7",
    "@angular/common": "13.3.8",
    "@angular/compiler": "13.3.8",
    "@angular/core": "13.3.8",
    "@angular/forms": "13.3.8",
    "@angular/platform-browser": "13.3.8",
    "@angular/platform-browser-dynamic": "13.3.8",
    "@angular/router": "13.3.8",
    "@ant-design/icons": "^4.7.0",
    "@ant-design/icons-angular": "^13.1.0",
    "@auth0/angular-jwt": "~5.0.2",
    "@frontend-libs/cdk": "2.0.4",
    "@frontend-libs/functions": "~2.0.1",
    "@frontend-libs/kit": "2.0.2",
    "@frontend-libs/pipes": "2.0.2",
    "@frontend-libs/redesign-kit": "0.0.53",
    "@ngneat/hot-toast": "4.1.0",
    "@ngneat/transloco": "2.23.2",
    "@ngneat/until-destroy": "~9.2.0",
    "@ngxs/store": "^3.7.3-dev.master-b00eeb3",
    "@nrwl/angular": "^14.5.6",
    "@tiptap/extension-blockquote": "^2.0.0-beta.209",
    "@tiptap/extension-bold": "^2.0.0-beta.209",
    "@tiptap/extension-bullet-list": "^2.0.0-beta.209",
    "@tiptap/extension-character-count": "^2.0.0-beta.26",
    "@tiptap/extension-code": "^2.0.0-beta.199",
    "@tiptap/extension-code-block-lowlight": "^2.0.0-beta.199",
    "@tiptap/extension-color": "^2.0.0-beta.9",
    "@tiptap/extension-document": "^2.0.0-beta.209",
    "@tiptap/extension-dropcursor": "^2.0.0-beta.209",
    "@tiptap/extension-gapcursor": "^2.0.0-beta.209",
    "@tiptap/extension-hard-break": "^2.0.0-beta.209",
    "@tiptap/extension-heading": "^2.0.0-beta.209",
    "@tiptap/extension-history": "^2.0.0-beta.209",
    "@tiptap/extension-horizontal-rule": "^2.0.0-beta.209",
    "@tiptap/extension-image": "^2.0.0-beta.27",
    "@tiptap/extension-italic": "^2.0.0-beta.209",
    "@tiptap/extension-link": "^2.0.0-beta.38",
    "@tiptap/extension-list-item": "^2.0.0-beta.209",
    "@tiptap/extension-ordered-list": "^2.0.0-beta.209",
    "@tiptap/extension-paragraph": "^2.0.0-beta.209",
    "@tiptap/extension-placeholder": "^2.0.0-beta.209",
    "@tiptap/extension-strike": "^2.0.0-beta.209",
    "@tiptap/extension-subscript": "^2.0.0-beta.10",
    "@tiptap/extension-superscript": "^2.0.0-beta.10",
    "@tiptap/extension-table": "^2.0.0-beta.49",
    "@tiptap/extension-table-cell": "^2.0.0-beta.20",
    "@tiptap/extension-table-header": "^2.0.0-beta.22",
    "@tiptap/extension-table-row": "^2.0.0-beta.19",
    "@tiptap/extension-task-item": "^2.0.0-beta.199",
    "@tiptap/extension-task-list": "^2.0.0-beta.199",
    "@tiptap/extension-text": "^2.0.0-beta.209",
    "@tiptap/extension-text-align": "^2.0.0-beta.29",
    "@tiptap/extension-text-style": "^2.0.0-beta.23",
    "@tiptap/extension-typography": "^2.0.0-beta.20",
    "@tiptap/extension-underline": "^2.0.0-beta.23",
    "@typescript-eslint/parser": "^5.18.0",
    "lowlight": "^2.7.0",
    "ng-zorro-antd": "~13.3.2",
    "ngx-scrollbar": "^8.0.0",
    "ngx-tippy-wrapper": "^5.0.2",
    "ngx-tiptap": "^5.0.0",
    "npx": "^10.2.2",
    "rxjs": "^7.5.5",
    "tslib": "~2.3.1",
    "turndown": "^7.1.1",
    "uuid": "^8.3.2",
    "zone.js": "0.11.5"
  },
  "devDependencies": {
    "@angular-builders/custom-webpack": "~13.1.0",
    "@angular-devkit/build-angular": "13.3.5",
    "@angular-eslint/builder": "~13.2.1",
    "@angular-eslint/eslint-plugin": "~13.2.1",
    "@angular-eslint/eslint-plugin-template": "~13.2.1",
    "@angular-eslint/schematics": "~13.2.1",
    "@angular-eslint/template-parser": "~13.2.1",
    "@angular/cli": "13.3.5",
    "@angular/compiler-cli": "13.3.8",
    "@angular/language-service": "13.3.8",
    "@ngxs/devtools-plugin": "^3.7.4",
    "@nrwl/cli": "^14.5.6",
    "@nrwl/cypress": "^14.5.6",
    "@nrwl/eslint-plugin-nx": "^14.5.6",
    "@nrwl/jest": "^14.5.6",
    "@nrwl/linter": "^14.5.6",
    "@nrwl/workspace": "^14.5.6",
    "@types/jest": "27.4.1",
    "@types/node": "16.11.7",
    "@types/turndown": "^5.0.1",
    "@types/uuid": "^8.3.4",
    "@typescript-eslint/eslint-plugin": "~5.18.0",
    "cypress": "^9.1.0",
    "eslint": "~8.12.0",
    "eslint-config-prettier": "8.1.0",
    "eslint-plugin-cypress": "^2.10.3",
    "eslint-plugin-import": "^2.26.0",
    "eslint-plugin-jest": "^26.4.6",
    "eslint-plugin-jsdoc": "^39.3.2",
    "eslint-plugin-prefer-arrow": "^1.2.3",
    "eslint-plugin-rxjs": "^5.0.2",
    "eslint-plugin-rxjs-angular": "^2.0.0",
    "eslint-plugin-unicorn": "^42.0.0",
    "file-loader": "^6.2.0",
    "jest": "27.5.1",
    "jest-preset-angular": "11.1.1",
    "nx": "^14.1.4",
    "prettier": "^2.7.1",
    "raw-loader": "~4.0.2",
    "ts-jest": "27.1.4",
    "ts-node": "~10.4.0",
    "typescript": "^4.6.4",
    "typescript-strict-plugin": "^2.0.0"
  }
}

@Gainutdinov
Copy link

----------------------------------------------------------------------------------------------------
------------------------------------ MegaLinter, by OX Security ------------------------------------
----------------------------------------------------------------------------------------------------
 - Image Creation Date: 2023-02-05T11:57:24Z
 - Image Revision: f8d535e
 - Image Version: 1.28.0
----------------------------------------------------------------------------------------------------
The MegaLinter documentation can be found at:
 - https://megalinter.io/1.28.0
----------------------------------------------------------------------------------------------------
�[0Ksection_start:`1678000190220630554`:megalinter-init[collapsed=true]
�[0KMegaLinter initialization (expand for details)

MARKDOWN_REMARK_LINT has been temporary disabled in MegaLinter, please use a previous MegaLinter version or wait for the next one !
Skipped linters: ACTION_ACTIONLINT, ANSIBLE_ANSIBLE_LINT, ARM_ARM_TTK, BASH_EXEC, BASH_SHELLCHECK, BASH_SHFMT, BICEP_BICEP_LINTER, CLOJURE_CLJ_KONDO, CLOUDFORMATION_CFN_LINT, COFFEE_COFFEELINT, COPYPASTE_JSCPD, CPP_CPPLINT, CSHARP_CSHARPIER, CSHARP_DOTNET_FORMAT, CSS_SCSS_LINT, CSS_STYLELINT, C_CPPLINT, DART_DARTANALYZER, DOCKERFILE_HADOLINT, EDITORCONFIG_EDITORCONFIG_CHECKER, ENV_DOTENV_LINTER, GHERKIN_GHERKIN_LINT, GO_GOLANGCI_LINT, GO_REVIVE, GRAPHQL_GRAPHQL_SCHEMA_LINTER, GROOVY_NPM_GROOVY_LINT, HTML_DJLINT, HTML_HTMLHINT, JAVASCRIPT_ES, JAVASCRIPT_PRETTIER, JAVASCRIPT_STANDARD, JAVA_CHECKSTYLE, JAVA_PMD, JSON_ESLINT_PLUGIN_JSONC, JSON_JSONLINT, JSON_NPM_PACKAGE_JSON_LINT, JSON_PRETTIER, JSON_V8R, JSX_ESLINT, KOTLIN_KTLINT, KUBERNETES_KUBECONFORM, KUBERNETES_KUBEVAL, LATEX_CHKTEX, LUA_LUACHECK, MAKEFILE_CHECKMAKE, MARKDOWN_MARKDOWNLINT, MARKDOWN_MARKDOWN_LINK_CHECK, MARKDOWN_MARKDOWN_TABLE_FORMATTER, MARKDOWN_REMARK_LINT, OPENAPI_SPECTRAL, PERL_PERLCRITIC, PHP_PHPCS, PHP_PHPLINT, PHP_PHPSTAN, PHP_PSALM, POWERSHELL_POWERSHELL, POWERSHELL_POWERSHELL_FORMATTER, PROTOBUF_PROTOLINT, PUPPET_PUPPET_LINT, PYTHON_BANDIT, PYTHON_BLACK, PYTHON_FLAKE8, PYTHON_ISORT, PYTHON_MYPY, PYTHON_PYLINT, PYTHON_PYRIGHT, RAKU_RAKU, REPOSITORY_CHECKOV, REPOSITORY_DEVSKIM, REPOSITORY_DUSTILOCK, REPOSITORY_GITLEAKS, REPOSITORY_GIT_DIFF, REPOSITORY_GOODCHECK, REPOSITORY_SECRETLINT, REPOSITORY_SEMGREP, REPOSITORY_SYFT, REPOSITORY_TRIVY, RST_RSTCHECK, RST_RSTFMT, RST_RST_LINT, RUBY_RUBOCOP, RUST_CLIPPY, R_LINTR, SALESFORCE_SFDX_SCANNER_APEX, SALESFORCE_SFDX_SCANNER_AURA, SALESFORCE_SFDX_SCANNER_LWC, SCALA_SCALAFIX, SNAKEMAKE_LINT, SNAKEMAKE_SNAKEFMT, SPELL_CSPELL, SPELL_MISSPELL, SPELL_PROSELINT, SQL_SQLFLUFF, SQL_SQL_LINT, SQL_TSQLLINT, SWIFT_SWIFTLINT, TEKTON_TEKTON_LINT, TERRAFORM_CHECKOV, TERRAFORM_KICS, TERRAFORM_TERRAFORM_FMT, TERRAFORM_TERRAGRUNT, TERRAFORM_TERRASCAN, TERRAFORM_TFLINT, TSX_ESLINT, TYPESCRIPT_STANDARD, VBDOTNET_DOTNET_FORMAT, XML_XMLLINT, YAML_PRETTIER, YAML_V8R, YAML_YAMLLINT
To receive reports as email, please set variable EMAIL_REPORTER_EMAIL
�[0Ksection_end:`1678000191111898605`:megalinter-init
�[0K
�[0Ksection_start:`1678000191111954200`:megalinter-file-listing[collapsed=true]
�[0KMegaLinter now collects the files to analyse (expand for details)
Listing all files in directory [/builds/project/project/src/frontend], then filter with:
- File extensions: .ts
Unable to list git ignored files ()
Kept [724] files on [1318] found files

+----MATCHING LINTERS---+----------+----------------+------------+
| Descriptor | Linter   | Criteria | Matching files | Format/Fix |
+------------+----------+----------+----------------+------------+
| TYPESCRIPT | eslint   | .ts      | 724            | no         |
| TYPESCRIPT | prettier | .ts      | 724            | no         |
+------------+----------+----------+----------------+------------+
�[0Ksection_end:`1678000191164593432`:megalinter-file-listing
�[0K
�[0Ksection_start:`1678000199307910847`:processed-TYPESCRIPT_PRETTIER[collapsed=true]
�[0K�[33m✅ Linted [TYPESCRIPT] files with [prettier]: Found 1 non blocking error(s) - (7.85s)�[0m (expand for details)
- Using [prettier v2.8.3] https://megalinter.io/1.28.0/descriptors/typescript_prettier
- MegaLinter key: [TYPESCRIPT_PRETTIER]
- Rules config: [/builds/project/project/src/frontend/.prettierrc.json]
- Number of files analyzed: [724]
--Error detail:
Checking formatting...
[warn] builds/project/project/src/frontend/libs/shared/src/lib/types/type-color.type.ts
[warn] Code style issues found in the above file. Forgot to run Prettier?

�[0Ksection_end:`1678000199311338660`:processed-TYPESCRIPT_PRETTIER
�[0K
�[0Ksection_start:`1678000239139554091`:processed-TYPESCRIPT_ES[collapsed=true]
�[0K�[31m❌ Linted [TYPESCRIPT] files with [eslint]: Found 724 error(s) - (47.67s)�[0m (expand for details)
- Using [eslint v8.33.0] https://megalinter.io/1.28.0/descriptors/typescript_eslint
- MegaLinter key: [TYPESCRIPT_ES]
- Rules config: [/builds/project/project/src/frontend/.eslintrc.json]
- Ignore file: [/builds/project/project/src/frontend/.eslintignore]
- Number of files analyzed: [724]
[Pre][TYPESCRIPT_ES] run: [cd /node-deps && npm install eslint-plugin-rxjs@latest eslint-plugin-rxjs-angular@latest] in cwd [/builds/project/project/src/frontend]
[Pre][TYPESCRIPT_ES] 
added 38 packages, removed 2 packages, changed 2 packages, and audited 2359 packages in 14s

234 packages are looking for funding
  run `npm fund` for details

15 vulnerabilities (2 moderate, 12 high, 1 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

--Error detail:

/builds/project/project/src/frontend/apps/project/jest.config.ts
  0:0  warning  File ignored because of a matching ignore pattern. Use "--no-ignore" to override

/builds/project/project/src/frontend/apps/project/src/app/app.component.ts
  0:0  error  Parsing error: Cannot read file '/tsconfig.json'

...

/builds/project/project/src/frontend/libs/workspace/src/lib/services/workspace-explorer.service.ts
  0:0  error  Parsing error: Cannot read file '/tsconfig.json'

/builds/project/project/src/frontend/libs/workspace/src/lib/services/workspace-toasts.service.ts
  0:0  error  Parsing error: Cannot read file '/tsconfig.json'

/builds/project/project/src/frontend/libs/workspace/src/lib/workspace.module.ts
  0:0  error  Parsing error: Cannot read file '/tsconfig.json'

✖ 724 problems (697 errors, 27 warnings)

�[0Ksection_end:`1678000239143152204`:processed-TYPESCRIPT_ES
�[0K

+----SUMMARY----+----------+---------------+-------+-------+--------+--------------+
| Descriptor    | Linter   | Mode          | Files | Fixed | Errors | Elapsed time |
+---------------+----------+---------------+-------+-------+--------+--------------+
| ❌ TYPESCRIPT | eslint   | list_of_files |   724 |       |    724 |       47.67s |
| ◬ TYPESCRIPT  | prettier | list_of_files |   724 |       |      1 |        7.85s |
+---------------+----------+---------------+-------+-------+--------+--------------+

�[34mYou could have same capabilities but better runtime performances if you use a MegaLinter flavor:�[0m
- [javascript] oxsecurity/megalinter/flavors/javascript@1.28.0 (56 linters) https://megalinter.io/1.28.0/flavors/javascript/
- [cupcake] oxsecurity/megalinter/flavors/cupcake@1.28.0 (80 linters) https://megalinter.io/1.28.0/flavors/cupcake/

[Gitlab Comment Reporter] No merge request has been found, so no comment has been posted
Unable to find git repository to list updated files
[Updated Sources Reporter] No source file has been formatted or fixed
�[31m❌ Error(s) have been found during linting�[0m
To disable linters or customize their checks, you can use a .mega-linter.yml file at the root of your repository
More info at https://megalinter.io/1.28.0/configuration/

@Gainutdinov
Copy link

Gainutdinov commented Mar 5, 2023

On a side note, I suspect you should have env.browser and env.jest set to true.

I will tell FrontendDevs regarding this thank you :)

@Kurt-von-Laven
Copy link
Collaborator

Kurt-von-Laven commented Mar 5, 2023

I believe @typescript-eslint/parser should be a devDependency, but I can't imagine this would break ESLint. What is in your tsconfig.json? Since you mentioned .tsconfig.json earlier, can you confirm that you only have a tsconfig.json and not also a .tsconfig.json? You have a bunch of @angular-eslint and *eslint-plugin* packages installed, but I don't see where they are used. You might want to confirm that you can reproduce the issue with a more minimal package.json.

@Gainutdinov
Copy link

Confirm, tsconfig.json is the only one in the DEFAULT_WORKSPACE: "/builds/project/project/src/frontend", besides I can find another tsconfig.json files in subdirectories but I think they doesn't disturb the linter execution.

.tsconfig.json doesn't occure in any repo directory.

here is the example of my tsconfig.json

{
  "extends": "./tsconfig.base.json",
  "compilerOptions": {
    "alwaysStrict": true,
    "noImplicitAny": true,
    "strictNullChecks": true,
    "strictFunctionTypes": true,
    "strictPropertyInitialization": true,
    "noImplicitThis": true,
    "strictBindCallApply": true,

    "noPropertyAccessFromIndexSignature": true,
    "noUncheckedIndexedAccess": true,
    "noImplicitReturns": true,
    "noFallthroughCasesInSwitch": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,

    "forceConsistentCasingInFileNames": true,
    "noImplicitOverride": true,

    "allowUnreachableCode": false,
    "allowUnusedLabels": false,
    "suppressExcessPropertyErrors": true,
    "suppressImplicitAnyIndexErrors": true
  },
  "angularCompilerOptions": {
    "strictInjectionParameters": true,
    "strictInputAccessModifiers": true,
    "strictTemplates": true
  }
}

@Kurt-von-Laven
Copy link
Collaborator

Kurt-von-Laven commented Mar 8, 2023

That looks reasonable to me. I would try again with LOG_LEVEL: DEBUG this time at MegaLinter v6.20.1 now that it's been released so we can get more visibility into what may be going wrong.

Kurt-von-Laven added a commit that referenced this issue Mar 12, 2023
In project mode, ESLint succeeds without doing anything when called
without any files, so pass it the current directory.
@Kurt-von-Laven
Copy link
Collaborator

My hack trying to use cli_lint_extra_args_after resulted in some test failures. I discovered we already have EslintLinter.py, and realized list_of_files mode does work in CI, so I modified EslintLinter.py to append . to ESLint's command line in project mode only.

Kurt-von-Laven added a commit that referenced this issue Apr 4, 2023
In project mode, ESLint succeeds without doing anything when called
without any files, so pass it the current directory.
nvuillam pushed a commit that referenced this issue Apr 4, 2023
In project mode, ESLint succeeds without doing anything when called
without any files, so pass it the current directory.
@Kurt-von-Laven
Copy link
Collaborator

Kurt-von-Laven commented Apr 27, 2023

I am moving this discussion from #2578, where this is the third of three issues, to this thread. If you've already read that thread, this will be redundant, but I will summarize it for those who haven't.

The root cause of the issue I reported on the other thread is related to the way we install our Node.js dependencies, namely to /node-deps. ESLint is leaning on TypeScript to find the @tsconfig/node18-strictest-esm package that gets installed to /node-deps via a pre-command, but TypeScript neither honors NODE_PATH nor has its own configuration option to search a directory other than node_modules when resolving packages in extends of tsconfig.json. I was able to confirm this theory in a minimal reproduction with a small Dockerfile, and the following workaround works for us:

POST_COMMANDS:
  - command: rm -r node_modules
    cwd: workspace
TYPESCRIPT_ES_PRE_COMMANDS:
  - command: npm install @tsconfig/node18-strictest-esm@1.0.1
  - command: cp --recursive /node-deps/node_modules/ node_modules
    cwd: workspace

One thing that would potentially make this all better would be supporting arrays in the *_CLI_EXECUTABLE config option, which I suspect might be a one-line change. That would allow users to configure MegaLinter to run ESLint via Yarn, allowing TypeScript to find ESLint installed via Yarn PnP. It's a little silly in the sense that it defeats the purpose of having ESLint and friends in the Docker image at all. On the other hand, it's generally a good idea to add those directly to your project as dev dependencies anyways to keep your IDE in sync with MegaLinter. Reopening this issue now that it's actionable again. Open to suggestions on better ways of dealing with this.

@Kurt-von-Laven
Copy link
Collaborator

Copied from #2578:

@Kurt-von-Laven I'm trying to fix that in #2601 ... the idea is the following:

  • check if PRE_COMMANDS contain "npm" or "yarn"
  • if yes:
    • copy /node_deps/node_modules into |workspace+"node_modules"
    • Add workspace+"node_modules" in PATH
    • run the PRE_COMMANDS

Do you think it could work ? ^^

@Kurt-von-Laven i tried to reproduce your issue and it seems to work... please can you confirm with the newest beta version ? :)

https://github.com/oxsecurity/megalinter/actions/runs/4846602288/jobs/8636240640

@nvuillam, #2601 does not fix the issue and breaks the workaround.

I tried adapting the workaround I described above by specifying cwd: /node-deps for the npm install command, but I encountered the following error:

multiprocessing.pool.RemoteTraceback:                                                                                                                                                        """                                                                                                                                                                                          Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                           ^^^^^^^^^^^^
  File "/megalinter/MegaLinter.py", line 39, in run_linters
    linter.run()
  File "/megalinter/Linter.py", line 711, in run
    pre_post_factory.run_linter_pre_commands(self.master, self)
  File "/megalinter/pre_post_factory.py", line 24, in run_linter_pre_commands
    return run_commands(
              ^^^^^^^^^^^
  File "/megalinter/pre_post_factory.py", line 52, in run_commands
    pre_command_result = run_command(command_info, log_key, mega_linter, linter)
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/megalinter/pre_post_factory.py", line 89, in run_command
    raise Exception(
Exception: [Pre][TYPESCRIPT_ES]: User command failed, stop running MegaLinter
npm ERR! Tracker "idealTree" already exists
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2023-05-01T06_35_02_059Z-debug-0.log                                                                                                                                                                                                                                                                                                                        

Leaving the workaround as-is results in the same crash on npm install. That doesn't seem like a sensible approach though since the workaround intends to install the package to /node-deps, and #2601 removed the logic that changes directories to /node-deps for npm install commands. I still think that makes the developer experience less friendly (c.f., #2578 (comment)). Removing the workaround gets us back to lots of: 0:0 error Parsing error: File '@tsconfig/node18-strictest-esm/tsconfig.json' not found. That error comes from the tsconfig we extend not being present in /tmp/lint/node_modules. Copying /node-deps/node_modules to /tmp/lint/node_modules before running pre-commands doesn't address this since @tsconfig/node18-strictest-esm correctly isn't in the MegaLinter Docker image. I don't think we want to be automatically copying /node-deps/node_modules to /tmp/lint/node_modules when a pre-command contains "npm" or "yarn". Our users are not likely to expect this surprising behavior, we don't have a use case to motivate it, and it seems likely to create confusion or even breakage in Yarn projects that don't use node_modules. From #2578 (comment):

In addition, if they didn't think to clean node_modules with a post-command, Yarn PnP (or pnpm) projects would end up with two copies of all of their the dependencies, one of which (the one in node_modules) is probably owned by root and isn't used by Yarn/pnpm. This can create confusion, particularly since the versions most likely wouldn't stay in sync as they have separate lock files managed by different tools.

@nvuillam
Copy link
Member

nvuillam commented May 1, 2023

@Kurt-von-Laven thanks for the feedback

The goal is to be able to perform npm install, yarn install, npm install xxx-dep or yarn add xxx-dep in PRE-COMMANDS without any workaround, don't worry i don't plan to release a new ML version before that :)

There was a bug, and i also added dir_exists_ok =True in another PR, it might work better

@Kurt-von-Laven
Copy link
Collaborator

Pre/post-commands containing npm install [<pacakge>] and yarn add [<pacakge>] already worked before to my knowledge, just not without the workaround of copying the package to /tmp/lint/node_modules when the missing package is referenced by the extends field of a tsconfig.json. I haven't ever tried running yarn add from a pre-command, but also I can't think of a plausible reason to. I still don't know a way to make that edge case smoother without making the situation worse overall beyond just permitting arrays in *_CLI_EXECUTABLE. Forcing the copy to overwrite /tmp/lint/node_modules seems likely to create even more disruption for users. What happens if they are using npm (or Yarn v1 or Yarn v2+ configured with nodeLinker: node-modules) but have a different version of, say, typescript?

@nvuillam
Copy link
Member

nvuillam commented May 1, 2023

Everything changes since all commands are called from workspace and not from docker image root, there are more advantages than inconveniences to call linters the same way they would be locally.

I'm currently working on your idea to allow to override cli_executable

@Kurt-von-Laven
Copy link
Collaborator

Changing the default working directory of pre-commands from / to /tmp/lint seems good. I still think overwriting /tmp/lint/node_modules is quite disruptive though.

@nvuillam
Copy link
Member

nvuillam commented May 1, 2023

@Kurt-von-Laven I'm trying again something else... :)

Would it be acceptable to say that if there are additional npm dependencies:

  • they must be in package.json
  • npm install must be called in PRE_COMMANDS
  • linter_CLI_EXECUTABLE must be overriden (this is now possible in beta, and it was not just 2 lines :D) with ["npm","run","somelinter"] or something similar ?

@Kurt-von-Laven
Copy link
Collaborator

That sounds promising!

@nvuillam
Copy link
Member

nvuillam commented May 1, 2023

@Kurt-von-Laven I think i have found the solution, published in the newest beta ! :)

--> https://github.com/nvuillam/CSpell-Bug/tree/nico (additional cspell dict is well added, without copying node_modules or messing with PATH)

Job with GHA: https://github.com/nvuillam/CSpell-Bug/actions/runs/4854021395/jobs/8650882718?pr=1

Job with mega-linter-runner: https://github.com/nvuillam/CSpell-Bug/actions/runs/4854021390/jobs/8650882471?pr=1

Please can you try by using cwd: workspace in your PRE_COMMANDS with npm install ?

@Kurt-von-Laven
Copy link
Collaborator

Running npm install [<package>] in the workspace breaks Yarn projects that don't use node_modules:

On the one hand, running npm install @tsconfig/node18-strictest-esm@1.0.1 in the workspace would install the module directly in the place TypeScript will ultimately look for it. On the other, it will also install everything in your package.json into the node_modules directory in your workspace that you didn't think you were using and create package-lock.json. I haven't checked pnpm, but neither npm nor Yarn offer a means by which to install only a single package to node_modules in their CLIs. Installing all packages could be a considerable slowdown for projects with large dependency trees, particularly since they may not think to cache npm dependencies in CI if they aren't using npm directly. In addition, if they didn't think to clean node_modules with a post-command, Yarn PnP (or pnpm) projects would end up with two copies of all of their the dependencies, one of which (the one in node_modules) is probably owned by root and isn't used by Yarn/pnpm. This can create confusion, particularly since the versions most likely wouldn't stay in sync as they have separate lock files managed by different tools. At the end of the day, I think this issue demonstrates the subtle pitfalls of running Node.js modules via a different package manager than the one a project is using. ~ #2578 (comment)

@Kurt-von-Laven
Copy link
Collaborator

I hit the same npm ERR! Tracker "idealTree" already exists error from #1572 (comment) when I put the following in .mega-linter.yaml:

TYPESCRIPT_ES_PRE_COMMANDS:
  - command: npm install @tsconfig/node18-strictest-esm@1.0.1
    continue_if_failed: false
    cwd: /node-deps

When I use cwd: workspace instead, I get lots of: 0:0 error Parsing error: File '@tsconfig/node18-strictest-esm/tsconfig.json' not found. This surprised me, and I also found it odd that node_modules/@tsconfig existed in the workspace but was empty. Even if we were to troubleshoot that though, running npm install with cwd: workspace will always create many problems in a Yarn Plug'n'Play project. Beyond the aforementioned issues, running yarn install after npm install causes Yarn PnP to migrate itself back to the incompatible v1 lock file format and configure itself to use node_modules.

Copying node_modules into the workspace means that it will be owned by root if you aren't using rootless Docker, breaking all subsequent npm commands. We shouldn't be doing that even behind a flag. The copy takes place before the user's pre-commands run, so it doesn't help TypeScript find an extended tsconfig. I'm not aware of another use case that copying node_modules addresses. Copying node_modules after a pre-command containing npm install runs would be even worse since at that point the linters are already running, resulting in unpredictable behavior.

@nvuillam
Copy link
Member

nvuillam commented May 2, 2023

@Kurt-von-Laven did you pull the latest beta ?
There is no more copy of node_modules from MegaLinter core
cwd can be only root or workspace (root by default), if you want to go in another folder you need to cd directly in the command

@Kurt-von-Laven
Copy link
Collaborator

Kurt-von-Laven commented May 2, 2023

Yeah, sorry, I think I was under the impression when I wrote that that the node_modules copying was still taking place in certain circumstances, but that was probably based on my outdated code review. I am not experiencing the node_modules copying now on the same beta image in any case, so that's exciting! Ohhh... thanks for the tip about cwd. ESLint ran this time with my original workaround where I copied /node-deps/node_modules/@tsconfig to /tmp/lint/node_modules. I hit hundreds of @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-member-access, and @typescript-eslint/restrict-template-expressions errors though, which makes me think typescript-eslint lost the ability to load dependencies properly since Thursday April 27th when it was last working. This is kind of a stab in the dark, but what are we doing with NODE_PATH and PATH at this point?

@nvuillam
Copy link
Member

nvuillam commented May 2, 2023

@Kurt-von-Laven with newest beta, NODE_PATH and PATH remain untouched, NODE_PATH is /node-deps and PATH still has /node-deps/node_modules

@nvuillam
Copy link
Member

nvuillam commented May 2, 2023

@Kurt-von-Laven I'm trying to add a test case, do you have suggestions about how it should be built ?

I'm trying that, probably it needs an additional package.json, maybe a 'yarn' to install dependencies as pre-command ?

image

@Kurt-von-Laven
Copy link
Collaborator

That is a great idea! In an empty directory, one can run yarn set version stable to use the latest stable version of Yarn instead of Yarn v1 and create a minimal package.json. Yarn v1 is already end-of-life, so I don't think we need to worry about a separate test case for it. yarn add --dev @tsconfig/node18-strictest-esm eslint typescript@4.9.5 @typescript-eslint/parser @typescript-eslint/eslinter-plugin will add dev dependencies to the package.json and install it to .yarn/cache. One could copy .pnp.cjs, .yarn, .yarnrc.yml, package.json, and yarn.lock to a test fixture directory within MegaLinter for use in a test. The npm install pre-command is only needed if one isn't running ESLint via Yarn. Running ESLint via Yarn necessitates adding the required ESLint-related dependencies via Yarn. See .eslintrc.yaml and tsconfig.json from https://github.com/ScribeMD/docker-cache/ for help configuring things properly if needed.

@Kurt-von-Laven
Copy link
Collaborator

Kurt-von-Laven commented May 2, 2023

One issue I expect this test will expose is that TYPESCRIPT_ES_CLI_EXECUTABLE: [yarn, run, eslint] is only sufficient to run Yarn v1, so hopefully the test will catch that and fail. jwkvam/bowtie#222 has some discussion of yarn not working properly when run via subprocess.run rather than via the command line. Personally, I encountered Yarn v1 being used instead of Yarn v2+ since the former is installed globally, but the latter is only installed locally to the project and hence isn't on the PATH. GPT-3.5 suggested TYPESCRIPT_ES_CLI_EXECUTABLE: [node, .yarn/releases/yarn-*.cjs, run, eslint] instead, which seems right to me and works when I test it outside of MegaLinter directly on the command line.

@Kurt-von-Laven
Copy link
Collaborator

TYPESCRIPT_ES_CLI_EXECUTABLE: [node, .yarn/releases/yarn-3.6.3.cjs, run, eslint] is working well for us running ESLint in list_of_files mode. This resolves the original Parsing error: Cannot read file '/tsconfig.json' issue for Yarn users. If anyone is still encountering this error, we can reopen this issue or open a new issue as appropriate. If anyone has suggestions regarding a better solution, I am all ears.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working nostale This issue or pull request is not stale, keep it open
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants