[6.0.0-alpha.1] RuleTester fails tests if parser property is a package name

[mysticatea]

From import-js/eslint-plugin-import#1362
From https://travis-ci.org/benmosher/eslint-plugin-import/jobs/533124402

Tell us about your environment

• ESLint Version: 6.0.0-alpha.1
• Node Version: 10.15.3
• npm Version: 6.4.1

What parser (default, Babel-ESLint, etc.) are you using?

n/a

Please show your full configuration:

n/a

What did you do? Please include the actual source code causing the issue, as well as the command that you used to run ESLint.

Use RuleTester with parser option with babel-eslint. (e.g., eslint-plugin-import:/tests/src/rules/default.js#L31)

What did you expect to happen?

No errors.

What actually happened? Please include the actual, raw output from ESLint.

The test failed with "Parsers provided as strings to RuleTester must be absolute paths."

Are you willing to submit a pull request to fix this bug?

Yes.

---

I think that RuleTester should allow package names (maybe it loads it relative to CWD).

Also, the Migration Guide must describe this change.

[not-an-aardvark]

The current behavior is intentional because it's not clear where packages should be loaded from with RuleTester. I suppose the CWD works, but it could be confusing because projects using RuleTester might be dependencies of the user's project, so they can't guarantee that their dependencies will be loadable from the CWD. It seems like it would be safer to require an absolute path so that users use require.resolve to avoid the ambiguity.

[platinumazure]

I think I agree with @not-an-aardvark. We should just document this as a breaking change and require users to specify a full path.

[mysticatea]

Thank you for the explanation.

But, my position is different. Because to use file paths for parser config is a pretty rare case (probably only we are). For 99.99% of users, parser is packages such as babel-eslint, vue-eslint-parser, @typescript-eslint/parser. and etc. Not a file path. Therefore, disallowing a package name would be a surprising behavior.

I think that good design is "people can do the most popular use case easily, and can do rare cases with detailed way." I believe we should allow a package name and show a understandable error message for rare cases (E.g., '${parser}' was not found on '${cwd}'. Make sure the package to be installed or use an absolute path with 'require.resolve()'.). The current behavior seems to enforce annoying to users by afraid of pretty rare cases.

[mysticatea]

TSC Summary:

This is an awareness that came from the failing CI build of eslint-plugin-import. We no longer support a package name in the parser option in RuleTester. This has been introduced in eslint/rfcs#7 intentionally because where it's unclear to load the package from. But for 99.99% of users, parser is packages such as babel-eslint, vue-eslint-parser, @typescript-eslint/parser. and etc. Therefore, disallowing a package name would be a surprising behavior.

Opinions:

• @not-an-aardvark and @platinumazure ... [6.0.0-alpha.1] RuleTester fails tests if parser property is a package name #11728 (comment)
• @mysticatea ... [6.0.0-alpha.1] RuleTester fails tests if parser property is a package name #11728 (comment)

TSC Question:

What is the direction we want to go?

1. The parser requires an absolute path. Just we add the description of that into the migration guide.
2. The parser allows a package name as similar to ESLint 5 and loads it from CWD. We add the change of the place that ESLint loads the package from, into the migration guide. (On ESLint 5, RuleTester had loaded packages from the location ESLint was installed.)
3. Others.

[mysticatea]

TSC Resolution: we won't change the current behavior, the parser option requires an absolute path. We will add it to the migration guide.