New: Add name to RuleTester

[G-Rath]

Prerequisites checklist

• I have read the contributing guidelines.

What is the purpose of this pull request? (put an "X" next to an item)

[ ] Documentation update
[ ] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofixing to a rule
[ ] Add a CLI option
[x] Add something to the core
[ ] Other, please explain:

What changes did you make? (Give an overview)

Implemented support for naming test cases, which can be useful when you're debugging some tests that have very similar code (especially when you're in the final cleanup stages and trying to update all those column & endColumn numbers 😅), or when your test code is very looooooooong so you want to give it a name to make it easier on your terminal if it fails.

If a name isn't provided or is an empty string, we fallback to the current behavior of using code.

Is there anything you'd like reviewers to focus on?

Closes #15090

[linux-foundation-easycla]

The committers are authorized under a signed CLA.

• ✅ Gareth Jones (54322ab)

[aladdin-add]

LGTM, thanks!

[ljharb]

What happens in older eslints’ RuleTesters when test cases have a name provided?

Most eslint plugins cover multiple eslint versions, so new features aren’t useful in a reasonable time frame unless they’re backwards compatible.

[nzakas]

LGTM. Thanks!

[nzakas]

@ljharb I think they will be ignored. I’m not sure it matters either way, as this only affects the rule developer who has control over the ESLint version used.

[ljharb]

@nzakas great! the rule developer has "control" in the sense that they could semver-major and drop all older eslint versions, but in practice the rule developer does not have control of this, because breaking changes cause churn and pain for their users.

[mdjermanovic]

Marked as accepted since the issue is accepted.

[mdjermanovic]

What happens in older eslints’ RuleTesters when test cases have a name provided?

Top-level properties that are not RuleTesterParameters are passed through as eslint config options, so the test case will fail due to additionalProperties: false, since there's no top-level property name in the eslintrc schema.

 Error: ESLint configuration in rule-tester is invalid:
        - Unexpected top-level property "name".

[ljharb]

@mdjermanovic thats what i was afraid of, that means that the eslint ecosystem can’t use this feature in RuleTester until eslint < 8 is no longer supported - given that i have plugins still supporting eslint 2, that will be a long time.

Any thought to removing the “throw on unknown properties” behavior, since that’s what breaks forward compatibility?

[mdjermanovic]

Any thought to removing the “throw on unknown properties” behavior, since that’s what breaks forward compatibility?

I'd prefer not in this case, as that would silently ignore typos in test cases, while it's relatively easy to extend RuleTester and remove unsupported properties.

const { RuleTester } = require("eslint");

const supportsName = false; // ... check ESLint version 

module.exports = class MyRuleTester extends RuleTester {
    run(ruleName, rule, test) {
        if (!supportsName) {
            [...test.valid, ...test.invalid].forEach(testCase => {
                if (typeof testCase === "object") {
                      delete testCase.name;
                }
            });
        }

        super.run(ruleName, rule, test);
    }
}

[nzakas]

Agree. I don’t holding back progress for the sake of forward compatibility is a good approach. You can always just not use “name” at all in your tests and there will be no difference.

[mdjermanovic]

LGTM, thanks!

[mdjermanovic]

Thanks for contributing!

[iliubinskii]

Thx for the feature