From bc44748e7f643cc209630450376df342649eab83 Mon Sep 17 00:00:00 2001 From: Toru Nagashima Date: Tue, 24 Sep 2019 07:24:47 +0900 Subject: [PATCH] New: Config Tester (#27) --- designs/2019-config-tester/README.md | 146 +++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 designs/2019-config-tester/README.md diff --git a/designs/2019-config-tester/README.md b/designs/2019-config-tester/README.md new file mode 100644 index 00000000..0a635e1a --- /dev/null +++ b/designs/2019-config-tester/README.md @@ -0,0 +1,146 @@ +- Start Date: 2019-06-14 +- RFC PR: https://github.com/eslint/rfcs/pull/27 +- Authors: Toru Nagashima <https://github.com/mysticatea> + +# Config Tester + +## Summary + +Providing `ConfigTester` API that tests shareable configs and plugin's preset configs. + +## Motivation + +Currently, we don't provide stuff to test shareable configs. This means the community doesn't have an easy way to check if their config is good or not. + +- Is the config structure valid? +- Does every rule exist? +- Does every rule have valid options? +- Aren't there any deprecated rules? +- Are there plugin settings of configured rules? +- Will this config work after publish? + - Does `package.json` have parsers, plugins, and extended configs? + +## Detailed Design + +This RFC adds `ConfigTester` API to check configs. + +```js +const { ConfigTester } = require("eslint") + +// Instantiate the tester. +const tester = new ConfigTester() +const options = { + ignoreDeprecatedRules: false, + ignoreDisabledUnknownRules: false, + ignoreMissingDependencies: false, + ignoreRulesMissingFromConfig: false, +} + +// Verify a shareable config (a path to the target file). +tester.runOnConfigFile("index.js", options) +tester.runOnConfigFile("es5.js", options) +tester.runOnConfigFile("es2015.js", options) + +// Or verify plugin's preset configs (a config name in the plugin). +tester.runOnPluginConfig("base", options) +tester.runOnPluginConfig("recommended", options) +tester.runOnPluginConfig("opinionated", options) +``` + +### § `ConfigTester(projectRoot)` constructor + +Instantiate a new `ConfigTester` instance. + +#### Parameters + +The constructor has an optional parameter. + +Name | Description +:----|:----------- +`projectRoot` | Default is `process.cwd()`. The path to the project root. The root should contain `package.json`. + +#### Behavior + +The tester reads `` `${projectRoot}/package.json` `` to use in `run()` method. + +
+🔗PoC: lib/config-tester/config-tester.js#L60-L66 +
+ +### § `tester.runOnConfigFile(filePath, options)` method / `tester.runOnPluginConfig(configName, options)` method + +Validates a config data. + +#### Parameters + +Name | Description +:----|:----------- +`filePath` | Required. This is a path to a file (relative from `projectRoot`). +`configName` | Required. This is a config name of the plugin. If it cannot load the entry file (`main` field in `${projectRoot}/package.json` or `${projectRoot}/index.js`), it throws `MODULE_NOT_FOUND_ERROR`. +`options.ignoreDeprecatedRules` | Default is `false`. If `true` then the tester ignores deprecated rules. +`options.ignoreDisabledUnknownRules` | Default is `false`. If `true` then the tester ignores unknown rules if the rule was configured as `0` (`"off"`). +`options.ignoreRulesMissingFromConfig` | Default is `false`. If `true` then the tester ignores missing rules. The missing rules mean the rules that ESLint or a plugin defined but not configured. +`options.ignoreMissingDependencies` | Default is `false`. If `true` then the tester ignores wrong dependency definition (`dependencies`/`peerDependencies`). + +#### Behavior + +Similarly to `RuleTester`, `ConfigTester` defines tests by `describe` and `it` global variables. Then it does: + +1. Validate the config object has the valid scheme with `validateConfigSchema()`. +
+ 🔗PoC: lib/config-tester/config-tester.js#L244-L248 +
+1. Validate the config content with `validateConfigArray()`. +
+ 🔗PoC: lib/config-tester/config-tester.js#L250-L265 +
+1. Report non-existence rules. + - Because `validateConfigArray(configArray)` ignores non-existence rules. + - Configured plugin's rules are in `configArray.pluginRules`. + - If `ignoreDisabledUnknownRules` option was `true` and non-existence rule's severity was `"off"`, the tester ignores it. +
+ 🔗PoC: lib/config-tester/config-tester.js#L267-L299 +
+1. Report deprecated rules. + - If `ignoreDeprecatedRules` option was `true`, the tester skips this step. + - If the rule severity was `"off"`, the tester ignores it. + - Check `meta.deprecated` in both core rules and `configArray.pluginRules`. +
+ 🔗PoC: lib/config-tester/config-tester.js#L301-L338 +
+1. Check whether the config configures all rules. + - If `ignoreRulesMissingFromConfig` option was `true`, the tester skips this step. + - This step lets people know about new rules. +
+ 🔗PoC: lib/config-tester/config-tester.js#L340-L363 +
+1. Check whether `${options.projectRoot}/package.json` contains the configured parser, plugins, and shareable configs. + - If `ignoreMissingDependencies` option was `true`, the tester skips this step. + - If `parser` or `extends` were a file path except `node_modules/**`, the file should be published; check `.npmignore` and `package.json`'s `lib` field. + - If `parser` or `extends` were a package or a file path to `node_modules/**`, the package should be in `dependencies` or `peerDependencies`. + - `plugins` should be in `peerDependencies` or `name`. +
+ 🔗PoC: lib/config-tester/config-tester.js#L365-L420 +
+ +## Documentation + +- [Node.js API](https://eslint.org/docs/developer-guide/nodejs-api) page should describe the new `ConfigTester` API. +- [Creating a Shareable Config](https://eslint.org/docs/developer-guide/shareable-configs#creating-a-shareable-config) section should note the tester. +- [Configs in Plugins](https://eslint.org/docs/developer-guide/working-with-plugins#configs-in-plugins) section should note the tester. + +## Drawbacks + +- If people can write the config with no mistakes, this tester may not be needed. + +## Backwards Compatibility Analysis + +- This is not a breaking change. It just adds a new API. + +## Alternatives + +- https://www.npmjs.com/package/eslint-find-rules - we can check missing rules and deprecated rules with this package. + +## Related Discussions + +- https://github.com/eslint/eslint/issues/10289