Change Request: Standardize documentation properties (e.g. allow configs to export description)

[bmish]

ESLint version

v8.55.0

What problem do you want to solve?

With the new flat config system, configs are only allowed to export known properties like rules, files, etc.

However, in third-party tooling I maintain like eslint-doc-generator and lintbase.com, I've been letting plugins set an unofficial description / meta.description / meta.docs.description property to annotate their configs (or processors) with a description that can be included in the auto-generated documentation for the plugin. This won't work anymore as flat config will throw an error like Error: Unexpected key "meta" found..

Furthermore, there are a variety of documentation-related properties used, allowed, or not allowed by ESLint core objects/concepts today, and this can be inconsistent and inflexible:

• Plugins: Can export meta.name and meta.version
• Rules: Can export anything, official properties are in meta or meta.docs, meta.docs.description is commonly used
• Rule options: Can export properties controlled by JSONSchema including description
• Configs: No meta/documentation properties currently allowed, but a name property can be included to help with debugging
• Processors: Can export meta.name and meta.version for debugging

What do you think is the correct solution?

In general, it would be useful to be able to accommodate at least the following rule documentation properties on any of the ESLint-controlled core objects/concepts: description, url, deprecated, replacedBy, plus space for arbitrary third-party/user-defined properties.

The challenge is how to place these properties in a consistent fashion across object types and in consideration for the already existing properties.

In particular, some of these rule properties are spread across meta and meta.docs. The dividing line between meta and meta.docs can be a bit blurry. I was thinking that meta would be for properties that are functionally used by ESLint, whereas meta.docs would be for non-critical/informational/custom properties that aren't necessarily used or needed by ESLint. By that division, description, url, deprecated, replacedBy would likely all fall under meta.docs.

Some ideas for improving the consistency and flexibility of properties on ESLint core objects/concepts:

1. Always allow meta.docs as an arbitrary object for any documentation / third-party properties.
2. Suggest using meta.docs for documentation properties like description, url, deprecated, replacedBy.
   • One exception to this: rules place deprecated and replacedBy directly on meta.
   • We can encourage using these common properties and suggest formats for them, as we do today with rules, as they are good for the health of the plugin ecosystem and often used by IDEs and third-party tooling, but ESLint itself wouldn't necessarily enforce anything about them.
   • The list of documentation properties I have mentioned so far is not necessarily comprehensive as it's just a starting point based on existing, commonly-used rule properties.
3. Whenever a name or version is needed to be specified, include it in the meta object. So we could update configs to accept meta.name (falling back to the current name property for backwards compatibility if needed).

To summarize: I took an initial stab at holistically considering documentation properties, but my top priority is really just to decide where documentation properties should go on each ESLint core object (in meta.docs in my proposal) without necessarily specifying a complete list of all potential documentation properties and their exact formats.

Related issue about the deprecation properties:

• Change Request: Support additional rule metadata for deprecations #18061

Participation

• I am willing to submit a pull request for this change.

Additional comments

No response

[aladdin-add]

I was trying to add a .meta to a plugin's config: https://github.com/eslint-community/eslint-plugin-n/blob/2e2bceb0d8404e705d973f5537239457aab45fa1/lib/configs/recommended-module.js#L7
but got an error now:

Error: Unexpected key "meta" found.
    at ObjectSchema.validate (/Users/weiran/repo/github/eslint-plugin-node/node_modules/@humanwhocodes/object-schema/src/object-schema.js:270:23)
    at /Users/weiran/repo/github/eslint-plugin-node/node_modules/@humanwhocodes/object-schema/src/object-schema.js:239:18
    at Array.reduce (<anonymous>)
    at ObjectSchema.merge (/Users/weiran/repo/github/eslint-plugin-node/node_modules/@humanwhocodes/object-schema/src/object-schema.js:237:24)
    at /Users/weiran/repo/github/eslint-plugin-node/node_modules/@humanwhocodes/config-array/api.js:966:42
    at Array.reduce (<anonymous>)
    at FlatConfigArray.getConfig (/Users/weiran/repo/github/eslint-plugin-node/node_modules/@humanwhocodes/config-array/api.js:965:39)
    at FlatConfigArray.isFileIgnored (/Users/weiran/repo/github/eslint-plugin-node/node_modules/@humanwhocodes/config-array/api.js:993:15)
    at /Users/weiran/repo/github/eslint-plugin-node/node_modules/eslint/lib/eslint/eslint-helpers.js:504:38
    at Array.forEach (<anonymous>)
husky - pre-commit hook exited with code 1 (error)

I suggest allowing the meta property (but not mandatory), where plugin authors can place some explanatory content as needed.

[bmish]

Allowing arbitrary meta could be sufficient as the solution (as opposed to just allowing meta.docs as I was originally suggesting). I'm open to this if others prefer it.

The downside I was worried about with that is that ESLint stores some official properties in meta, and if it adds more such properties in the future, there could be naming conflicts with user/third-party-defined properties that have since been stored there. So I was trying to leave meta for ESLint and meta.docs for user/third-party-defined properties.

[aladdin-add]

good point. meta.docs sounds good to me👍!

[nzakas]

Flat configs actually already support a name property, I just haven't had the time to wire that up to anything useful. The idea was to output that name in any validation error messages to make them easier to track down. It seems like that would serve the needs of your custom description property?

I'm not sure it makes sense to standardize meta properties across all types of objects as they all have different purposes. Rules, necessarily, need more information because they are end-user-facing. Plugins adding meta is specifically because we need a way to serialize them for caching and --print-config, which doesn't apply to configs. Nor do I think version can logically apply to configs.

[ljharb]

name and description are conceptually distinct things; this feels to me like import.meta - an explicitly bounded free-for-all area so that the ecosystem can safely innovate without conflicting with eslint.

[nzakas]

@ljharb are you suggesting that we just standardize a meta object on everything and let people do what they want?

If so, how do we reconcile the existing name and version properties we've defined in meta? And how do we avoid collisions if we need to add more properties in the future?

[ljharb]

Yes, and you can define “name and version” as something they aren’t allowed to do :-)

I assume eslint would then add top-level properties when needed.

[JoshuaKGoldberg]

I'm in favor of this. I think it'd be helpful for the ecosystem to collect the ad-hoc ways plugins document themselves into more standardized approaches. That way, tools like lintbased.com can improve discoverability of those plugins.

For reference: in typescript-eslint, each user-facing config intentionally has a one-liner description in https://typescript-eslint.io/linting/configs#recommended-configurations.

[nzakas]

Okay, it seems like there's general agreement that this is a good idea, so marking as accepted. Does anyone want to take the action item to flesh out a full RFC?

[bmish]

There seem to be some conflicting thoughts so far about whether user-defined properties should go in meta (where some official properties are already located) or meta.docs. I would find it helpful to hear some more input on this and ideally make a decision on it before the RFC.

[nzakas]

I'd say put together an RFC with your current thoughts and let people comment on that. It's a lot easier to have a full proposal to comment on then trying to peel off an individual preference here or there.