Enabling presets for streamlined configuration

[charliermarsh]

Background

Ruff is relatively un-opinionated when it comes to configuration -- Ruff's default configuration is fairly minimal, and the expectation today is that user's will explicitly opt-in to the rule sets that they want to enable for their projects.

In response, we often hear that users want more streamlined and standardized configuration (#3356, #809, Twitter).

The goal of this Discussion is to debate the various options we could pursue to make Ruff easier to configure.

presets

To open the Discussion: one concrete option is to add a presets concept to Ruff, in which a preset would represent an opinionated set of defaults that the user could opt into. Semantically, enabling a preset would be equivalent to the current extends behavior (which lets Ruff extend another local configuration file), and also equivalent to changing Ruff's default settings.

For now, presets would ship with Ruff.

For example, we could allow users to express a configuration like:

[tool.ruff]
preset = "ruff:recommended"

...which might, in theory, be equivalent to something like (exact rules TBD):

[tool.ruff]
select = ["E", "F", "B", "UP"]

Users could modify the preset by adding additional options to their configuration. For example:

[tool.ruff]
preset = "ruff:recommended"
extend-select = ["SIM"]
line-length = 120

...would in theory be equivalent to:

[tool.ruff]
select = ["E", "F", "B", "UP"]
extend-select = ["SIM"]
line-length = 120

Alongside ruff:recommended, we would ship something like ruff:strict, and so on.

In the future, we could allow users to publish and share presets, just as in ESLint.

Questions

Some unanswered questions in this design, and in the proposal more broadly:

1. How do preset and extends interact? Are users allowed to both enable a preset and extend another configuration?
2. Do we want, or need, to support enabling "multiple" presets? ESLint allows this, but how would that work?
3. How should we go about determining the presets to ship with Ruff, and what to include in them? Is there data we can draw on?
4. Does this even solve the problem?

On that last question: I suspect that adding presets would help with users migrating to Ruff from a no-linter configuration, or from Pylint, and would help with the "There are too many rules, how do I configure this thing?" problem.

But users that are coming from the Flake8 ecosystem tend to have specific plugins enabled that they want to replicate, and those users tend to look for a 1:1 migration, so moving to a preset would typically mean turning off rules, which (in my experience) users tend to dislike. At the same time, perhaps users coming from the Flake8 ecosystem don't have this problem anyway, since they're typically mirroring their existing configuration?

[charliermarsh]

A different philosophy would be to say that we want Ruff to feel more like Clippy, with most rules enabled by default (apart from the "pedantic" and experimental rules) and coarser categorizations -- so Ruff becomes easier to configure by way of having fewer knobs and more opinions.

I'll admit that I don't know whether this is possible in Python, where there's so much more heterogeneity in projects and project needs: docstrings, type annotations, language versions, etc. All of these things lead to different needs that tend towards different linter configurations.

[MichaReiser]

Here's a link to Clippy's categories for reference.

I'll admit that I don't know whether this is possible in Python, where there's so much more heterogeneity in projects and project needs: docstrings, type annotations, language versions,

Would it be possible to detect the appropriate defaults for many or most of these and ask the user for input or assume good defaults for the few where it is not? Related to that. Can we develop a reasonable ruff preset if the ecosystem is so diverse? Wouldn't we need different presets depending on the docstyle the user uses?

I would find categories similar to Clippy useful even if Ruff adopts presets because being explicit about the categories of issues a lint catches is useful regardless.

[MithicSpirit]

1. same as two, see ↓
2. I think that this would require reworking the way that configuring ruff works as a whole to be sensible. Currently it seems like it would be confusing on what should merge or override and even more complicated to be able to selectively do one or the other. At the level of control I'd personally like it becomes more like a programming language (reminds me of Haskell's module system with import ModuleName hiding (functionName)) than just a configuration, so I think that it might be too much. This would also probably require the config to be its own file and format, which sounds like a bad idea even if it is opt-in.
3. Not sure if this is possible, but it would be interesting if ruff could check other python packages for presets (requiring them to be installed in some regular way or having a common name pattern so that it's faster, perhaps). Then, the most popular of these packages could be added to ruff itself as necessary to improve ease of use. Regardless, at first I think that just the recommended, strict (interpreting this to mean strictly following PEP8), and flake8 compatibility presets (as well as the current minimal default preset) should be included. (see EDIT)
4. It seems like the main issue is decision paralysis, which I would be probably solved by this if properly documented (e.g., present this as the main configuration method people should be using and recommend others only for tweaks or powerusers). There is also the issue of it not being entirely a drop-in tool, and, if you even want to fix this (maybe requiring the user to do at least a bit of research is preferable to make sure they pick what they want), it could be done by changing the default preset to the recommended one once it has been refined a bit. That would be a majorly breaking change though, so maybe it should wait for something like the v0.1 release.

EDIT (extension to q3): just checked out the linked issues again and it seems like eslint does the thing where packages can be released on npm, which is basically what I desired. I'm not too fond of the idea of just direct URLs though, as the idea of ruff making arbitrary network requests doesn't sit that well with me, but it might just be me being paranoid. Something like an environment variable or global config for ruff to prompt the user before making network requests would be sufficient for me, though (these wouldn't be present in CI servers, of course, so it would still work fine there).

[MichaReiser]

I want to take one step back from technical discussion to understand better the users' needs. I want to bring this up because it needs to be clarified if this discussion also touches on the categorization of rules: per tooling vs. per lint category (correctness, style, ...)

The following is how I think about linters, and I would love to hear your perspectives.

Ruff Users

Our users want to craft excellent products quickly, and Ruffs helps them achieve that by giving immediate feedback about potential errors and teaching best practices.

Our users don't want to be linter experts, know all rules by heart, nor spend hours configuring their tooling. We should be the experts and give them access to our knowledge with minimal effort.

When I think about presets, the following user groups come to my mind (there are probably more):

• new to linters: Users who hadn't used a linter before but heard that it helps them write better software. They want to try it out but don't know the rules nor how to set up a linter. Ideally, they can try Ruff without fiddling with any configuration or knowing about any presets or lints.
• users migrating from another linter: They use another linter today but heard that Ruff could speed up their development. They want to use at least the same rules as before but may want to experiment with Ruff's recommended lints to get the most out of it.
• users upgrading Ruff: They already use Ruff, but they're upgrading Ruff (or all their tooling) to a newer version. They want to spend as little time as possible upgrading because they want to work on their products. But they also want to benefit from the newly introduced Ruff rules.
• users changing their software stack: Users that start using a new library or framework, upgrade Python to a more recent version, or make other changes to their stack. They want Ruff to give them the feedback that is relevant to them (no warnings for older Python versions, but helps them write idiomatic code for the frameworks they use).

Ideal Properties

Again, these are the ideal properties that I deduce from the user groups above. Some of these properties touch on fundamental principles we haven't yet defined for Ruff.

• Good defaults: Users can try Ruff with zero configuration. This is what I like about Prettier and clippy. It just works. That's one of the main pain points I experienced with eslint. You can't just run it to try it. You have to go through a configuration wizard and must know new presets to add after introducing new libraries to your stack.
• Preference-focused configuration: Ruff should deduce as much information as possible from the user's environment to run the proper lints for that stack. Ideally, users won't have to configure presets or lints at all (maybe disable a few that they dislike) but only the preferred docstyle format, the supported python versions, do they want to use black's formatting or another, etc.
• "Zero" effort migration: Migrating from an existing linter should be a no-brainer, and Ruff should use precisely the same rules.

Questions

There are a few things that are unclear to me about what users want:

• Do users want to keep up-to-date with tooling specific (pylint, flake8) lints after migrating to Ruff? Do they want to enable all new pylint lints when upgrading Ruff because they're committed to that "style*? Or is it unimportant to them if they get the recommended rules in the python ecosystem (regardless of the source)? This question is important to understand if the migration is a one-step process or the different tools form a "ruleset".
• Do users prefer to think about rules by what kind of errors they catch (Clippy's approach) or group them by frameworks or their source?
• How composable must the solution be: ESLint allows users to enable multiple presets, and a package (e.g., type lints) can export various presets.

[Avasam]

A ESLint-like sharable configs would work great for me and our teams (that's what we use internally for JS/TS: https://www.npmjs.com/package/eslint-config-beslogic ). Big companies already have private package repos (or proxies + CDN) for npm, pip, etc., for dependencies they don't want to publish publicly, to avoid name collisions, and/or want to avoid downtime from an external host.

A URL-like config extension similar to https://github.com/andreoliwa/nitpick#style-library-presets can also work.

[charliermarsh]

Great comment.

To be explicit: in that framing, Ruff has historically biased towards "users migrating from another linter", and has put a strong emphasis on "Zero" effort migration. We made a lot of choices that optimized for that kind of adoption. I think they were good choices, though that doesn't mean that the current API is the "best" API for Ruff in the long term.

Do users prefer to think about rules by what kind of errors they catch (Clippy's approach) or group them by frameworks or their source?

This is an area where there's some path dependence: in the abstract, I would prefer a Clippy-like approach, and I think users would (or could) too...

That said, users coming from Flake8 will naturally think of rules in terms of analogy to existing plugins and tools. And my impression is that being "drop-in" compatible with those existing tools is enormously useful for adoption: you aren't learning new concepts or evaluating new rules, you're just mapping your existing configuration for a 1:1 replacement (with a bunch of other advantages, hopefully).

To that end, my hope is that we can map from Flake8-like configuration to any future configuration schemas in an automated way. I don't think users should have to think in terms of flake8-simplify; rather, they should think about the "complexity" rules.

[charliermarsh]

Some prior discussion on "keeping up with new rules" (no major conclusions in there): #2179

[suimong]

I want to take one step back from technical discussion to understand better the users' needs. I want to bring this up because it needs to be clarified if this discussion also touches on the categorization of rules: per tooling vs. per lint category (correctness, style, ...)

The following is how I think about linters, and I would love to hear your perspectives.

Ruff Users

Our users want to craft excellent products quickly, and Ruffs helps them achieve that by giving immediate feedback about potential errors and teaching best practices.

Our users don't want to be linter experts, know all rules by heart, nor spend hours configuring their tooling. We should be the experts and give them access to our knowledge with minimal effort.

When I think about presets, the following user groups come to my mind (there are probably more):

• new to linters: Users who hadn't used a linter before but heard that it helps them write better software. They want to try it out but don't know the rules nor how to set up a linter. Ideally, they can try Ruff without fiddling with any configuration or knowing about any presets or lints.
• users migrating from another linter: They use another linter today but heard that Ruff could speed up their development. They want to use at least the same rules as before but may want to experiment with Ruff's recommended lints to get the most out of it.
• users upgrading Ruff: They already use Ruff, but they're upgrading Ruff (or all their tooling) to a newer version. They want to spend as little time as possible upgrading because they want to work on their products. But they also want to benefit from the newly introduced Ruff rules.
• users changing their software stack: Users that start using a new library or framework, upgrade Python to a more recent version, or make other changes to their stack. They want Ruff to give them the feedback that is relevant to them (no warnings for older Python versions, but helps them write idiomatic code for the frameworks they use).

Ideal Properties

Again, these are the ideal properties that I deduce from the user groups above. Some of these properties touch on fundamental principles we haven't yet defined for Ruff.

• Good defaults: Users can try Ruff with zero configuration. This is what I like about Prettier and clippy. It just works. That's one of the main pain points I experienced with eslint. You can't just run it to try it. You have to go through a configuration wizard and must know new presets to add after introducing new libraries to your stack.
• Preference-focused configuration: Ruff should deduce as much information as possible from the user's environment to run the proper lints for that stack. Ideally, users won't have to configure presets or lints at all (maybe disable a few that they dislike) but only the preferred docstyle format, the supported python versions, do they want to use black's formatting or another, etc.
• "Zero" effort migration: Migrating from an existing linter should be a no-brainer, and Ruff should use precisely the same rules.

Questions

There are a few things that are unclear to me about what users want:

• Do users want to keep up-to-date with tooling specific (pylint, flake8) lints after migrating to Ruff? Do they want to enable all new pylint lints when upgrading Ruff because they're committed to that "style*? Or is it unimportant to them if they get the recommended rules in the python ecosystem (regardless of the source)? This question is important to understand if the migration is a one-step process or the different tools form a "ruleset".
• Do users prefer to think about rules by what kind of errors they catch (Clippy's approach) or group them by frameworks or their source?
• How composable must the solution be: ESLint allows users to enable multiple presets, and a package (e.g., type lints) can export various presets.

Not sure how it would fit in the broader picture, but I'd like to share my intended use case for "preset" from a user's perspective FYR. I'd imagine having two "presets" in my workflow, namely "dev" & "release". The "dev" preset, which is selected in local development environment, has more rules ignored due to the messy & unstable nature of the code (e.g. F401, F841), whereas in the "release" preset, typically selected for CI environment, rules tends to be much more "strict".

I'm currently using the "extend" keyword to mimic this behavior, but a dedicated preset feature is definitely a plus.

[netopsengineer]

User-Friendly Presets for Legacy Codebases and NetDevOps Beginners in ruff Linter

Introduction

I am proposing the addition of two specific presets in ruff: one tailored for legacy codebases, and another designed for beginners in the NetDevOps space. This aims to lower the barrier to entry for network automation developers and ease the onboarding process, making Python code linting a more accessible and less daunting task for individuals who are more "side of desk" developers.

Feature Request Details

1. Legacy Codebase Preset

   • Objective: Facilitate the transition to modern Python practices for developers working on legacy code.
   • Functionality: Implement selective linting rules focusing on critical aspects like basic PEP8 compliance, security, and performance, while avoiding overwhelming users with extensive legacy code changes.
   • Benefit: Helps maintain and gradually improve legacy systems without requiring significant immediate code overhauls.

2. NetDevOps Beginners Preset

   • Objective: Aid network engineers who are new to Python, making linting an intuitive part of their development process.
   • Functionality: Focus on essential linting rules relevant to network scripting and automation, such as syntax errors, basic structure, and naming conventions.
   • Benefit: Ensures code quality and introduces best practices in a manageable, non-intimidating manner.

Integration with VS Code Extension

• Propose an intuitive interface within the VS Code extension for easy selection and understanding of these presets.
• Allow for gradual increment in linting strictness as users grow more confident.

Importance

• These presets are crucial in making Python development more approachable for network engineers and those dealing with legacy systems.
• They serve as a stepping stone, encouraging better coding practices without overwhelming the users.

Conclusion

Integrating these presets into ruff will significantly enhance the developer experience, particularly for those new to Python or managing legacy codebases. This feature will not only improve code quality but also foster a smoother transition into using linters effectively in the network automation domain.

Next Steps

• I look forward to any feedback or discussion on this proposal.
• Eager to provide further insights to make this feature a reality.

Thank you to @charliermarsh and contributors of the ruff project for considering this proposal. I believe these enhancements will make a meaningful impact in the NetDevOps community and beyond.

[psirenny]

How is everyone currently sharing their configs? My organization is starting to feel the pain of having multiple configs floating out in the wild and it would be great to centralize them. I was thinking of packaging it into a PyPi package similar to how ESLint Configs are distributed on NPM. Then the config could be referenced by file path if it's a package dependency. Which, is easy in our case because our virtual environments happen to be saved to the local git repository. It doesn't feel very ergonomic though.

[Avasam]

I think it's possible to extend from another ruff config file by using the extend field https://docs.astral.sh/ruff/configuration/#pyprojecttoml-discovery

But there's no "online sharing" mechanism in Ruff atm. You can either pass the file around (which will fall out of sync), or use a git submodule with a project that only contains the configs (which kindof allows for version-locking the configs, but is unwieldy).

If/when this gets implemented in Ruff, I'd ask to consider a system that allows downloading the configs at install rather than at runtime to avoid issues with pre-commit.ci restrictions. (like dprint is having, see dprint/dprint#442 )

[ofek]

The next release of Hatch will introduce a fmt command backed by Ruff. I select almost all rules except those that I chose to exclude.

I aim to maintain a selection of rules that would be desirable by most code bases, without caring about transition costs.

[inoa-jboliveira]

Hi everyone,

Is there a way you can categorize the linter rules into buckets such as "critical", "important", "useful", "style", "pedantic". I don't know the actual names you would chose, just that I have this sentiment that some checks are validating syntax (critical), others are like undefined-name is very important if it fails, but can have false positives due to things like "from xxx import *", some bing useful insights but are not necessary, others only care about whitespace and some checks are actually just opinons where you may want to avoid.

The reason for this is that I could have something like this on my config file for everyday use:

[tool.ruff]
select = [ "E:critical", "E:important", "F", "B" ]

But then I could also have a command line on my code validation step for pull requests saying code is rejected regarding critical checks only

ruff check  --select-only "critical" 

[ofek]

FYI Hatch now maintains an extensive set of default settings that are desirable for the majority of users https://hatch.pypa.io/latest/blog/2023/12/11/hatch-v180/#static-analysis

In all of my repos I am now wiping out most of my config in favor of these defaults!

[ofek]

What do you mean exactly?

[dimaqq]

I would just like to know what the default actually are...

> ruff -v some_file.py
[2024-02-02][10:40:25][ruff::resolve][DEBUG] Using Ruff default settings
[2024-02-02][10:40:25][ruff::commands::check][DEBUG] Identified files to lint in: 2.783833ms
[2024-02-02][10:40:25][ruff::commands::check][DEBUG] Checked 1 files in: 405µs

Looks like it's happy... but my file has some weird stuff in it, so 🤷🏻 ?

[charliermarsh]

The defaults are the "F" rules here, and a subset of the "E" rules: https://docs.astral.sh/ruff/rules/#pyflakes-f

[dimaqq]

Here, specifically, isort doesn't appear to be ran.

My pyproject.toml has no ruff config whatsoever, doesn't have a tool-ruff section.

Simple format fixes do get applied though, like missing/extra spaces get corrected, so it's working, but according to what rules?

[dimaqq]

I'm getting somewhere, extend-select = ["I"] warns me about wrong import order, but doesn't fix it on ruff format 😖

[charliermarsh]

Import sorting is part of the linter, rather than part of the formatter (mostly because reordering imports can actually change the behavior of the code, while the formatter tries to avoid changing the behavior of the code at all).

So you can run ruff check --fix with extend-select = ["I"] to enable isort-like import sorting.

[sobolevn]

My use-case is that I have multiple open-source projects: I want to have a shared base config.

[ruff]
preset = "https://link.to/my_config.toml"

Then, I want to change some settings locally:

• Add new rules without overriding everything everywhere like extend-select = ["SIM"]
• Disable some rules that don't apply to this project remove-select = ["PYUP"]
• Maybe change some specific config values: like line-length or complexity rules

This would also be the same for companies using ruff. They can define one config in one source of truth and the use it in all of their projects.