Add B019 check to find cache decorators on class methods #218
Conversation
|
For 2, I just discovered a related FAQ entry that was added a few months ago (via: bpo-44310) which might help craft some (brief) guidance? |
|
that whole |
|
At first glance I think dropping it is pretty easy, mostly just removing a conditional layer here: Lines 60 to 62 in 1c47a16 Which still passes the test suite. Given that this method is an intentional design decision:
If its removal is desired, what would be the best way to update the language in the README, and what version of Something like this? To enable these checks, specify a ``--extend-select`` command-line option or
``extend-select=`` option in your config file (requires Flake8 4.0+)::
[flake8]
max-line-length = 80
max-complexity = 12
...
extend-ignore = E501
extend-select = B901
Some checks might need other flake8 checks disabled - e.g. E501 must be disabled for
B950 to be hit.
If you'd like all optional warnings to be enabled for you (future proof your config!),
say ``B9`` instead of ``B901``. You will need Flake8 3.2+ for this feature.
For Flake8 versions older than 4.0, you will need to use the ``--select`` command-line
option or ``select=`` option in your config file. As of Flake8 3.0, this option is a
whitelist (checks not listed are implicitly disabled), so you have to explicitly specify
all checks you want enabled.
The ``--extend-ignore`` command-line option and ``extend-ignore=`` config file option
require Flake8 3.6+. For older Flake8 versions, the ``--ignore`` and ``ignore=`` options
can be used. Specifying this option will override all codes that are disabled by
default, so you will need to specify these codes in your configuration to silence them. |
Starting in Python 3.8, the function node definition's `lineno` is changed to index its `def ...` line rather than the first line where its decorators start. This causes inconsistent line numbers across Python versions for the line reported by Flake8. We can use the decorator node location instead, which provides a consistent location, and makes sense because this hits on decorators.
|
So I think we should look at moving to all the latest select hotness, my only worry is will we break existing configuration for users of flake8-bugbear - Is there anything we can do to support both for now and I can do a release advertising the deprecation of things we need to deprecate? |
|
The filter method can be left in & just made aware of Depending on the answer to question 3 in the OP, the method's |
Yeah I'm happy to start recommending the new and lets state we're deprecating the old in README + Change log (even tho its same file) as I think some tooling parse out he change log + I paste into the GitHub Release page. |
* Prefer `extend-select` and `extend-ignore` for configuring opinionated warnings (`B9`) * Add deprecation note for Bugbear's internal handling of whether or not to emit `B9` codes * Add an example for `extend-immutable-call` specification
|
I've updated the README to (hopefully!) capture all the possible configuration caveats. I've also included a few minor fixes for other issues:
I've edited the OP down to the remaining questions I have on this draft implementation. |
* The code for Bugbear's built-in filtering for opinionated warnings predates the addition of `extend-select` to flake8 (`v4.0`) so it's not part of the check for explicit specification of `B9` codes. * Switch from `Mock` to `argparse.Namespace` for specifying options to tests to match the incoming type from `flake8` and avoid mocking side-effects.
bugbear.py
Outdated
| # Preserve decorator order so we can get the lineno from the decorator node rather than | ||
| # the function node (this location definition changes in Python 3.8) |
There was a problem hiding this comment.
So we can clean this up when >=3.8 as minimum version, yes? If so maybe explicitly say that so I and other maintainers remember.
There was a problem hiding this comment.
It depends where we ultimately want the emitted error to point. This isn't as slick as the set intersection but does consistently point the error at the first offending decorator rather than the function's def line, or the line where the decorators start (Python <3.8).
|
If 3.6 is a pain, since it's EOL'd cpython support, I'm happy to drop it here too. |
|
It's not a major pain, it's mostly me just doing a dumb thing and trying to use |
Co-authored-by: Cooper Ry Lees <me@cooperlees.com>
|
If you have it fixed lets do 1 more 3.6 capable release and drop it next release :) Thanks! |
Flake8-bugbear correctly pointed out that TabBar instances would not be reliably cleaned up because the `self` reference would be cached in the lru_caches on a couple of the methods. And the caches are on the class so they last for the whole lifetime of the process. This commit move the caches to be created per instance, instead of on the class. Other options: 1. get rid of the caches From running the benchmark tests (eg `python3 -m pytest --benchmark-columns Min,Max,Median,Rounds -k test_update_tab_titles_benchmark`) it seems like the caches can still be helpful (even though when they were introduced in #3122 we didn't have the config cache either) 2. use cachetools to manage our own cache instead of using lru_cache in a non-standard way I don't feel like introducing a new dependency given this change didn't end up being too offensive. 3. clear the cache whenever a window get closed That would solve the "not getting deleted issue" but flake8 would still complain :) Possibly the cache size could be reduced now that there is going to be one per window. But the aren't caching large objects anyway. Flake8-bugbear change: PyCQA/flake8-bugbear#218 Video that pointed out this way of using lru_cache: https://youtu.be/sVjtp6tGo0g
|
@sco1 I am totally confused. The error message says "class methods", but the actual behavior is to throw an error when "instance methods" are cached. I looked into the tests and this is how I read them. What do I miss? |
|
@khrapovs I don't understand your question. It would probably be a good idea open a new issue with an example. |
|
Answered in #250 |
|
I may be missing it just scanning the above, but can someone mention the thinking on:
My reading of the upstream issue also seems to agree with the above (that you're likely OK if you're not setting |
Creating lots of RefResolvers shouldn't be common (as creating lots of Validators should be reasonably uncommon itself), and even if one did so, we have a fixed size LRU cache, so this seems OK. Refs: PyCQA/flake8-bugbear#218
|
Just some thoughts as a non-maintainer:
Maybe, but I'm not sure many are even aware of the potential pitfall. Not to get into the hypothetical weeds too much, but the point where you'd incur a non-trivial performance cost is going to vary between projects, if you even approach it at all; theoretically it could even be well below the default FWIW, there's also a similar discussion happening over at Pylint: pylint-dev/pylint#5670. It looks like they're opting to only warn for |
Flake8-bugbear correctly pointed out that TabBar instances would not be reliably cleaned up because the `self` reference would be cached in the lru_caches on a couple of the methods. And the caches are on the class so they last for the whole lifetime of the process. This commit move the caches to be created per instance, instead of on the class. Other options: 1. get rid of the caches From running the benchmark tests (eg `python3 -m pytest --benchmark-columns Min,Max,Median,Rounds -k test_update_tab_titles_benchmark`) it seems like the caches can still be helpful (even though when they were introduced in qutebrowser#3122 we didn't have the config cache either) 2. use cachetools to manage our own cache instead of using lru_cache in a non-standard way I don't feel like introducing a new dependency given this change didn't end up being too offensive. 3. clear the cache whenever a window get closed That would solve the "not getting deleted issue" but flake8 would still complain :) Possibly the cache size could be reduced now that there is going to be one per window. But the aren't caching large objects anyway. Flake8-bugbear change: PyCQA/flake8-bugbear#218 Video that pointed out this way of using lru_cache: https://youtu.be/sVjtp6tGo0g
As described in #217, using
functools.cache/functools.lru_cacheon class methods can potentially cause their instances to live longer than expected.I took a stab at adding a rule to catch this pattern but I have a few questions:
flake8-bugbear's error messages generally try to propose a solution to make the linting error go away (besides ignoring the line), but I'm not sure about the verbiage to include. For reference, the starting point is the following:functools.cacheandfunctools.lru_cache(maxsize=None). This seems to more specifically address the issue though some still may not want to potentially keep these instances alive, bounded cache or not.Related reading:
Resolves: #217
Resolves: #215
Resolves: #212
Resolves: #221