Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash maximum recursion depth exceeded - something to do with decorators #4982

Closed
Timmmm opened this issue Sep 7, 2021 · 6 comments · Fixed by #5339
Closed

Crash maximum recursion depth exceeded - something to do with decorators #4982

Timmmm opened this issue Sep 7, 2021 · 6 comments · Fixed by #5339
Labels
Astroid Related to astroid Crash 💥 A bug that makes pylint crash Needs astroid update Needs an astroid update (probably a release too) before being mergable

Comments

@Timmmm
Copy link

Timmmm commented Sep 7, 2021 •
edited by Pierre-Sassoulas

Bug description

When parsing the following file:

import verif.sim

and verif/sim.py is

class foo:
    @classmethod
    def foo(cls):
        return cls.__subclasses__()

bar = foo.foo()

class sim(bar):
    pass

pylint crashed with a RecursionError and with the following stacktrace:

Traceback (most recent call last):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1024, in _check_files
    self._check_file(
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1063, in _check_file
    check_astroid_module(ast_node)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1200, in check_astroid_module
    retval = self._check_astroid_module(
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1245, in _check_astroid_module
    walker.walk(ast_node)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 78, in walk
    self.walk(child)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    callback(astroid)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/checkers/imports.py", line 510, in visit_import
    imported_module = self._get_imported_module(node, name)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/pylint/checkers/imports.py", line 784, in _get_imported_module
    return importnode.do_import_module(modname)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/mixins.py", line 101, in do_import_module
    return mymodule.import_module(
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 699, in import_module
    return AstroidManager().ast_from_module_name(absmodname)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/manager.py", line 206, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/manager.py", line 119, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/builder.py", line 139, in file_build
    return self._post_build(module, encoding)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/builder.py", line 167, in _post_build
    module = self._manager.visit_transforms(module)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/manager.py", line 96, in visit_transforms
    return self._transform.visit(node)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 96, in visit
    return self._visit(module)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 61, in _visit
    visited = self._visit_generic(value)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 68, in _visit_generic
    return [self._visit_generic(child) for child in node]
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 68, in <listcomp>
    return [self._visit_generic(child) for child in node]
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 74, in _visit_generic
    return self._visit(node)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 64, in _visit
    return self._transform(node)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/transforms.py", line 45, in _transform
    if predicate is None or predicate(node):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/brain/brain_namedtuple_enum.py", line 554, in _is_enum_subclass
    for klass in cls.mro()
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 3030, in mro
    return self._compute_mro(context=context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2999, in _compute_mro
    inferred_bases = list(self._inferred_bases(context=context))
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2982, in _inferred_bases
    baseobj = next(
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2982, in <genexpr>
    baseobj = next(
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/bases.py", line 154, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/bases.py", line 154, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/inference.py", line 244, in infer_call
    yield from callee.infer_call_result(caller=self, context=callcontext)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 1806, in infer_call_result
    yield from returnnode.value.infer(context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/inference.py", line 235, in infer_call
    for callee in self.func.infer(context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/inference.py", line 318, in infer_attribute
    yield from owner.igetattr(self.attrname, context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2606, in igetattr
    attributes = self.getattr(name, context, class_context=class_context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2523, in getattr
    result = [self.special_attributes.lookup(name)]
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py", line 126, in lookup
    return getattr(self, IMPL_PREFIX + name)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py", line 485, in attr___subclasses__
    classes = [
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py", line 488, in <listcomp>
    if cls != self._instance and cls.is_subtype_of(qname)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2203, in is_subtype_of
    for anc in self.ancestors(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2362, in ancestors
    for baseobj in stmt.infer(context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/bases.py", line 154, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/bases.py", line 154, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/inference.py", line 244, in infer_call
    yield from callee.infer_call_result(caller=self, context=callcontext)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 1806, in infer_call_result
    yield from returnnode.value.infer(context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/inference.py", line 235, in infer_call
    for callee in self.func.infer(context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/inference.py", line 318, in infer_attribute
    yield from owner.igetattr(self.attrname, context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2606, in igetattr
    attributes = self.getattr(name, context, class_context=class_context)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2523, in getattr
    result = [self.special_attributes.lookup(name)]
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py", line 126, in lookup
    return getattr(self, IMPL_PREFIX + name)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py", line 485, in attr___subclasses__
    classes = [
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/interpreter/objectmodel.py", line 488, in <listcomp>
    if cls != self._instance and cls.is_subtype_of(qname)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2203, in is_subtype_of
    for anc in self.ancestors(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/scoped_nodes.py", line 2362, in ancestors
    for baseobj in stmt.infer(context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/bases.py", line 154, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 129, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/decorators.py", line 98, in wrapped
    for res in _func(node, context, **kwargs):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/bases.py", line 154, in _infer_stmts
    for inf in stmt.infer(context=context):
  File "/home/ap/python/site/3.8.7/20210903/lib/python3.8/site-packages/astroid/nodes/node_ng.py", line 113, in infer
    for i, result in enumerate(generator):
...

Command used

pylint test.py

Pylint output

See above.

Expected behavior

It shouldn't crash.

Pylint version

pylint 2.10.2
astroid 2.7.3
Python 3.8.7 (default, Jan  9 2021, 01:55:12) 
[Clang 9.0.0 (tags/RELEASE_900/final)]

OS / Environment

CentOS Linux release 7.9.2009 (Core)

Additional dependencies

No response
@Timmmm Timmmm added Bug 🪲 Needs triage 📥 Just created, needs acknowledgment, triage, and proper labelling labels Sep 7, 2021
@DudeNr33
Copy link
Collaborator

DudeNr33 commented Sep 7, 2021 •
edited

I can reproduce the crash, but as I see it, the given example is not valid Python code.
If you try to run the code itself, you will get a TypeError, because you pass in a list instead of individual base classes:

class sim(bar)

Instead, you'd need to unpack the list of subclasses:

class sim(*bar)

With this modification, pylint no longer crashes.

@DudeNr33 DudeNr33 added Astroid Related to astroid Crash 💥 A bug that makes pylint crash Needs investigation 🔬 A bug or crash where it's not immediately obvious what is happenning and removed Needs triage 📥 Just created, needs acknowledgment, triage, and proper labelling Bug 🪲 Astroid Related to astroid Crash 💥 A bug that makes pylint crash labels Sep 7, 2021
@Timmmm
Copy link
Author

Timmmm commented Sep 7, 2021

Right but pylint shouldn't crash even on invalid code surely?

@DudeNr33 DudeNr33 added Astroid Related to astroid Crash 💥 A bug that makes pylint crash labels Sep 7, 2021
@DudeNr33
Copy link
Collaborator

DudeNr33 commented Sep 7, 2021

Yes, that is true.

@Timmmm
Copy link
Author

Timmmm commented Sep 7, 2021

The following is valid Python and still produces the crash:

class Base:
    @classmethod
    def get_first_subclass(cls):
        for subclass in cls.__subclasses__():
            return subclass
        return object

subclass = Base.get_first_subclass()

class Another(subclass):
    pass

This is based on real code by the way; not a contrived example.

@DudeNr33
Copy link
Collaborator

Thank you for the additional example.
I could at least narrow it down that the RecursionError is triggered when astroid.nodes.scoped_nodes.ClassDef.mro() is called on the ClassDef object representing Another.

@zachliu
Copy link

zachliu commented Sep 29, 2021 •
edited

i can reproduce the error using your examples in this env:

pylint 2.11.1
astroid 2.8.0
Python 3.8.4 (default, Nov 20 2020, 22:14:52)
[GCC 9.3.0]

but not in this env where the same error occurred on other Python files 😹 :

pylint 2.6.0
astroid 2.4.2
Python 3.8.4 (default, Nov 20 2020, 22:14:52)
[GCC 9.3.0]

in both envs, if i put the python files inside a directory, say, pkg/ and do

pylint pkg/

then everything is fine, no errors 🤔

@DanielNoord DanielNoord mentioned this issue Nov 18, 2021
Merged
2 tasks
@DanielNoord DanielNoord added Needs astroid update Needs an astroid update (probably a release too) before being mergable and removed Needs investigation 🔬 A bug or crash where it's not immediately obvious what is happenning labels Nov 18, 2021
@DanielNoord DanielNoord mentioned this issue Nov 18, 2021
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Astroid Related to astroid Crash 💥 A bug that makes pylint crash Needs astroid update Needs an astroid update (probably a release too) before being mergable
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add regression test for #4982 DanielNoord/pylint
4 participants
@Timmmm @DudeNr33 @DanielNoord @zachliu