[MacOS] astroid.exceptions.AstroidError thrown when multi-processing

[jacekziembla]

Bug description

I am using python 3.8.
When running on the jenkins agent in dockerized (Alpine-based image) script everything works fine. When runnning locally on MacOS 14.4.1 (with Intel Chip), I am getting the attached output.
I've tried to align python versions between one used in our image (3.8.16) and the issue was still there. I've also tried to upgrade pylint version to 3.1.0 (together with Astroid - initially it was locked at pylint was locked at 2.4.4), but it did not help.

MacOS Sonoma 14.4.1 (Intel Chip)
Python versions (installed via pyenv: 3.8.9 and 3.8.16)
Pylint versions: 2.4.4; 2.17.4; 3.1.0 (astroid versions 2.3.3 and 3.1.0)

The thing that mitigates the impact of this issue for local execution is passing -j 1 to use single thread for processing.

Configuration

# Use multiple processes to speed up Pylint.
jobs=0

Command used

poetry run pylint src/

Pylint output

Exception on node <FunctionDef.get_column_content l.29 at 0x1032ee1f0> in file '/Users/jacekz/workspace/kit/kit-admin-models/src/kit_admin_models/parts/tag_mappings_table.py'
Traceback (most recent call last):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 96, in walk
    callback(astroid)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1000, in leave_functiondef
    self._check_consistent_returns(node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1916, in _check_consistent_returns
    ) and self._is_node_return_ended(node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2002, in _is_node_return_ended
    if self._is_function_def_never_returning(funcdef_node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2057, in _is_function_def_never_returning
    if isinstance(node, (nodes.FunctionDef, astroid.BoundMethod)) and node.returns:
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/astroid/bases.py", line 138, in __getattr__
    return getattr(self._proxied, name)
AttributeError: 'Lambda' object has no attribute 'returns'
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 830, in _check_file
    check_astroid_module(ast_node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1016, in check_astroid_module
    retval = self._check_astroid_module(
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1068, in _check_astroid_module
    walker.walk(node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 96, in walk
    callback(astroid)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1000, in leave_functiondef
    self._check_consistent_returns(node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1916, in _check_consistent_returns
    ) and self._is_node_return_ended(node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2002, in _is_node_return_ended
    if self._is_function_def_never_returning(funcdef_node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2057, in _is_function_def_never_returning
    if isinstance(node, (nodes.FunctionDef, astroid.BoundMethod)) and node.returns:
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/astroid/bases.py", line 138, in __getattr__
    return getattr(self._proxied, name)
AttributeError: 'Lambda' object has no attribute 'returns'
​
The above exception was the direct cause of the following exception:
​
Traceback (most recent call last):
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 198, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 198, in <listcomp>
    return [fn(*args) for args in chunk]
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/parallel.py", line 79, in _worker_check_single_file
    _worker_linter.check_single_file_item(file_item)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 739, in check_single_file_item
    self._check_file(self.get_ast, check_astroid_module, file)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 832, in _check_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""
​
The above exception was the direct cause of the following exception:
​
Traceback (most recent call last):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/__init__.py", line 34, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/run.py", line 211, in __init__
    linter.check(args)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 678, in check
    check_parallel(
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/parallel.py", line 153, in check_parallel
    for (
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 484, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
    yield fs.pop().result()
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
astroid.exceptions.AstroidError
(kit-admin-models-py3.8) jacekz@Q-C02Z25ABLVDL-jacekz kit-admin-models (pylint-start) $ pylint src/kit_admin_models/parts/tag_mappings_table.py
************* Module /Users/jacekz/workspace/kit/kit-admin-models/pylintrc
pylintrc:1:0: W0012: Unknown option value for '--disable', expected a valid pylint message and got 'specific_check_code' (unknown-option-value)
Exception on node <FunctionDef.get_column_content l.29 at 0x10e229220> in file '/Users/jacekz/workspace/kit/kit-admin-models/src/kit_admin_models/parts/tag_mappings_table.py'
Traceback (most recent call last):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 96, in walk
    callback(astroid)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1000, in leave_functiondef
    self._check_consistent_returns(node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1916, in _check_consistent_returns
    ) and self._is_node_return_ended(node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2002, in _is_node_return_ended
    if self._is_function_def_never_returning(funcdef_node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2057, in _is_function_def_never_returning
    if isinstance(node, (nodes.FunctionDef, astroid.BoundMethod)) and node.returns:
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/astroid/bases.py", line 138, in __getattr__
    return getattr(self._proxied, name)
AttributeError: 'Lambda' object has no attribute 'returns'
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 830, in _check_file
    check_astroid_module(ast_node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1016, in check_astroid_module
    retval = self._check_astroid_module(
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1068, in _check_astroid_module
    walker.walk(node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 94, in walk
    self.walk(child)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 96, in walk
    callback(astroid)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1000, in leave_functiondef
    self._check_consistent_returns(node)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1916, in _check_consistent_returns
    ) and self._is_node_return_ended(node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in _is_node_return_ended
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2034, in <genexpr>
    return any(self._is_node_return_ended(_child) for _child in node.get_children())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2002, in _is_node_return_ended
    if self._is_function_def_never_returning(funcdef_node):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 2057, in _is_function_def_never_returning
    if isinstance(node, (nodes.FunctionDef, astroid.BoundMethod)) and node.returns:
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/astroid/bases.py", line 138, in __getattr__
    return getattr(self._proxied, name)
AttributeError: 'Lambda' object has no attribute 'returns'
​
The above exception was the direct cause of the following exception:
​
Traceback (most recent call last):
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 198, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 198, in <listcomp>
    return [fn(*args) for args in chunk]
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/parallel.py", line 79, in _worker_check_single_file
    _worker_linter.check_single_file_item(file_item)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 739, in check_single_file_item
    self._check_file(self.get_ast, check_astroid_module, file)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 832, in _check_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError
"""
​
The above exception was the direct cause of the following exception:
​
Traceback (most recent call last):
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/__init__.py", line 34, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/run.py", line 211, in __init__
    linter.check(args)
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 678, in check
    check_parallel(
  File "/Users/jacekz/workspace/kit/kit-admin-models/.venv/lib/python3.8/site-packages/pylint/lint/parallel.py", line 153, in check_parallel
    for (
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/process.py", line 484, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
    yield fs.pop().result()
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/Users/jacekz/.pyenv/versions/3.8.9/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
astroid.exceptions.AstroidError

Expected behavior

pylint does not crash when running locally without specifying # of threads

Pylint version

pylint 3.1.0
astroid 3.1.0
Python 3.8.16

and

pylint 2.4.4
astroid 2.3.3
Python 3.8.16

OS / Environment

MacOS Sonoma 14.4.1 (Intel Chip)