Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execution as python module broken #4161

Closed
cdce8p opened this issue Mar 1, 2021 · 4 comments · Fixed by #4164
Closed

Execution as python module broken #4161

cdce8p opened this issue Mar 1, 2021 · 4 comments · Fixed by #4164
Labels
Blocker 🙅 Blocks the next release Crash 💥 A bug that makes pylint crash Regression
Milestone

Comments

@cdce8p
Copy link
Member

cdce8p commented Mar 1, 2021

Steps to reproduce

git@github.com:PyCQA/pylint.git
cd pylint
Python3.8 -m venv venv-38
source venv-38/bin/activate
python -m pip install -U pip setuptools wheel
pip install -e .

python -m pylint a.py

Given a file a.py:

# can be empty

Current behavior

Traceback below

Expected behavior

No errors

Additional notes

Reverting #4153 solve the issue
At first glance it seems to be related to the handling of the Python virtual environment.

pylint --version output

pylint 2.8.0-dev1
astroid 2.5.1
Python 3.8.8 (v3.8.8:024d8058b0, Feb 19 2021, 08:48:17) 
[Clang 6.0 (clang-600.0.57)]

Traceback

Problem importing module logging.py: Unable to find module for /Users/.../pylint/pylint/checkers/logging.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module spelling.py: Unable to find module for /Users/.../pylint/pylint/checkers/spelling.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module python3.py: Unable to find module for /Users/.../pylint/pylint/checkers/python3.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module misc.py: Unable to find module for /Users/.../pylint/pylint/checkers/misc.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module typecheck.py: Unable to find module for /Users/.../pylint/pylint/checkers/typecheck.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module variables.py: Unable to find module for /Users/.../pylint/pylint/checkers/variables.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module deprecated.py: Unable to find module for /Users/.../pylint/pylint/checkers/deprecated.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module base_checker.py: Unable to find module for /Users/.../pylint/pylint/checkers/base_checker.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module format.py: Unable to find module for /Users/.../pylint/pylint/checkers/format.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module mapreduce_checker.py: Unable to find module for /Users/.../pylint/pylint/checkers/mapreduce_checker.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module imports.py: Unable to find module for /Users/.../pylint/pylint/checkers/imports.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module utils.py: Unable to find module for /Users/.../pylint/pylint/checkers/utils.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module raw_metrics.py: Unable to find module for /Users/.../pylint/pylint/checkers/raw_metrics.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module newstyle.py: Unable to find module for /Users/.../pylint/pylint/checkers/newstyle.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module exceptions.py: Unable to find module for /Users/.../pylint/pylint/checkers/exceptions.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module classes.py: Unable to find module for /Users/.../pylint/pylint/checkers/classes.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module stdlib.py: Unable to find module for /Users/.../pylint/pylint/checkers/stdlib.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module async.py: Unable to find module for /Users/.../pylint/pylint/checkers/async.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module refactoring: Unable to find module for /Users/.../pylint/pylint/checkers/refactoring in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module similar.py: Unable to find module for /Users/.../pylint/pylint/checkers/similar.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module design_analysis.py: Unable to find module for /Users/.../pylint/pylint/checkers/design_analysis.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module base.py: Unable to find module for /Users/.../pylint/pylint/checkers/base.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module strings.py: Unable to find module for /Users/.../pylint/pylint/checkers/strings.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module ureports: Unable to find module for /Users/.../pylint/pylint/reporters/ureports in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module json_reporter.py: Unable to find module for /Users/.../pylint/pylint/reporters/json_reporter.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module reports_handler_mix_in.py: Unable to find module for /Users/.../pylint/pylint/reporters/reports_handler_mix_in.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module base_reporter.py: Unable to find module for /Users/.../pylint/pylint/reporters/base_reporter.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module collecting_reporter.py: Unable to find module for /Users/.../pylint/pylint/reporters/collecting_reporter.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module text.py: Unable to find module for /Users/.../pylint/pylint/reporters/text.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Traceback (most recent call last):
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 511, in _load_reporter
    reporter_class = self._load_reporter_class()
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 519, in _load_reporter_class
    module = modutils.load_module_from_name(modutils.get_module_part(qname))
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 435, in get_module_part
    file_from_modpath(
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 331, in file_from_modpath
    return file_info_from_modpath(modpath, path, context_file).location
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 381, in file_info_from_modpath
    return _spec_from_modpath(modpath, path, context)
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 612, in _spec_from_modpath
    found_spec = spec.find_spec(modpath, path)
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py", line 368, in find_spec
    finder, spec = _find_spec_with_path(
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py", line 337, in _find_spec_with_path
    raise ImportError("No module named %s" % ".".join(module_parts))
ImportError: No module named text

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/.../pylint/pylint/__main__.py", line 20, in <module>
    pylint.run_pylint()
  File "/Users/.../pylint/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/Users/.../pylint/pylint/lint/run.py", line 264, in __init__
    linter.load_default_plugins()
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 480, in load_default_plugins
    self._load_reporter()
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 513, in _load_reporter
    raise exceptions.InvalidReporterError(name) from e
pylint.exceptions.InvalidReporterError: text
@Pierre-Sassoulas Pierre-Sassoulas added Regression Crash 💥 A bug that makes pylint crash Blocker 🙅 Blocks the next release labels Mar 1, 2021
@Pierre-Sassoulas Pierre-Sassoulas added this to the 2.7.3 milestone Mar 1, 2021
@sbraz
Copy link
Contributor

sbraz commented Mar 1, 2021

I can't reproduce except if I cd into venv/lib/python3.9/site-packages, what is your working directory when running this? I assume it would only fail if you're running the module from the parent of the package folder.

@cdce8p
Copy link
Member Author

cdce8p commented Mar 1, 2021

My mistake, I just assumed it would occur in a normal context.
I noticed the issue in my pylint dev environment, since this the the VS Code command for debugging.

My setup:

git@github.com:PyCQA/pylint.git
cd pylint
Python3.8 -m venv venv-38
source venv-38/bin/activate
python -m pip install -U pip setuptools wheel
pip install -e .

touch a.py
python -m pylint a.py

I also did just look at sys.path. It seems that in editable mode the CWD is added as last path.
Before:

['/Users/.../pylint',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
 '/Users/.../pylint/venv-38/lib/python3.8/site-packages',
 '/Users/.../pylint']

After:

['/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
 '/Users/.../pylint/venv-38/lib/python3.8/site-packages']

This was the output with the old version

['/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
 '/Users/.../pylint/venv-38/lib/python3.8/site-packages',
 '/Users/.../pylint']

--
As you can see here: https://bugs.python.org/msg313966, the CWD is only added as first element in sys.path. So the previous version should be enough. I would recommend to revert #4153.

@sbraz Did you had any issues before you submitted the MR?

@sbraz
Copy link
Contributor

sbraz commented Mar 1, 2021

@cdce8p Yup, it's explained in my PR (it fixes #3636). If PYTHONPATH ends or starts with a colon, two tests fail because the CWD is added twice to sys.path.

@cdce8p cdce8p mentioned this issue Mar 1, 2021
4 tasks
@cdce8p
Copy link
Member Author

cdce8p commented Mar 1, 2021

Opened #4164 to address the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Blocker 🙅 Blocks the next release Crash 💥 A bug that makes pylint crash Regression
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants