Skip to content

Execution as python module broken #4161

Closed
@cdce8p

Description

@cdce8p
Member

Steps to reproduce

git@github.com:PyCQA/pylint.git
cd pylint
Python3.8 -m venv venv-38
source venv-38/bin/activate
python -m pip install -U pip setuptools wheel
pip install -e .

python -m pylint a.py

Given a file a.py:

# can be empty

Current behavior

Traceback below

Expected behavior

No errors

Additional notes

Reverting #4153 solve the issue
At first glance it seems to be related to the handling of the Python virtual environment.

pylint --version output

pylint 2.8.0-dev1
astroid 2.5.1
Python 3.8.8 (v3.8.8:024d8058b0, Feb 19 2021, 08:48:17) 
[Clang 6.0 (clang-600.0.57)]

Traceback

Problem importing module logging.py: Unable to find module for /Users/.../pylint/pylint/checkers/logging.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module spelling.py: Unable to find module for /Users/.../pylint/pylint/checkers/spelling.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module python3.py: Unable to find module for /Users/.../pylint/pylint/checkers/python3.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module misc.py: Unable to find module for /Users/.../pylint/pylint/checkers/misc.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module typecheck.py: Unable to find module for /Users/.../pylint/pylint/checkers/typecheck.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module variables.py: Unable to find module for /Users/.../pylint/pylint/checkers/variables.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module deprecated.py: Unable to find module for /Users/.../pylint/pylint/checkers/deprecated.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module base_checker.py: Unable to find module for /Users/.../pylint/pylint/checkers/base_checker.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module format.py: Unable to find module for /Users/.../pylint/pylint/checkers/format.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module mapreduce_checker.py: Unable to find module for /Users/.../pylint/pylint/checkers/mapreduce_checker.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module imports.py: Unable to find module for /Users/.../pylint/pylint/checkers/imports.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module utils.py: Unable to find module for /Users/.../pylint/pylint/checkers/utils.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module raw_metrics.py: Unable to find module for /Users/.../pylint/pylint/checkers/raw_metrics.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module newstyle.py: Unable to find module for /Users/.../pylint/pylint/checkers/newstyle.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module exceptions.py: Unable to find module for /Users/.../pylint/pylint/checkers/exceptions.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module classes.py: Unable to find module for /Users/.../pylint/pylint/checkers/classes.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module stdlib.py: Unable to find module for /Users/.../pylint/pylint/checkers/stdlib.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module async.py: Unable to find module for /Users/.../pylint/pylint/checkers/async.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module refactoring: Unable to find module for /Users/.../pylint/pylint/checkers/refactoring in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module similar.py: Unable to find module for /Users/.../pylint/pylint/checkers/similar.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module design_analysis.py: Unable to find module for /Users/.../pylint/pylint/checkers/design_analysis.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module base.py: Unable to find module for /Users/.../pylint/pylint/checkers/base.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module strings.py: Unable to find module for /Users/.../pylint/pylint/checkers/strings.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module ureports: Unable to find module for /Users/.../pylint/pylint/reporters/ureports in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module json_reporter.py: Unable to find module for /Users/.../pylint/pylint/reporters/json_reporter.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module reports_handler_mix_in.py: Unable to find module for /Users/.../pylint/pylint/reporters/reports_handler_mix_in.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module base_reporter.py: Unable to find module for /Users/.../pylint/pylint/reporters/base_reporter.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module collecting_reporter.py: Unable to find module for /Users/.../pylint/pylint/reporters/collecting_reporter.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Problem importing module text.py: Unable to find module for /Users/.../pylint/pylint/reporters/text.py in /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8, 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages, 
/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/brain
Traceback (most recent call last):
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 511, in _load_reporter
    reporter_class = self._load_reporter_class()
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 519, in _load_reporter_class
    module = modutils.load_module_from_name(modutils.get_module_part(qname))
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 435, in get_module_part
    file_from_modpath(
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 331, in file_from_modpath
    return file_info_from_modpath(modpath, path, context_file).location
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 381, in file_info_from_modpath
    return _spec_from_modpath(modpath, path, context)
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/modutils.py", line 612, in _spec_from_modpath
    found_spec = spec.find_spec(modpath, path)
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py", line 368, in find_spec
    finder, spec = _find_spec_with_path(
  File "/Users/.../pylint/venv-38/lib/python3.8/site-packages/astroid/interpreter/_import/spec.py", line 337, in _find_spec_with_path
    raise ImportError("No module named %s" % ".".join(module_parts))
ImportError: No module named text

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/.../pylint/pylint/__main__.py", line 20, in <module>
    pylint.run_pylint()
  File "/Users/.../pylint/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/Users/.../pylint/pylint/lint/run.py", line 264, in __init__
    linter.load_default_plugins()
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 480, in load_default_plugins
    self._load_reporter()
  File "/Users/.../pylint/pylint/lint/pylinter.py", line 513, in _load_reporter
    raise exceptions.InvalidReporterError(name) from e
pylint.exceptions.InvalidReporterError: text

Activity

added this to the 2.7.3 milestone on Mar 1, 2021
sbraz

sbraz commented on Mar 1, 2021

@sbraz
Contributor

I can't reproduce except if I cd into venv/lib/python3.9/site-packages, what is your working directory when running this? I assume it would only fail if you're running the module from the parent of the package folder.

cdce8p

cdce8p commented on Mar 1, 2021

@cdce8p
MemberAuthor

My mistake, I just assumed it would occur in a normal context.
I noticed the issue in my pylint dev environment, since this the the VS Code command for debugging.

My setup:

git@github.com:PyCQA/pylint.git
cd pylint
Python3.8 -m venv venv-38
source venv-38/bin/activate
python -m pip install -U pip setuptools wheel
pip install -e .

touch a.py
python -m pylint a.py

I also did just look at sys.path. It seems that in editable mode the CWD is added as last path.
Before:

['/Users/.../pylint',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
 '/Users/.../pylint/venv-38/lib/python3.8/site-packages',
 '/Users/.../pylint']

After:

['/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
 '/Users/.../pylint/venv-38/lib/python3.8/site-packages']

This was the output with the old version

['/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
 '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
 '/Users/.../pylint/venv-38/lib/python3.8/site-packages',
 '/Users/.../pylint']

--
As you can see here: https://bugs.python.org/msg313966, the CWD is only added as first element in sys.path. So the previous version should be enough. I would recommend to revert #4153.

@sbraz Did you had any issues before you submitted the MR?

sbraz

sbraz commented on Mar 1, 2021

@sbraz
Contributor

@cdce8p Yup, it's explained in my PR (it fixes #3636). If PYTHONPATH ends or starts with a colon, two tests fail because the CWD is added twice to sys.path.

cdce8p

cdce8p commented on Mar 1, 2021

@cdce8p
MemberAuthor

Opened #4164 to address the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      Participants

      @sbraz@Pierre-Sassoulas@cdce8p

      Issue actions

        Execution as python module broken · Issue #4161 · pylint-dev/pylint