Skip to content

No module importlib.readers breaks pytest==7.0.0 with python 3.9 #9608

Closed
@kdelee

Description

@kdelee
Contributor

My project is using python 3.9 and we hit this today when 7.0.0 released:

if sys.version_info >= (3, 9):
def get_resource_reader(self, name: str) -> importlib.abc.TraversableResources: # type: ignore
from types import SimpleNamespace
from importlib.readers import FileReader

Because this import does not seem to be working/reference existing code

/var/lib/awx/venv/awx/lib64/python3.9/site-packages/_pytest/assertion/rewrite.py:283: in get_resource_reader
    from importlib.readers import FileReader
E   ModuleNotFoundError: No module named 'importlib.readers'

If you'd like to see where this is happening on our CI, I'm pretty sure you can see the logs for the api-test stage here https://github.com/ansible/awx/runs/5072172116?check_suite_focus=true

Downgrading pytest fixes the issue, so it seems to a problem exclusively in 7.0.0 right now.

  • a detailed description of the bug or problem you are having
    output of pip list from the virtual environment you are using piplist.txt
    pytest==7.0.0
    CentOS Stream release 9
    minimal example if possible

This minimal reproducer re-creates the error:
say we have broken.py with the following contents

from pip._internal.req import parse_requirements


def test_broken():
    assert True

then running pytest broken.py results in

==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.9.10, pytest-7.0.0, pluggy-1.0.0
django: settings: awx.settings.development (from ini)
rootdir: /awx_devel, configfile: pytest.ini
plugins: mock-1.11.1, forked-1.4.0, django-4.5.2, xdist-1.34.0, cov-3.0.0, timeout-2.1.0
collected 0 items / 1 error                                                                                                                                                                  

=========================================================================================== ERRORS ===========================================================================================
_________________________________________________________________________________ ERROR collecting broken.py _________________________________________________________________________________
ImportError while importing test module '/awx_devel/broken.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
broken.py:1: in <module>
    from pip._internal.req import parse_requirements
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_internal/req/__init__.py:5: in <module>
    from pip._internal.utils.logging import indent_log
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_internal/utils/logging.py:23: in <module>
    from pip._internal.exceptions import DiagnosticPipError
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_internal/exceptions.py:13: in <module>
    from pip._vendor.requests.models import Request, Response
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_vendor/requests/__init__.py:135: in <module>
    from . import utils
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_vendor/requests/utils.py:42: in <module>
    DEFAULT_CA_BUNDLE_PATH = certs.where()
/var/lib/awx/venv/awx/lib64/python3.9/site-packages/pip/_vendor/certifi/core.py:50: in where
    _CACERT_CTX = get_path("pip._vendor.certifi", "cacert.pem")
/usr/lib64/python3.9/importlib/resources.py:161: in path
    reader = _get_resource_reader(_get_package(package))
/usr/lib64/python3.9/importlib/resources.py:76: in _get_resource_reader
    spec.loader.get_resource_reader(spec.name))
E   ModuleNotFoundError: No module named 'importlib.readers'
================================================================================== short test summary info ===================================================================================
ERROR broken.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================ 5 warnings, 1 error in 1.15s ================================================================================

Activity

changed the title [-]No module importlib.readers[/-] [+]No module importlib.readers breaks pytest==7.0.0[/+] on Feb 4, 2022
added a commit that references this issue on Feb 4, 2022
tsibley

tsibley commented on Feb 4, 2022

@tsibley

I also just ran into this today and came here to report the same issue.

mdmintz

mdmintz commented on Feb 4, 2022

@mdmintz

Here's the temporary fix for requirements.txt files:

pytest==7.0.0;python_version>="3.6" and python_version<"3.9"
pytest==6.2.5;python_version>="3.9" and python_version<"3.10"
pytest==7.0.0;python_version>="3.10"

From my testing, only Python 3.9 seems to be effected. The improvements of pytest 7.0.0 are so nice, that this temporary workaround may be good enough until the next release is out with the fix. Just because Python 3.9 needs to sit this one out doesn't mean that 3.6, 3.7, 3.8, and 3.10 need to sit out with it. 🙂

changed the title [-]No module importlib.readers breaks pytest==7.0.0[/-] [+]No module importlib.readers breaks pytest==7.0.0 with python 3.9[/+] on Feb 4, 2022
added a commit that references this issue on Feb 4, 2022
tsibley

tsibley commented on Feb 4, 2022

@tsibley

@mdmintz Ah, I also took the same approach of using environment markers in the dependency specs, but used a simpler spec:

pytest; python_version != '3.9'
pytest !=7.0.0; python_version == '3.9'

The hope is that the next release of pytest will fix this bug and thus the spec will be self-healing and allow 3.9 to use 7.0.x. (And CI will break again for 3.9 if it's not fixed in the next release.)

added a commit that references this issue on Feb 4, 2022
90eb17e
added
type: bugproblem that needs to be addressed
type: regressionindicates a problem that was introduced in a release which was working previously
on Feb 4, 2022
added a commit that references this issue on Feb 5, 2022
added a commit that references this issue on Feb 5, 2022

27 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugproblem that needs to be addressedtype: regressionindicates a problem that was introduced in a release which was working previously

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @tsibley@nicoddemus@mdmintz@kdelee

      Issue actions

        No module importlib.readers breaks pytest==7.0.0 with python 3.9 · Issue #9608 · pytest-dev/pytest