pytest 7.2.0: ModuleNotFoundError: No module named 'py.xml'; 'py' is not a package

[KevinMarquette]

When running pytest 7.2.0, I am getting this error (output truncated):

> pytest --version
Traceback (most recent call last):
  File ".venv/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File ".venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 190, in console_main
    code = main()
    ...
  File ".venv/lib/python3.9/site-packages/pytest_html/html_report.py", line 11, in <module>
    from py.xml import html
  ModuleNotFoundError: No module named 'py.xml'; 'py' is not a package

If I revert back to pytest 7.1.3, the error goes away.

I have these testing related modules in my requirements.txt (full pip list below):

pytest==7.2.0
pytest-cov
pytest-html
pytest-mock
pytest-run-changed
pytest-subprocess
pytest-testmon
coverage
coverage[toml]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using (see below)
• pytest and operating system versions
  • base container image: ubuntu:20.04
  • pytest 7.2.0
• minimal example if possible (working on it)

Possibly related to:

• vendor py.path and py.error #10396
• Remove dependency on py pytest-html#350

pip list:

Package                    Version
-------------------------- --------------------
astroid                    2.12.11
attrs                      22.1.0
azure-common               1.1.28
azure-core                 1.26.0
azure-eventgrid            4.9.0
azure-functions            1.12.0
azure-identity             1.11.0
azure-keyvault-secrets     4.6.0
azure-storage-blob         12.14.1
azure-storage-queue        12.5.0
bandit                     1.7.4
black                      22.10.0
certifi                    2019.11.28
cffi                       1.15.1
cfgv                       3.3.1
chardet                    3.0.4
charset-normalizer         2.1.1
click                      8.1.3
colorama                   0.4.5
coverage                   6.5.0
cryptography               38.0.1
dbus-python                1.2.16
defusedxml                 0.7.1
dill                       0.3.5.1
distlib                    0.3.6
distro-info                0.23ubuntu1
docformatter               1.5.0
exceptiongroup             1.0.0rc9
filelock                   3.8.0
flake8                     5.0.4
ghp-import                 2.1.0
gitdb                      4.0.9
GitPython                  3.1.29
griffe                     0.22.2
identify                   2.5.6
idna                       2.8
importlib-metadata         5.0.0
iniconfig                  1.1.1
isodate                    0.6.1
isort                      5.10.1
Jinja2                     3.1.2
lazy-object-proxy          1.7.1
Markdown                   3.3.7
MarkupSafe                 2.1.1
mccabe                     0.7.0
mergedeep                  1.3.4
mkdocs                     1.4.1
mkdocs-autorefs            0.4.1
mkdocs-gen-files           0.4.0
mkdocs-literate-nav        0.5.0
mkdocs-material            8.5.7
mkdocs-material-extensions 1.1
mkdocstrings               0.19.0
mkdocstrings-python        0.7.1
msal                       1.20.0
msal-extensions            1.0.0
msrest                     0.7.1
mypy                       0.982
mypy-extensions            0.4.3
nodeenv                    1.7.0
oauthlib                   3.2.2
packaging                  21.3
pathspec                   0.10.1
pbr                        5.10.0
pip                        22.0.4
platformdirs               2.5.2
pluggy                     1.0.0
portalocker                2.6.0
pre-commit                 2.20.0
py                         1.11.0
pycodestyle                2.9.1
pycparser                  2.21
pyflakes                   2.5.0
Pygments                   2.13.0
PyGObject                  3.36.0
PyJWT                      2.6.0
pylint                     2.15.4
pymdown-extensions         9.7
pyparsing                  3.0.9
pytest                     7.2.0
pytest-cov                 4.0.0
pytest-html                3.2.0
pytest-metadata            2.0.2
pytest-mock                3.10.0
pytest-run-changed         1.0.1
pytest-subprocess          1.4.2
pytest-testmon             1.4.0
python-apt                 2.0.0+ubuntu0.20.4.7
python-dateutil            2.8.2
pyupgrade                  3.1.0
PyYAML                     6.0
pyyaml_env_tag             0.1
requests                   2.28.1
requests-oauthlib          1.3.1
requests-unixsocket        0.2.0
setuptools                 65.5.0
six                        1.14.0
smmap                      5.0.0
stevedore                  4.0.1
tokenize-rt                5.0.0
toml                       0.10.2
tomli                      2.0.1
tomlkit                    0.11.5
types-PyYAML               6.0.12
types-requests             2.28.11.2
types-urllib3              1.26.25.1
typing_extensions          4.4.0
unattended-upgrades        0.1
untokenize                 0.1.1
urllib3                    1.25.8
virtualenv                 20.16.5
watchdog                   2.1.9
wheel                      0.34.2
wrapt                      1.14.1
zipp                       3.10.0

The pip list difference from the working and not working is this:

pytest                     7.1.3
exceptiongroup             <not installed>

I first noticed this error in my pipelines where I don't have the version pinned.

[The-Compiler]

See pytest-dev/pytest-html#555 and #10420 for some context. Not a pytest issue.

[nicoddemus]

Pinning this too, as this mentions py.xml so might be more visible for users which encounter this exact message.

[amezin]

I have py library installed, but I'm still getting this error.

py is installed in system site-packages, pytest and pytest-html - in a virtualenv (with system site packages enabled).
Seems that in this case py module from pytest takes precedence over pylib from system site packages (and breaks pytest-html).

https://github.com/ddterm/gnome-shell-extension-ddterm/actions/runs/3330468879/jobs/5509011844

[The-Compiler]

@amezin I suggest you open a new issue, since this will get lost here otherwise - that one does seem like a pytest issue on first glance.

[aws-taylor]

Note that pytest relies on the fact that <directory>/py/__init__.py takes precedence over <directory>/py.py, so that if both pytest and py are installed, the py version wins. This behavior is specified in https://peps.python.org/pep-0420/#specification

A potential problem with this approach is that it assumes that both pytest and py are installed under the same <directory>. Since this is the case for 99% of all people, it's a reasonable assumption, but if for some reason you have pytest and py installed under different site-packages then you'll still get the above error depending on the order in which pytest and py appear on PYTHONPATH.

Minimal repro:

> python3 -m venv venv-for-pytest
> source venv-for-pytest/bin/activate
> pip install pytest
> deactivate
> python3 -m venv venv-for-py
> source venv-for-py/bin/activate
> pip install py
> deactivate
> env PYTHONPATH=venv-for-py/lib/python3.7/site-packages/:venv-for-pytest/lib/python3.7/site-packages/ python3 -c "from py.xml import html"
> echo $?
0
> env PYTHONPATH=venv-for-pytest/lib/python3.7/site-packages/:venv-for-py/lib/python3.7/site-packages/ python3 -c "from py.xml import html"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'py.xml'; 'py' is not a package
> echo $?
1