pytest 7: ModuleNotFoundError: No module named '_pytest.store'

[hroncok]

Hello, in Fedora, we are testing the impact of pytest 7.0.0rc1 on our packages.

black/tests/optional.py

Line 24 in 61fe841

from _pytest.store import StoreKey

Here we get:

+ /usr/bin/pytest -rs
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 732, in import_plugin
    __import__(importspec)
  File "/usr/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 171, in exec_module
    exec(co, module.__dict__)
  File "/builddir/build/BUILD/black-21.11b0/tests/optional.py", line 24, in <module>
    from _pytest.store import StoreKey
ModuleNotFoundError: No module named '_pytest.store'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 188, in console_main
    code = main()
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 146, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 325, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.10/site-packages/_pytest/helpconfig.py", line 102, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/usr/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1002, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1290, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1192, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/usr/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1069, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 515, in _set_initial_conftests
    self._try_load_conftest(current, namespace.importmode, rootpath)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 525, in _try_load_conftest
    self._getconftestmodules(x, importmode, rootpath)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 554, in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 609, in _importconftest
    self.consider_conftest(mod)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 690, in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 453, in register
    self.consider_module(plugin)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 698, in consider_module
    self._import_plugin_specs(getattr(mod, "pytest_plugins", []))
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 705, in _import_plugin_specs
    self.import_plugin(import_spec)
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 734, in import_plugin
    raise ImportError(
  File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 732, in import_plugin
    __import__(importspec)
  File "/usr/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 171, in exec_module
    exec(co, module.__dict__)
  File "/builddir/build/BUILD/black-21.11b0/tests/optional.py", line 24, in <module>
    from _pytest.store import StoreKey
ImportError: Error importing plugin "tests.optional": No module named '_pytest.store'

Since this is a private API, it is not a documented change in pytest changelog. I wonder if a public API that cold be used instead exists.

[bluetech]

It is now pytest.Stash/pytest.StashKey/foo.stash, and is public and stable.

[hroncok]

Awesome, I'll submit a PR.

[hroncok]

Ok, there's more:

___________________ ERROR collecting tests/test_no_ipynb.py ____________________
ImportError while importing test module '/builddir/build/BUILD/black-21.11b0/tests/test_no_ipynb.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_no_ipynb.py:7: in <module>
    from _pytest.tmpdir import tmpdir
E   ImportError: cannot import name 'tmpdir' from '_pytest.tmpdir' (/usr/lib/python3.10/site-packages/_pytest/tmpdir.py)

[hroncok]

I'll try replacing that with tmp_path

[bluetech]

It should be possible to replace all _pytest imports for typing purposes with pytest imports - see pytest-dev/pytest#7469 (except Node which should be replaced with Union[pytest.Item, pytest.Collector]). Unfortunately some of them were only added in pytest 7.

The tmpdir one in particular is incorrect though, probably they meant to use py.path.local. However replacing that with tmp_path is definitely better.

[hroncok]

#2705