Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shell-style wildcard support to 'testpaths' #9897

Merged
merged 1 commit into from May 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS
Expand Up @@ -164,6 +164,7 @@ Jeff Widman
Jenni Rinker
John Eddie Ayson
John Towler
Jon Parise
Jon Sonesen
Jonas Obrist
Jordan Guymon
Expand Down
1 change: 1 addition & 0 deletions changelog/9897.feature.rst
@@ -0,0 +1 @@
Added shell-style wildcard support to ``testpaths``.
2 changes: 2 additions & 0 deletions doc/en/reference/reference.rst
Expand Up @@ -1761,6 +1761,8 @@ passed multiple times. The expected format is ``name=value``. For example::
Sets list of directories that should be searched for tests when
no specific directories, files or test ids are given in the command line when
executing pytest from the :ref:`rootdir <rootdir>` directory.
File system paths may use shell-style wildcards, including the recursive
``**`` pattern.
Useful when all project tests are in a known location to speed up
test collection and to avoid picking up undesired tests by accident.

Expand Down
26 changes: 25 additions & 1 deletion src/_pytest/config/__init__.py
Expand Up @@ -3,6 +3,7 @@
import collections.abc
import copy
import enum
import glob
import inspect
import os
import re
Expand Down Expand Up @@ -899,6 +900,19 @@ class InvocationParams:
dir: Path
"""The directory from which :func:`pytest.main` was invoked."""

class ArgsSource(enum.Enum):
"""Indicates the source of the test arguments.

.. versionadded:: 7.2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a guess at the next version in which this might appear.

"""

#: Command line arguments.
ARGS = enum.auto()
#: Invocation directory.
INCOVATION_DIR = enum.auto()
#: 'testpaths' configuration value.
TESTPATHS = enum.auto()

def __init__(
self,
pluginmanager: PytestPluginManager,
Expand Down Expand Up @@ -1308,15 +1322,25 @@ def parse(self, args: List[str], addopts: bool = True) -> None:
self.hook.pytest_cmdline_preparse(config=self, args=args)
self._parser.after_preparse = True # type: ignore
try:
source = Config.ArgsSource.ARGS
args = self._parser.parse_setoption(
args, self.option, namespace=self.option
)
if not args:
if self.invocation_params.dir == self.rootpath:
args = self.getini("testpaths")
source = Config.ArgsSource.TESTPATHS
testpaths: List[str] = self.getini("testpaths")
if self.known_args_namespace.pyargs:
args = testpaths
else:
args = []
for path in testpaths:
args.extend(sorted(glob.iglob(path, recursive=True)))
if not args:
source = Config.ArgsSource.INCOVATION_DIR
args = [str(self.invocation_params.dir)]
self.args = args
self.args_source = source
except PrintHelp:
pass

Expand Down
4 changes: 2 additions & 2 deletions src/_pytest/terminal.py
Expand Up @@ -728,8 +728,8 @@ def pytest_report_header(self, config: Config) -> List[str]:
if config.inipath:
line += ", configfile: " + bestrelpath(config.rootpath, config.inipath)

testpaths: List[str] = config.getini("testpaths")
if config.invocation_params.dir == config.rootpath and config.args == testpaths:
if config.args_source == Config.ArgsSource.TESTPATHS:
testpaths: List[str] = config.getini("testpaths")
line += ", testpaths: {}".format(", ".join(testpaths))

result = [line]
Expand Down
18 changes: 11 additions & 7 deletions testing/test_collection.py
Expand Up @@ -244,28 +244,32 @@ def test_testpaths_ini(self, pytester: Pytester, monkeypatch: MonkeyPatch) -> No
pytester.makeini(
"""
[pytest]
testpaths = gui uts
testpaths = */tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is a backwards-incompatible change (and thus a problem) if the testpaths with --pyargs doesn't work like before anymore. I don't think this test still tests what it was supposed to (namely, that testpaths are Python modules rather than filesystem paths with --pyargs). See #4405 for context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for flagging that! I think I understand. The testpaths concept is overloaded to both refer to file system locations ("paths") and Python module names, and my change thus far requires testpaths to unconditionally resolve to file system locations.

I think a backwards-compatible solution would apply some additional intelligence to the way I'm processing testpaths. If an entry contains a wildcard, then treat it as a file system path and run it through glob; otherwise, accept it as a literal name that could either be path or a module (to be evaluated by later, existing logic).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe I have this working as intended in the latest revision.

While this comment was associated with the test_testpaths_ini function, it's test_collect_pyargs_with_testpaths that needed to be reverted back to its original form. It specifically tests for the case you mention, and it was previously failing before I added the additional "does this path contain glob-style wildcards?" check.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jparise the real painful problem here is that "testpaths" is in fact only paths, not packages, pyargs was a tacked on hack to also look for tests in modules, its not soundly integrated

i'd love to eventually be able to havea consistent idea of collection roots and test id sources

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RonnyPfannschmidt fortunately, I think we've arrived at a safe middle ground for this particular change, which only applies the glob expansion to paths when --pyargs isn't specified, which should be unambiguous.

"""
)
tmp_path = pytester.path
ensure_file(tmp_path / "env" / "test_1.py").write_text("def test_env(): pass")
ensure_file(tmp_path / "gui" / "test_2.py").write_text("def test_gui(): pass")
ensure_file(tmp_path / "uts" / "test_3.py").write_text("def test_uts(): pass")
ensure_file(tmp_path / "a" / "test_1.py").write_text("def test_a(): pass")
ensure_file(tmp_path / "b" / "tests" / "test_2.py").write_text(
"def test_b(): pass"
)
ensure_file(tmp_path / "c" / "tests" / "test_3.py").write_text(
"def test_c(): pass"
)

# executing from rootdir only tests from `testpaths` directories
# are collected
items, reprec = pytester.inline_genitems("-v")
assert [x.name for x in items] == ["test_gui", "test_uts"]
assert [x.name for x in items] == ["test_b", "test_c"]

# check that explicitly passing directories in the command-line
# collects the tests
for dirname in ("env", "gui", "uts"):
for dirname in ("a", "b", "c"):
items, reprec = pytester.inline_genitems(tmp_path.joinpath(dirname))
assert [x.name for x in items] == ["test_%s" % dirname]

# changing cwd to each subdirectory and running pytest without
# arguments collects the tests in that directory normally
for dirname in ("env", "gui", "uts"):
for dirname in ("a", "b", "c"):
monkeypatch.chdir(pytester.path.joinpath(dirname))
items, reprec = pytester.inline_genitems()
assert [x.name for x in items] == ["test_%s" % dirname]
Expand Down