Should pytest fixtures stop being dependency injected?

[ssbarnea]

As we know, part of pytest design was to magically discover/load fixtures and make them available to tests, so test writer would not have to remember to import them.

That magic used was dependency injection and it was made very long time ago, far before python typing was introduced. Sadly,it does not play very well with typing because runtime checkers like mypy cannot really infer the types for test method signatures. Lack of it causes confusiones and usually breaks auto-complete in IDEs.

Inside #5981 it was suggested that maybe we should reconsider this architecture decision, so we do not cripple the typing support.

Please note that this poll result does not mean it would be acten on, think about it as a survey.

[RonnyPfannschmidt]

I use explicit type annotation just fine, jedi and a number of ides also have basic support for inferring

[The-Compiler]

Getting rid of the existing fixture mechanism is a non-starter, IMHO - it would be a major rewrite for anything using pytest. That massive code churn is hard to justify, and not going to be liked. Also, not everyone likes using types (I do!).

One thing I had in the back of my head for a while is class fixtures. In a nutshell:

• Allow @pytest.fixture on a class (with an __init__ taking other fixtures, or no arguments)
• Such fixtures are not auto-discovered like normal fixtures
• Instead, it gets injected whenever there's a type annotation (with associated import) for that class
• tbd: How to do teardown and such

This would be backwards compatible, but allow using imports and type annotations for fixtures where desired.

Probably should see how hard it is to implement and/or cook up a more detailed proposal at some point...

[RonnyPfannschmidt]

I'd much rather see nicely typed context managers than a new protocol

[nicoddemus]

I'd much rather see nicely typed context managers than a new protocol

Something like:

@contextmanager
def session_fixture(url: str):
   result = Session.connect()
   yield result
   result.disconnect()

def pytest_fixtures():
   return {"session": session_fixture}      

?

I see, but then it is basically what we have already with @pytest.fixture, except it is not auto discovered (the pytest_fixtures provides the fixtures explicitly).

But yeah, redesigning the fixture system to make it more flexible so plugins can customize it is a different and more complex topic.

[The-Compiler]

@nicoddemus Having to define a name defeats the whole purpose, in my eyes: Not having to refer to fixtures by some kind of "magical" global name, but importing a type from somewhere (nicely namespaced) and using the fixture by having an argument which is annotated by that type.

As for teardown, that would indeed be some kind of custom protocol. We could have some kind of pytest_teardown method called, for example. Or we could use __enter__ and __exit__ perhaps (i.e. making class fixtures context managers), which might or might not be what @RonnyPfannschmidt meant.

As for encouraging users to use @pytest.fixture in production code: Fair, but people could already do all kinds of pytest specific things in their production code (e.g. a pytest plugin). I don't think it's our job to prevent that.

[nicoddemus]

@nicoddemus Having to define a name defeats the whole purpose, in my eyes: Not having to refer to fixtures by some kind of "magical" global name, but importing a type from somewhere (nicely namespaced) and using the fixture by having an argument which is annotated by that type.

Fair. 👍

[RonnyPfannschmidt]

I think we need a Mindmap and a Design document, I'm under the impression our understanding and direction for those ideas display a level of diversity that dooms it unless we converge and experiment

We also should take a look at libraries like di and consider ways in which to bundle correlated fixtures in a consistent manner (failsafe for details of pytest selenium)

[nicoddemus]

Sadly,it does not play very well with typing because runtime checkers like mypy cannot really infer the types for test method signatures.

Not sure I follow this statement... mypy does not infer types for signatures AFAIK, if you don't add a type hint it is considered Any, not inferred automatically.

Lack of it causes confusiones and usually breaks auto-complete in IDEs.

IDEs (well at least PyCharm) infer the type of the fixtures in tests (because they have specific support for pytest). I believe VSCode does too.

Could you elaborate on both points please? I think it is important to clarify them because someone might "fly by" over the pool and leave with the impression that they can't use type annotations with pytest, or they are incompatible somehow (which is not true).

[ssbarnea]

Lets pick:

def test_foo(tmp_dir):
    tmp_dir.xxx = "x"

The IDE will not be able to tell you that .xxx does not exist on tmp_dir, because it assumes Any. As tmp_dir does not have a type, user needs to add it. Only when he adds the full signature, he will be able to get auto-complete to work and also the useful hint that .xxx is not really there.

Basically until you add types, you cannot really get useful help from tooling.

As most of our documentation examples do not have types, people copy/paste those and wonder what is that object. Is that string, is that a Path? (no need to answer me that, just trying to explain it).

Sadly I do not know what we can do to improve the situation other than advertising code using types almost everywhere. I am more than curious to see if someone comes with better ideas.

[nicoddemus]

Basically until you add types, you cannot really get useful help from tooling.

Oh I agree, but that's the same as with any other library: you will get the same problem if you don't add types to a function which receives a numpy array or a pandas data-frame. This is not pytest-specific, like the wording in the original post suggests.

The solution here is to just add the type to the fixture parameter:

def test_foo(tmp_dir: Path):
    tmp_dir.xxx = "x"

But I'm sure you know that.

Seems like you want (and sorry if I'm wrong) to change pytest to force users to use types, which I think is a non-starter, for the reasons @The-Compiler outlined.

What we can improve, I think, is having a dedicated page in the docs like "How to use types with pytest" or something, showing examples and explaining the current situation.

[nicoddemus]

Just to be clear: I love types and add types everywhere I can (including to tests in my recent projects), but I certainly don't think pytest should force everyone to use types.

[JacobCallahan]

This is a hard pass from me. One of the big advantages pytest offers over other test platforms is the "magic" of fixture dependency injection. Changing that behavior, to require explicit imports in order to benefit the subsection of users that also use types, would likely keep most users from upgrading to that pytest version.

I think a better way would be to recommend specific rules for fixture organization in a project. IDEs and tools like mypy could then make a best effort to discover fixtures based on these rules. This is the approach I'm taking with a project I'm working on that also needs to solve the fixture resolution problem.

[tgross35]

Out of curiosity - what would the imported version look like? The current "magic" way of course can't be removed: that being said, I'm in the camp that magic isn't real and providing an alternative certainly wouldn't be bad.

I haven't seen a concrete list of reasons (goals) that an import-related syntax might be preferred but here are a couple things I can think of:

• IDE support out of the box, rather than using plugins
• Static checkers can understand what's going on better (there is definitely value in running mypy on tests)
• (arguably) better code readability
• Potential namespace / scope benefits (e.g. easier fixture definition in different files, easy to find which fixture is used)

However, I'm having a bit of a tough time imagining how this might look practically. Importing the fixtures directly to use them has the slight disadvantage that property-like objects now need to be called as functions (I think, unless pytest can change the function definitions to objects. But that sounds messy for typing). Another downside is confusing during migration - global imported callable fixtures would have the same name as kwargs.

One random brainstorming thought was that @pytest.fixture could register fixtures as properties to a global importable class, maybe something like below. I actually like the appearance a lot and keeping fixtures in a fx namespace would help avoid backward-compatibility confusion (like above), as well as avoiding circular import issues or our-of-order definitions. This might not easily resolve the goals of IDE support and correct typing though, not sure the state of support for dynamically added objects.

# Current syntax
# my_fixture in conftest.py, my_local_fixture in this file

def test_something(my_fixture: str, my_local_fixture: int):
    assert my_fixture == "abcd"
    assert my_local_fixture == 4

# Potential new syntax
# Same fixture definitions as above assumed
# Plus nonlocal_fixture defined in tests/otherfile.py

from pytest import fx

# decorator optional, something to indicate can't use fixtures as arguments
@pytest.mark.strict_fixtures
def test_something():
    fix_str = fx.my_fixture. # property-like syntax possible
    fix_int = fx.my_local_fixture
    fix_other = fx.otherfile.nonlocal_fixture # Nonlocal fixtures accessible
    assert fix_str == "abcd"
    ...

Interested to see what the original idea looked like

[RonnyPfannschmidt]

@tgross35 any api for explicit fixtures would involve a way to sensibly do all fixture setup at call time, your example completely sabotages that

[tgross35]

@RonnyPfannschmidt Are you referring to having the fixture set up before entering the test_ function?

In any case, I'm mostly curious to see how the original direct import idea looked since I can't figure out how that would work.

[The-Compiler]

The original proposal in #5981 didn't propose imports for fixtures at all. I have a proposal in #9946 (comment) which would use classes for fixtures and type annotations, which I believe would work nicely with imports rather than a global(-ish) fixture namespace.

[ytimenkov]

First, I think there is nothing wrong with injecting fixtures as a parameters and otherwise it would be difficult to manage scopes (i.e. how to express session-scoped fixture?) with ordinary imports. The most important is to have type annotation for function parameter - how fixture can be used. This is pretty much covered in #9947.

It also looks like the discussion is between using type or parameter name as a key for injection. Right now pytest uses parameter name and type is ignored completely.

However, both type and parameter name are part of function interface (if we think about a test as an ordinary API function) - you can't change either freely without breaking users. And it doesn't really matter where pytest takes the actual values to be passed as parameters to the test.

There are other alternatives, each has pros and cons.

1. Use only type. For example, opyoid. Only declaring type and then specifying what provides instances of this type allows to name parameters freely and have, for example, 2 instances of the same type/class - e.g. if I need 2 temporary directories or DB connections.
2. Use both to some extent. This is how typer uses to inject command-line options. This is somewhat close to the proposed pytest + annotations, but leaves the name to be an external contract.

[burnpanck]

While this post seems to have some age, as it is still open, I would like to point out that the magic dependency injection also has significant downsides. As with any magic operating on a global namespace, it fails to respect modularity principles, which makes composition hard. The fact that typing tools have a problem with that is just one manifestation. The current design makes that namespace as global as it just could be, which has led to hard-to-debug bugs in the past (see e.g. #3966) and has led to the request of alternate ways of pulling fixtures (see e.g. #3834). Thus, anything that follows the rules of python should be preferred over domain-specific magic.

[nicoddemus]

I think while the discussion here is interesting, we can close this as pytest will not implement such a breaking change.

We are trying to improve the internal so plugins might be able to change and add to the existing fixture mechanism, so alternatives can be implemented outside the core.

Thanks everyone!

[RonnyPfannschmidt]

theres many ways to play this out - ultimately i'd love to see a way where fixture choice is optionally strictly controlled by annotations

when the fixture system was first invented, type annotations where not even in the planning
now a decade later type annotations are a very different story and building block

i have a rough idea for the necessary baseline refactoring to enable such usage (most of them will not add the capability)
i hope we ca line those out in the sprint

[burnpanck]

ultimately i'd love to see a way where fixture choice is optionally strictly controlled by annotations

That is exactly what I'm looking for. I don't care if the annotation is a decorator or a type annotation, but I would like to be able to follow variables, declarations and imports from the actual point where it is attached to some test code all the way to the implementation of the fixture, without having to consider a global scope of potential dependency injectors. After all, PEP-20 says "Explicit is better than implicit.".

[JacobCallahan]

Regarding plugins changing fixture behavior, plugin authors can strongly recommend a supported usage pattern for the plugin without the ability to turn off native pytest behavior. The native behavior can just exist and not be used if the plugin's users determine the plugin satisfies all their needs. There isn't a need to take a paternalistic position against your own users.

---

Regarding "Explicit is better than implicit", keep in mind the lines before and after:

• "Beautiful is better than ugly."
• "Simple is better than complex."

In my (very subjective) opinion, the current fixture pattern is both beautiful and simple (in most cases).

[RonnyPfannschmidt]

Indeed, it's just that in the last decade a different way to do it emerged and we need to start exploring and navigating it

[burnpanck]

The native behavior can just exist and not be used if the plugin's users determine the plugin satisfies all their needs. There isn't a need to take a paternalistic position against your own users.

This seems to miss the point. It's not the plugin author who wants to "forbid" their uses to use any other behaviour, but the plugin user who intends to select a specific fixture (using that plugin), without having control over the full codebase. The (current) native pytest behaviour means that changes to completely unrelated code may suddenly start to inject fixtures, without either author intending it. See #3966. Say you develop an application using two libraries "A" and "B". You test your application using testing helpers from both libraries (say, "A" is a networking library and you use their mock fixtures for the server, "B" is a GUI library and you use mock fixtures representing the GUI-user). "A" uses the new shiny plugin in their fixture, in the hope to achieve modularisation. "B" uses name-matching magic. At some point in time, both now have an internal fixture namedtest_client. Neither of them fails any tests, but you, as the application developer, who has no knowledge of the internal details of either library, now all of a sudden have failing tests, because the fixtures of "B" accidentally slip into "A", despite "A" using the new shiny plugin without name matching. Of, course, that depends on how that plugin interacts with pytest to achieve its goal, and I wanted to drive attention to that point.

I admit, name-matched fixtures are simple to write, and beautiful as long as you see through all involved code, globally. But they are extremely hard to read if the fixture definition is not in the same file, they don't provide any form of namespaceing, leading to ambiguity among which fixture is meant and break the rules of python name resolution. On the other hand, importing a fixture and applying it using either a decorator or an annotation to attach it to a test is also beautiful, and still simple enough IMHO.