Design discussion: support higher-scoped (class/module/session level) fixtures?

[oremanj]

I'm pretty sure it's possible if we're willing to use greenlet.

Here's a sketch that's missing important details like exception propagation.

Hook pytest_runtest_loop to keep a Trio event loop running in another greenlet for the whole pytest invocation:

async def _manage_fixture(teardown_evt, done_evt, coroutine_or_agen, *, task_status):
    teardown = False
    if not hasattr(coroutine_or_agen, "asend"):
        # fixture with a return statement
        result = await coroutine_or_agen
    else:
        # fixture with a yield statement
        result = await coroutine_or_agen.asend(None)
        teardown = True
    task_status.started(result)
    if teardown:
        try:
            await coroutine_or_agen.asend(None)
        except StopAsyncIteration:
            pass
        else:
            raise RuntimeError("too many yields in fixture")
    done_evt.set()

async def _pytest_trio_main(pytest_greenlet):
    async with trio.open_nursery() as nursery:
        fixtures = {}
        next_msg = "ready"
        while True:
            request = pytest_greenlet.switch(next_msg)
            if request[0] == "setup":
                fixtures[request[1]] = (teardown_evt, done_evt) = (trio.Event(), trio.Event())
                next_switch = await nursery.start(_manage_fixture, teardown_evt, done_evt, request[1])
            elif request[0] == "teardown":
                teardown_evt, done_evt = fixtures.pop(request[1])
                teardown_evt.set()
                await done_evt.wait()
                next_switch = None
            elif request[0] == "run":
                next_switch = await request[1]
            else:
                assert request[0] == "exit"
                return "done"

@pytest.hookimpl(hookwrapper=True)
def pytest_runtestloop(session):
    pytest_greenlet = greenlet.getcurrent()
    session._pytest_trio_trio_greenlet = pytest_greenlet  # or somewhere better
    trio_greenlet = greenlet.greenlet(trio.run)
    first_send = trio_greenlet.switch(_pytest_trio_main, pytest_greenlet)
    assert first_send == "ready"
    yield
    del session._pytest_trio_trio_greenlet
    assert trio_greenlet.switch("exit") == "done"

Hook fixture setup to run async fixtures in the Trio greenlet:

@pytest.hookimpl(hookwrapper=True)
def pytest_fixture_setup(fixturedef, request):
    wrapped_result = yield
    result = wrapped_result.get_result()
    if not hasattr(result, "__await__") and not hasattr(result, "__aiter__"):
        return
    trio_greenlet = request.session._pytest_trio_trio_greenlet
    true_result = trio_greenlet.switch("setup", result)
    if hasattr(result, "__aiter__"):
        request.addfinalizer(partial(trio_greenlet.switch, "teardown", result))
    wrapped_result.force_result(true_result)
    my_cache_key = fixturedef.cache_key(request)
    fixturedef.cached_result[my_cache_key] = (true_result, my_cache_key, None)

Hook running-the-test to run async tests in the Trio greenlet (not shown).

The details are decidedly nontrivial, but I don't see any showstoppers... thoughts? Maybe this could totally replace pytest-trio's current strategy, but it's not a win across the board, so I'm imagining it more as an alternative mode.

Benefits:

• higher-scoped fixtures!
• can trivially take advantage of pytest's ordering of fixtures; no need for most of the current tricky setup/teardown logic
• could imagine running multiple tests in parallel, since you have the shared event loop all set up -- this would involve setting up multiple greenlets to do pytest_runtest_protocol on different subsets of the session.items

Drawbacks:

• depends on greenlet (though PyPy does support it at least)
• if you don't propagate the exceptions totally right, can expect super confusing tracebacks
• can't use custom clocks/instruments per test
• running fixture setup in parallel is much harder

[touilleMan]

Using the same trio loop with different tests is a very slippy slope (typically with the current nursery fixture we can endup with a test spawning coroutines that will outlive it)

Adding greenlet to the mix is a whole new layer of complexity, pytest-trio is already far from trivial so this seems like another big red flag.

I think the best way to implement higher level fixture is to use them to start a separate trio loop within a thread.
The two isolated trio loops (one for the class/module/session fixture, one for the test) is a double edged sword: you have to do extra work if you want to synchronize the two loop, BUT it also mean you cannot break stuff due to subtle interactions between coroutines.
So I guess we shouldn't even add this feature to the pytest-trio library given the user has to understand the limitations of the class/module/session fixture. The best we could do would be to add a recipe in the documentation.

[tomprince]

I was investigating using pytest-trio to write some integration tests, where I have some expensive services to setup, so want to have session scoped fixtures for the services. I also want to collect logs from the services as they run, and sometimes restart those services from the tests. It is definitely less appealing if I need to manage my own loop for anything session scoped, or that interacts with code from the session scoped things.

In terms of implementation, it looks like how trio is structured to support guest mode could be used to support suspending and resuming the loop. With the current public API, you could have one long-running trio loop, which is what is exposed to users; then, when running a async test or fixture, iterate the host-loop calls (perhaps using a separate trio loop). This would require at least a little glue to capture the guest calls to run_sync_soon_threadsafe and pass them to the current host loop, or batch them until it starts. Or, with access to internals, or a new API, the loop could be suspended after each fixture or callback is completed.

It is definitely true that you can't use the current nursery fixture from non-function scoped fixtures; the same is true of pytest's temp_path fixture, which is why there is the temp_path_factory fixture for use from those.

[svermeulen]

I'm unclear on what the danger is in using the same trio loop for both session and module fixtures in addition to just function fixtures. I don't understand why we couldn't have a session-scope nursery, a module-scope nursery under that, then the function-level nursery under that, then automatically cancel the nurseries when the module is complete and at the end of the session, in the same way the function level nursery is already cancelled. Is it somehow possible for an async method to outlive the nursery it was spawned in?