pytest.raises swallows warning about unawaited coroutine

[seifertm]

Binding the return value of the pytest.raises context manager to a variable prevents some warnings from being triggered as part of the test. As a result, the warning is not displayed in the test run summary, but only on stderr, and -W error will not fail the test.

Reproducible example:

import asyncio
import pytest

async def my_task():
    pass

def test_scheduler_must_be_created_within_running_loop() -> None:
    with pytest.raises(RuntimeError) as _:
        asyncio.create_task(my_task())

Running the above code:

$ pytest -s -W error
===== test session starts =====
platform linux -- Python 3.10.8, pytest-7.1.3, pluggy-1.0.0
rootdir: /tmp/t
collected 1 item                                                                                                                                                                                                                                                             

test_a.py .

===== 1 passed in 0.00s =====
sys:1: RuntimeWarning: coroutine 'my_task' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

When replacing with pytest.raises(RuntimeError) as _: with with pytest.raises(RuntimeError):, everything works as expected and the warning causes the test run to fail.

My understanding is that the coroutine triggers the warning when it gets garbage collected. It looks like with pytest.raises as _ keeps a reference to the coroutine that should trigger the warning. The reference prevents garbage collection of the coroutine until after the hook wrappers in unraisableexception have returned. As a result, the warning appears outside of the test case.

Additional information

$ pip list
Package    Version
---------- -------
attrs      22.1.0
iniconfig  1.1.1
packaging  21.3
pip        22.3
pluggy     1.0.0
py         1.11.0
pyparsing  3.0.9
pytest     7.1.3
setuptools 65.4.1
tomli      2.0.1

Related issues: #8021 #5676

[Zac-HD]

This sounds like a classic "holding the traceback keeps everything in referenced frames alive" problem, which we will want to fix.

That said, there is no guarantee that garbage collection will run at any particular time (or at all!) even if we do fix it, so I don't consider this a serious bug.

[Dreamsorcerer]

That said, there is no guarantee that garbage collection will run at any particular time (or at all!)

Even if you do a gc.collect() at the end of each test run?

[RonnyPfannschmidt]

A next generation raises could return a intentionally cleaned up traveback representation

I'm not Fond of the py.code remnant, but it's quite a bit of work to make it nice

[AIGeneratedUsername]

I wasted some time reading all issues here to find how to fail a test when a coroutine was not awaited. In case if someone will need a solution:

# pytest.ini

filterwarnings=
    error:coroutine .* was never awaited:RuntimeWarning
    # A trick to fail the `PytestUnraisableExceptionWarning: Exception ignored in:` which happens when we catch a `was never awaited` warning
    error:.*Exception ignored in.*::_pytest

[Dreamsorcerer]

I wasted some time reading all issues here to find how to fail a test when a coroutine was not awaited.

That's unrelated to this issue though. We use filterwarnings = error to error on all warnings. The issue here is that a warning occurs outside the actual test run and therefore doesn't cause a test failure.