use anyio in "core" psycopg #146

All references to asyncio in comments are turned into a more generic 'async libraries' term.

In preparation for adding support for anyio as asynchronous I/O library, this makes it clearer that these functions are asyncio-specific.

This is to support other async libraries than asyncio. Functions fake_resolve() and fail_resolve() in conninfo tests are no longer fixtures monkeypatching asyncio but simply implement a fake getaddrinfo(). On the other hand, fake_resolve() in test_connection_async.py (previously imported from test_conninfo.py) still monkeypatches asyncio as this is an indirect test for resolve_hostaddr_async().

Trio will raise this ResourceWarning: Async generator 'psycopg.cursor_async.AsyncCursor.__aiter__' was garbage collected before it had been e xhausted. Surround its use in 'async with aclosing(...):' to ensure that it gets cleaned up as soon as you're done using it. So we explicitly close async generators when we're partially consuming them (because of a 'break' in the loop).

This adds a new AnyIOConnection class to be used instead of AsyncConnection in combination with the 'anyio' async library. The class essentially uses an anyio.Lock instead of an asyncio.Lock (and same for getaddrinfo()) and relies on AnyIO-specific waiting functions, also introduced here. The same is done for crdb connection class, though with more repetition due to typing issues mentioned in inline comments. All anyio-related code lives in the _anyio sub-package. An 'anyio' setuptools extra is defined to pull required dependencies. AnyIOConnection is exposed on the psycopg namespace, a runtime check is performed when instantiating possibly producing an informative message about missing dependencies. In tests, overall, the previous anyio_backend fixture is now parametrized with both asyncio and trio backends and 'aconn_cls' returns either AsyncConnection or AnyIOConnection depending on backend name. Test dependencies now include 'anyio[trio]'. In "waiting" tests, we define 'wait_{conn_,}_async' fixtures that will pick either asyncio or anyio waiting functions depending on the value of 'anyio_backend' fixture. Concurrency tests (e.g. test_concurrency_async.py or respective crdb ones) are not run with the trio backend as then explicitly use asyncio API. Porting them does not seem strictly needed, at least now. So they get marked with asyncio_backend. Finally, we ignore an invalid error (raised by a deprecation warning during test) about usage of the 'loop' parameter in asyncio API that is due to Python bug as mentioned in comment.

We add a --anyio={asyncio,trio} option to pytest to select the AnyIO backend to run async tests. When this option is set, we'll use the AnyIO implementations of psycopg API (i.e. AnyIOConnection, resp. waiting functions, etc.); otherwise (the default), we'll use plain asyncio implementations (i.e. AsyncConnection). This way, we can now run the test suite with: - plain asyncio implementations (previously achieved through the asyncio backend for AnyIO pytest plugin), - AnyIO implementations, with asyncio backend (new from this commit), - AnyIO implementations, with trio backend (previously achieved through the Trio backend for AnyIO pytest plugin). Accordingly, the anyio_backend fixture is no longer parametrized and its value depends on the --anyio option. Selection of whether to use AnyIO or plain asyncio implementations is done through the new 'use_anyio' fixture, which simply detects if no --anyio option got specified. This new fixture is used everywhere we used anyio_backend_name fixture previously to select either plain asyncio or anyio variants of test cases. The fixture pulls (but does not use) anyio_backend so that all tests using it will be detected as async. In CI, we add a new pytest_opts axis to the matrix to cover all configurations in various environments.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use anyio in "core" psycopg #146

use anyio in "core" psycopg #146

Commits on Feb 4, 2023