pyo3-testing: initial proc_macro to make testing pyo3functions easier

.github/workflows/ci-quick.yml

@@ -0,0 +1,66 @@
+name: Quick CI
+
+# A short version of the CI, designed to run in seconds (<200s) on every push.
+# Runs lints & basic tests on ALL branches EXCEPT main.
+
+on:
+  push:
+    branches-ignore:
+      - main
+
+env:
+  CARGO_TERM_COLOR: always
+
+jobs:
+  fmt:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+      - run: python -m pip install --upgrade pip && pip install nox
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: rustfmt
+      - name: Check python formatting and lints (ruff)
+        run: nox -s ruff
+      - name: Check rust formatting (rustfmt)
+        run: nox -s rustfmt
+
+  semver-checks:
+    if: github.ref != 'refs/heads/main'
+    needs: [fmt]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+      - uses: obi1kenobi/cargo-semver-checks-action@v2
+
+  clippy:
+    needs: [fmt]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy,rust-src
+      - uses: actions/setup-python@v5
+      - uses: Swatinem/rust-cache@v2
+        with:
+          save-if: ${{ github.event_name != 'merge_group' }}
+      - run: python -m pip install --upgrade pip && pip install nox
+      - run: nox -s clippy
+
+  tests:
+    name: test rust skip-full
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+      - uses: Swatinem/rust-cache@v2
+        with:
+          save-if: ${{ github.event_name != 'merge_group' }}
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: rust-src
+      - run: python -m pip install --upgrade pip && pip install nox
+      - run: nox -s test-rust -- skip-full

Cargo.toml

@@ -32,6 +32,9 @@ unindent = { version = "0.2.1", optional = true }
 # support crate for multiple-pymethods feature
 inventory = { version = "0.3.0", optional = true }
 
+# support crate for testing pyo3 wrapped functions
+pyo3-testing = { path = "pyo3-testing", version = "=0.1.0", optional = true}
+
 # crate integrations that can be added using the eponymous features
 anyhow = { version = "1.0", optional = true }
 chrono = { version = "0.4.25", default-features = false, optional = true }
@@ -63,7 +66,7 @@ futures = "0.3.28"
 pyo3-build-config = { path = "pyo3-build-config", version = "=0.21.2", features = ["resolve-config"] }
 
 [features]
-default = ["macros"]
+default = ["macros", "testing"]
 
 # Enables support for `async fn` for `#[pyfunction]` and `#[pymethods]`.
 experimental-async = ["macros", "pyo3-macros/experimental-async"]
@@ -81,6 +84,9 @@ macros = ["pyo3-macros", "indoc", "unindent"]
 # Enables multiple #[pymethods] per #[pyclass]
 multiple-pymethods = ["inventory", "pyo3-macros/multiple-pymethods"]
 
+# Enables #[pyo3test] and #[pyo3import]
+testing = ["pyo3-testing", "macros"]
+
 # Use this feature when building an extension module.
 # It tells the linker to keep the python symbols unresolved,
 # so that the module can also be used with statically linked python interpreters.
@@ -113,6 +119,7 @@ nightly = []
 # This is mostly intended for testing purposes - activating *all* of these isn't particularly useful.
 full = [
     "macros",
+    "testing",
     # "multiple-pymethods", # TODO re-add this when MSRV is greater than 1.62
     "anyhow",
     "chrono",
@@ -137,6 +144,7 @@ members = [
     "pyo3-build-config",
     "pyo3-macros",
     "pyo3-macros-backend",
+    "pyo3-testing",
     "pytests",
     "examples",
 ]

guide/src/SUMMARY.md

@@ -15,6 +15,7 @@
       - [Basic object customization](class/object.md)
       - [Emulating numeric types](class/numeric.md)
       - [Emulating callable objects](class/call.md)
+  - [Testing your code](testing.md)
 - [Calling Python from Rust](python-from-rust.md)
   - [Python object types](types.md)
   - [Python exceptions](exception.md)

guide/src/testing.md

@@ -0,0 +1,216 @@
+# Testing Rust code wrapped for use in Python
+
+Testing that the Rust code you have created works, and works in Python can be simple. PyO3 includes
+some helper macros to make this task easier when coupled with a few good practices.
+
+This chapter of the Guide explains:
+
+- [How to structure your code to make testing easier](#structuring-for-testability)
+- [How to test your functionality](#testing-your-functionality-in-rust)
+- [Testing your wrapping with `#[pyo3test]`](#testing-your-wrapped-functions-in-rust)
+- [Final integration testing in Python](#testing-the-final-integration-in-python)
+- [Compatibility with older Python versions (CI)](#compatibility-with-older-python-versions)
+
+## Structuring for testability
+
+If your code contains anything more than the most basic logic, you will probably want to test that it
+functions correctly. This is best done in the Rust eco-system. Depending on
+
+- whether you want to provide your library for use in rust (via crates.io)
+- the overall complexity of your code base
+
+you have two options:
+
+1. For more complex libraries, or where you wish to provide a rust library as well as your Python
+package: you should create a dedicated crate for your rust library and a second crate for the PyO3
+bindings.
+1. For simpler cases, or where your code is only destined to be used in Python: you should create your
+basic functionality as rust modules and functions, without wrapping them using `[#pyo3...]`
+
+In the first case: you can create both unit- and integration tests as defined and described in
+["The Book"](https://doc.rust-lang.org/stable/book/ch11-00-testing.html) to validate your functionality.
+
+In the second case: you are restricted to "unit tests" within the same source file as the code itself.
+This can be perfectly adequate, as you will test integration with Python later...
+
+For the remainder of this guide we will focus on the second case.
+
+## Testing your functionality in Rust
+
+Comprehensively testing your functionality directly in Rust gives you the fastest test execution and
+makes finding any issues easier, as you know that they can only originate in the underlying Rust functions,
+not in your wrapping, importing or use in Python.
+
+Let's say your library should add one to any integer:
+
+```rust
+fn o3_addone(num: isize) -> isize {
+    num + 1
+}
+```
+
+You can test this in the same file. More details on how to do this are described in
+[Unit tests](https://doc.rust-lang.org/stable/book/ch11-03-test-organization.html#unit-tests)
+in "The Book":
+
+```rust
+fn o3_addone(num: isize) -> isize {
+    num + 1
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_one_plus_one () {
+        let result = o3_addone(1_isize);
+        asserteq!(result, 2_isize)
+    }
+```
+
+## Testing your wrapped functions in Rust
+
+Once you are confident that your functionality is sound, you can wrap it for Python with a simple
+one-liner:
+
+```rust
+#[pyfunction]
+#[pyo3(name = "addone")]
+fn py_addone(num: isize) -> isize {
+    o3_addone(num)
+}
+```
+
+and then create a Python module which can be imported:
+
+```rust
+#[pymodule]
+#[pyo3(name = "adders")]
+fn py_adders(module: &Bound<'_, PyModule>) -> PyResult<()> {
+    module.add_function(wrap_pyfunction!(py_addone, module)?)?;
+    Ok(())
+}
+```
+
+Still in Rust, you can test that the wrapped functionality can be executed by the Python interpreter.
+PyO3 provides the `#[pyo3test]` proc-macro and associated `#[pyo3import(...)]` attribute to make this
+simpler:
+
+```rust
+#[pyo3test]
+#[pyo3import(py_adders: from adders import addone)]
+fn test_one_plus_one_wrapped() {
+    let result: PyResult<isize> = match addone.call1((1_isize,)) {
+        Ok(r) => r.extract(),
+        Err(e) => Err(e),
+    };
+    let result = result.unwrap();
+    let expected_result = 2_isize;
+    assert_eq!(result, expected_result);
+}
+```
+
+`#[pyo3test]` takes care of wrapping the whole test case in `Python::with_gil(|py| {...})` and making
+`addone` available in Rust.
+
+> **Note:** running multiple tests which `#[pyo3import]` the same wrapped module requires _at least python3.9_.
+>
+> This does not affect which systems you can build and release for, only the interpreter used for these tests.
+
+In a non-trivial case, you will likely have Type conversions and Error handling which you wish to
+validate at this point.
+
+## The full example in Rust
+
+The full code then looks like this:
+
+```rust
+use pyo3::prelude::*;
+
+/// Add one to an isize
+fn o3_addone(num: isize) -> isize {
+    num + 1
+}
+
+/// Rust function for use in Python which adds one to a given int
+#[pyfunction]
+#[pyo3(name = "addone")]
+fn py_addone(num: isize) -> isize {
+    o3_addone(num)
+}
+
+/// A module containing various "adders", written in Rust, for use in Python.
+#[pymodule]
+#[pyo3(name = "adders")]
+fn py_adders(module: &Bound<'_, PyModule>) -> PyResult<()> {
+    module.add_function(wrap_pyfunction!(py_addone, module)?)?;
+    Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// Check that the `o3_addone` function correctly adds one to 1_isize
+    #[test]
+    fn test_one_plus_one () {
+        let result = o3_addone(1_isize);
+        asserteq!(result, 2_isize)
+    }
+
+    /// Check that the Python function `adders.addone` can be run in Python
+    #[pyo3test]
+    #[pyo3import(py_adders: from adders import addone)]
+    fn test_one_plus_one_wrapped() {
+        let result: PyResult<isize> = match addone.call1((1_isize,)) {
+            Ok(r) => r.extract(),
+            Err(e) => Err(e),
+        };
+        let result = result.unwrap();
+        let expected_result = 2_isize;
+        assert_eq!(result, expected_result);
+    }
+}
+```
+
+## Testing the final integration in Python
+
+Now that you are confident that your functionality is correct and your wrappings work, you can create
+your final tests in Python, using either pytest or unittest. In this guide we will use pytest for the
+examples.
+
+```python
+from adders import addone
+
+def test_one_plus_one ():
+    assert addone(1) == 2
+```
+
+Before you can execute this test, you need to compile and install your rust library.
+
+The easiest way to do this, with both maturin and setuptools-rust is to run `pip install -e .` in the
+root of your Python package. This will build and install the package in editable mode.
+
+Note: If you have additional dependencies for development, e.g.: pytest, then you will need to install
+these manually, or include them as optional dependencies in `pyproject.toml` e.g.:
+
+```toml
+[project.optional-dependencies]
+dev = [
+    "pytest",
+    ]
+```
+
+and then run `pip install -e .[dev]`
+
+## Compatibility with older Python versions
+
+Due to limitations in older Python interpreters the `#[pyo3test]` macro can only be used with cPython >= 3.9,
+it is also not compatible with PyPy or GraalPy. This is because the macro attempts to (re-)initialise your
+wrapped `PyModule` for each test case and this is not handled well in older versions if done in the same
+interpreter instance.
+
+Your wrapped code can still be built for, and used in, other versions of Python as per standard Pyo3 compatibility.
+You should ensure that any automated CI tasks which run on multiple versions of Python skip these tests where
+applicable and only execute the rust unit tests and python-side integration tests.

newsfragments/4099.added.md

@@ -0,0 +1 @@
+Add `#[pyo3test]` to support easier testing of code wrapped by `#[pyfunction]`. See section 2.4 of the Guide: "Testing your code" for more details.

noxfile.py

@@ -45,6 +45,7 @@ def test_rust(session: nox.Session):
     _run_cargo_test(session, package="pyo3-build-config")
     _run_cargo_test(session, package="pyo3-macros-backend")
     _run_cargo_test(session, package="pyo3-macros")
+    _run_cargo_test(session, package="pyo3-testing")
     _run_cargo_test(session, package="pyo3-ffi")
 
     _run_cargo_test(session)

pyo3-testing/Cargo.toml

@@ -0,0 +1,18 @@
+[package]
+name = "pyo3-testing"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+name = "pyo3_testing"
+path = "src/lib.rs"
+# crate-type = ["rlib"]  # cdylib required for python import, rlib required for rust tests.
+proc-macro = true
+
+[dependencies]
+quote = "1.0.35"
+proc-macro2 = "1.0.74"
+syn = {version = "2.0.55", features = ["full"]}
+
+[lints]
+workspace = true