Skip to content

Commit

Permalink
guide: tidy up doctests
Browse files Browse the repository at this point in the history
  • Loading branch information
davidhewitt committed Mar 4, 2022
1 parent 576818d commit 2bd64c4
Show file tree
Hide file tree
Showing 5 changed files with 87 additions and 45 deletions.
1 change: 1 addition & 0 deletions Cargo.toml
Expand Up @@ -47,6 +47,7 @@ rustversion = "1.0"
# 1.0.0 requires Rust 1.50
proptest = { version = "0.10.1", default-features = false, features = ["std"] }
send_wrapper = "0.5"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.61"

[build-dependencies]
Expand Down
2 changes: 1 addition & 1 deletion guide/src/class/call.md
Expand Up @@ -14,7 +14,7 @@ is linked at the end.

An example crate containing this pyclass can be found [here](https://github.com/PyO3/pyo3/tree/main/examples/decorator)

```rust
```rust,ignore
{{#include ../../../examples/decorator/src/lib.rs}}
```

Expand Down
6 changes: 6 additions & 0 deletions guide/src/features.md
Expand Up @@ -115,6 +115,11 @@ Enables (de)serialization of Py<T> objects via [serde](https://serde.rs/).
This allows to use [`#[derive(Serialize, Deserialize)`](https://serde.rs/derive.html) on structs that hold references to `#[pyclass]` instances

```rust
# #[cfg(feature = "serde")]
# #[allow(dead_code)]
# mod serde_only {
# use pyo3::prelude::*;
# use serde::{Deserialize, Serialize};

#[pyclass]
#[derive(Serialize, Deserialize)]
Expand All @@ -128,4 +133,5 @@ struct User {
username: String,
permissions: Vec<Py<Permission>>
}
# }
```
51 changes: 46 additions & 5 deletions guide/src/memory.md
Expand Up @@ -23,11 +23,16 @@ held. (If PyO3 could not assume this, every PyO3 API would need to take a
very simple and easy-to-understand programs like this:

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
Python::with_gil(|py| -> PyResult<()> {
let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?;
println!("Python says: {}", hello);
Ok(())
})?;
# Ok(())
# }
```

Internally, calling `Python::with_gil()` or `Python::acquire_gil()` creates a
Expand All @@ -39,6 +44,9 @@ it owns are decreased, releasing them to the Python garbage collector. Most
of the time we don't have to think about this, but consider the following:

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
Python::with_gil(|py| -> PyResult<()> {
for _ in 0..10 {
let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?;
Expand All @@ -47,6 +55,8 @@ Python::with_gil(|py| -> PyResult<()> {
// There are 10 copies of `hello` on Python's heap here.
Ok(())
})?;
# Ok(())
# }
```

We might assume that the `hello` variable's memory is freed at the end of each
Expand All @@ -62,20 +72,28 @@ In general we don't want unbounded memory growth during loops! One workaround
is to acquire and release the GIL with each iteration of the loop.

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
for _ in 0..10 {
Python::with_gil(|py| -> PyResult<()> {
let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?;
println!("Python says: {}", hello);
Ok(())
})?; // only one copy of `hello` at a time
}
# Ok(())
# }
```

It might not be practical or performant to acquire and release the GIL so many
times. Another workaround is to work with the `GILPool` object directly, but
this is unsafe.

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
Python::with_gil(|py| -> PyResult<()> {
for _ in 0..10 {
let pool = unsafe { py.new_pool() };
Expand All @@ -85,6 +103,8 @@ Python::with_gil(|py| -> PyResult<()> {
}
Ok(())
})?;
# Ok(())
# }
```

The unsafe method `Python::new_pool` allows you to create a nested `GILPool`
Expand Down Expand Up @@ -112,11 +132,16 @@ What happens to the memory when the last `Py<PyAny>` is dropped and its
reference count reaches zero? It depends whether or not we are holding the GIL.

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
Python::with_gil(|py| -> PyResult<()> {
let hello: Py<PyString> = py.eval("\"Hello World!\"", None, None)?.extract())?;
let hello: Py<PyString> = py.eval("\"Hello World!\"", None, None)?.extract()?;
println!("Python says: {}", hello.as_ref(py));
Ok(())
});
})?;
# Ok(())
# }
```

At the end of the `Python::with_gil()` closure `hello` is dropped, and then the
Expand All @@ -129,8 +154,11 @@ This example wasn't very interesting. We could have just used a GIL-bound
we are *not* holding the GIL?

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
let hello: Py<PyString> = Python::with_gil(|py| {
py.eval("\"Hello World!\"", None, None)?.extract())
py.eval("\"Hello World!\"", None, None)?.extract()
})?;
// Do some stuff...
// Now sometime later in the program we want to access `hello`.
Expand All @@ -142,7 +170,10 @@ drop(hello); // Memory *not* released here.
// Sometime later we need the GIL again for something...
Python::with_gil(|py|
// Memory for `hello` is released here.
# ()
);
# Ok(())
# }
```

When `hello` is dropped *nothing* happens to the pointed-to memory on Python's
Expand All @@ -154,15 +185,20 @@ We can avoid the delay in releasing memory if we are careful to drop the
`Py<Any>` while the GIL is held.

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
let hello: Py<PyString> = Python::with_gil(|py| {
py.eval("\"Hello World!\"", None, None)?.extract())
py.eval("\"Hello World!\"", None, None)?.extract()
})?;
// Do some stuff...
// Now sometime later in the program:
Python::with_gil(|py| {
println!("Python says: {}", hello.as_ref(py));
drop(hello); // Memory released here.
});
# Ok(())
# }
```

We could also have used `Py::into_ref()`, which consumes `self`, instead of
Expand All @@ -172,8 +208,11 @@ that rather than being released immediately, the memory will not be released
until the GIL is dropped.

```rust
# use pyo3::prelude::*;
# use pyo3::types::PyString;
# fn main() -> PyResult<()> {
let hello: Py<PyString> = Python::with_gil(|py| {
py.eval("\"Hello World!\"", None, None)?.extract())
py.eval("\"Hello World!\"", None, None)?.extract()
})?;
// Do some stuff...
// Now sometime later in the program:
Expand All @@ -183,4 +222,6 @@ Python::with_gil(|py| {
// Do more stuff...
// Memory released here at end of `with_gil()` closure.
});
# Ok(())
# }
```
72 changes: 33 additions & 39 deletions src/lib.rs
Expand Up @@ -417,48 +417,42 @@ pub mod doc_test {
};
}

macro_rules! doctest {
($path:expr, $mod:ident) => {
doctest_impl!(include_str!(concat!("../", $path)), $mod);
macro_rules! doctests {
($($path:expr => $mod:ident),* $(,)?) => {
$(doctest_impl!(include_str!(concat!("../", $path)), $mod);)*
};
}

doctest!("README.md", readme_md);
doctest!("guide/src/advanced.md", guide_advanced_md);
doctest!(
"guide/src/building_and_distribution.md",
guide_building_and_distribution_md
);
doctest!("guide/src/class.md", guide_class_md);
doctest!("guide/src/class/protocols.md", guide_class_protocols_md);
doctest!("guide/src/conversions.md", guide_conversions_md);
doctest!(
"guide/src/conversions/tables.md",
guide_conversions_tables_md
);
doctests! {
"README.md" => readme_md,
"guide/src/advanced.md" => guide_advanced_md,
"guide/src/building_and_distribution.md" => guide_building_and_distribution_md,
"guide/src/building_and_distribution/multiple_python_versions.md" => guide_bnd_multiple_python_versions_md,
"guide/src/class.md" => guide_class_md,
"guide/src/class/call.md" => guide_class_call,
"guide/src/class/object.md" => guide_class_object,
"guide/src/class/numeric.md" => guide_class_numeric,
"guide/src/class/protocols.md" => guide_class_protocols_md,
"guide/src/conversions.md" => guide_conversions_md,
"guide/src/conversions/tables.md" => guide_conversions_tables_md,
"guide/src/conversions/traits.md" => guide_conversions_traits_md,
"guide/src/debugging.md" => guide_debugging_md,

doctest!(
"guide/src/conversions/traits.md",
guide_conversions_traits_md
);
doctest!("guide/src/debugging.md", guide_debugging_md);
doctest!("guide/src/exception.md", guide_exception_md);
doctest!("guide/src/function.md", guide_function_md);
doctest!("guide/src/migration.md", guide_migration_md);
doctest!("guide/src/module.md", guide_module_md);
doctest!("guide/src/parallelism.md", guide_parallelism_md);
doctest!("guide/src/python_from_rust.md", guide_python_from_rust_md);
doctest!("guide/src/rust_cpython.md", guide_rust_cpython_md);
doctest!("guide/src/trait_bounds.md", guide_trait_bounds_md);
doctest!("guide/src/types.md", guide_types_md);
doctest!("guide/src/faq.md", faq);
doctest!(
"guide/src/python_typing_hints.md",
guide_python_typing_hints
);
doctest!("guide/src/class/object.md", guide_class_object);
doctest!("guide/src/class/numeric.md", guide_class_numeric);
// deliberate choice not to test guide/ecosystem because those pages depend on external
// crates such as pyo3_asyncio.

// deliberate choice not to test guide/ecosystem because those pages depend on external crates
// such as pyo3_asyncio.
"guide/src/exception.md" => guide_exception_md,
"guide/src/faq.md" => guide_faq_md,
"guide/src/features.md" => guide_features_md,
"guide/src/function.md" => guide_function_md,
"guide/src/memory.md" => guide_memory_md,
"guide/src/migration.md" => guide_migration_md,
"guide/src/module.md" => guide_module_md,
"guide/src/parallelism.md" => guide_parallelism_md,
"guide/src/python_from_rust.md" => guide_python_from_rust_md,
"guide/src/python_typing_hints.md" => guide_python_typing_hints_md,
"guide/src/rust_cpython.md" => guide_rust_cpython_md,
"guide/src/trait_bounds.md" => guide_trait_bounds_md,
"guide/src/types.md" => guide_types_md,
}
}

0 comments on commit 2bd64c4

Please sign in to comment.