Merge branch 'callback' of https://github.com/mejrs/pyo3 into callback

Architecture.md

@@ -135,12 +135,12 @@ to ensure Rust's borrow rules.
 See [the documentation](https://docs.rs/pyo3/latest/pyo3/pycell/struct.PyCell.html) for more.
 
 `PyCell<T>` requires that `T` implements `PyClass`.
-This trait is somewhat complex and derives many traits, but the most important one is `PyTypeObject`
+This trait is somewhat complex and derives many traits, but the most important one is `PyTypeInfo`
 in [`src/type_object.rs`].
-`PyTypeObject` is also implemented for built-in types.
+`PyTypeInfo` is also implemented for built-in types.
 In Python, all objects have their types, and types are also objects of `type`.
 For example, you can see `type({})` shows `dict` and `type(type({}))` shows `type` in Python REPL.
-`T: PyTypeObject` implies that `T` has a corresponding type object.
+`T: PyTypeInfo` implies that `T` has a corresponding type object.
 
 ## 4. Protocol methods
 
@@ -153,8 +153,8 @@ implemented in Python, such as GC support.
 
 ## 5. Procedural macros to simplify usage for users.
 
-[`pyo3-macros`] provides six proc-macro APIs: `pymodule`, `pyproto`, `pyfunction`, `pyclass`,
-`pymethods`, and `#[derive(FromPyObject)]`.
+[`pyo3-macros`] provides five proc-macro APIs: `pymodule`, `pyfunction`, `pyclass`,
+`pymethods`, and `#[derive(FromPyObject)]`. (And a deprecated `pyproto` macro.)
 [`pyo3-macros-backend`] has the actual implementations of these APIs.
 [`src/derive_utils.rs`] contains some utilities used in code generated by these proc-macros,
 such as parsing function arguments.

CHANGELOG.md

@@ -6,12 +6,18 @@ PyO3 versions, please see the [migration guide](https://pyo3.rs/latest/migration
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## Unreleased
+## [Unreleased]
 
 ### Added
 
 - Implement `ToPyObject` for `[T; N]`. [#2313](https://github.com/PyO3/pyo3/pull/2313)
 
+### Changed
+
+- Move `PyTypeObject::type_object` method to `PyTypeInfo` trait, and deprecate `PyTypeObject` trait. [#2287](https://github.com/PyO3/pyo3/pull/2287)
+- The deprecated `pyproto` feature is now disabled by default. [#2322](https://github.com/PyO3/pyo3/pull/2322)
+- Deprecate `ToBorrowedObject` trait (it is only used as a wrapper for `ToPyObject`). [#2333](https://github.com/PyO3/pyo3/pull/2333)
+
 ## [0.16.4] - 2022-04-14
 
 ### Added
@@ -399,7 +405,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 
 - Remove FFI definition `PyCFunction_ClearFreeList` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425)
-- `PYO3_CROSS_LIB_DIR` enviroment variable no long required when compiling for x86-64 Python from macOS arm64 and reverse. [#1428](https://github.com/PyO3/pyo3/pull/1428)
+- `PYO3_CROSS_LIB_DIR` environment variable no long required when compiling for x86-64 Python from macOS arm64 and reverse. [#1428](https://github.com/PyO3/pyo3/pull/1428)
 - Fix FFI definition `_PyEval_RequestCodeExtraIndex`, which took an argument of the wrong type. [#1429](https://github.com/PyO3/pyo3/pull/1429)
 - Fix FFI definition `PyIndex_Check` missing with the `abi3` feature. [#1436](https://github.com/PyO3/pyo3/pull/1436)
 - Fix incorrect `TypeError` raised when keyword-only argument passed along with a positional argument in `*args`. [#1440](https://github.com/PyO3/pyo3/pull/1440)
@@ -408,7 +414,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - No longer include `__doc__` in `__all__` generated for `#[pymodule]`. [#1509](https://github.com/PyO3/pyo3/pull/1509)
 - Always use cross-compiling configuration if any of the `PYO3_CROSS` family of environment variables are set. [#1514](https://github.com/PyO3/pyo3/pull/1514)
 - Support `EnvironmentError`, `IOError`, and `WindowsError` on PyPy. [#1533](https://github.com/PyO3/pyo3/pull/1533)
-- Fix unneccessary rebuilds when cycling between `cargo check` and `cargo clippy` in a Python virtualenv. [#1557](https://github.com/PyO3/pyo3/pull/1557)
+- Fix unnecessary rebuilds when cycling between `cargo check` and `cargo clippy` in a Python virtualenv. [#1557](https://github.com/PyO3/pyo3/pull/1557)
 - Fix segfault when dereferencing `ffi::PyDateTimeAPI` without the GIL. [#1563](https://github.com/PyO3/pyo3/pull/1563)
 - Fix memory leak in `FromPyObject` implementations for `u128` and `i128`. [#1638](https://github.com/PyO3/pyo3/pull/1638)
 - Fix `#[pyclass(extends=PyDict)]` leaking the dict contents on drop. [#1657](https://github.com/PyO3/pyo3/pull/1657)
@@ -992,7 +998,7 @@ Yanked
 ### Changed
 
 - Removes the types from the root module and the prelude. They now live in `pyo3::types` instead.
-- All exceptions are consturcted with `py_err` instead of `new`, as they return `PyErr` and not `Self`.
+- All exceptions are constructed with `py_err` instead of `new`, as they return `PyErr` and not `Self`.
 - `as_mut` and friends take and `&mut self` instead of `&self`
 - `ObjectProtocol::call` now takes an `Option<&PyDict>` for the kwargs instead of an `IntoPyDictPointer`.
 - `IntoPyDictPointer` was replace by `IntoPyDict` which doesn't convert `PyDict` itself anymore and returns a `PyDict` instead of `*mut PyObject`.
@@ -1162,7 +1168,7 @@ Yanked
 - Initial release
 
 [Unreleased]: https://github.com/pyo3/pyo3/compare/v0.16.4...HEAD
-[0.16.3]: https://github.com/pyo3/pyo3/compare/v0.16.3...v0.16.4
+[0.16.4]: https://github.com/pyo3/pyo3/compare/v0.16.3...v0.16.4
 [0.16.3]: https://github.com/pyo3/pyo3/compare/v0.16.2...v0.16.3
 [0.16.2]: https://github.com/pyo3/pyo3/compare/v0.16.1...v0.16.2
 [0.16.1]: https://github.com/pyo3/pyo3/compare/v0.16.0...v0.16.1

Cargo.toml

@@ -18,7 +18,7 @@ cfg-if = "1.0"
 libc = "0.2.62"
 parking_lot = ">= 0.11, < 0.13"
 
-# ffi bindings to the python interpreter, split into a seperate crate so they can be used independently
+# ffi bindings to the python interpreter, split into a separate crate so they can be used independently
 pyo3-ffi = { path = "pyo3-ffi", version = "=0.16.4" }
 
 # support crates for macros feature
@@ -53,7 +53,7 @@ serde_json = "1.0.61"
 pyo3-build-config = { path = "pyo3-build-config", version = "0.16.4", features = ["resolve-config"] }
 
 [features]
-default = ["macros", "pyproto"]
+default = ["macros"]
 
 # Enables macros: #[pyclass], #[pymodule], #[pyfunction] etc.
 macros = ["pyo3-macros", "indoc", "unindent"]

guide/src/class.md

@@ -52,7 +52,7 @@ enum MyEnum {
 
 Because Python objects are freely shared between threads by the Python interpreter, all types annotated with `#[pyclass]` must implement `Send` (unless annotated with [`#[pyclass(unsendable)]`](#customizing-the-class)).
 
-The above example generates implementations for [`PyTypeInfo`], [`PyTypeObject`], and [`PyClass`] for `MyClass` and `MyEnum`. To see these generated implementations, refer to the [implementation details](#implementation-details) at the end of this chapter.
+The above example generates implementations for [`PyTypeInfo`] and [`PyClass`] for `MyClass` and `MyEnum`. To see these generated implementations, refer to the [implementation details](#implementation-details) at the end of this chapter.
 
 ## Constructor
 
@@ -880,7 +880,7 @@ Python::with_gil(|py| {
 })
 ```
 
-All methods defined by PyO3 can be overriden. For example here's how you override `__repr__`:
+All methods defined by PyO3 can be overridden. For example here's how you override `__repr__`:
 
 ```rust
 # use pyo3::prelude::*;
@@ -940,19 +940,14 @@ The `#[pyclass]` macro expands to roughly the code seen below. The `PyClassImplC
 # #[cfg(not(feature = "multiple-pymethods"))] {
 # use pyo3::prelude::*;
 // Note: the implementation differs slightly with the `multiple-pymethods` feature enabled.
-
-/// Class for demonstration
 struct MyClass {
     # #[allow(dead_code)]
     num: i32,
 }
-
-unsafe impl pyo3::PyTypeInfo for MyClass {
-    type AsRefTarget = PyCell<Self>;
-
+unsafe impl ::pyo3::type_object::PyTypeInfo for MyClass {
+    type AsRefTarget = ::pyo3::PyCell<Self>;
     const NAME: &'static str = "MyClass";
-    const MODULE: Option<&'static str> = None;
-
+    const MODULE: ::std::option::Option<&'static str> = ::std::option::Option::None;
     #[inline]
     fn type_object_raw(py: pyo3::Python<'_>) -> *mut pyo3::ffi::PyTypeObject {
         use pyo3::type_object::LazyStaticType;
@@ -961,10 +956,14 @@ unsafe impl pyo3::PyTypeInfo for MyClass {
     }
 }
 
-impl pyo3::pyclass::PyClass for MyClass {
-    type Dict = pyo3::impl_::pyclass::PyClassDummySlot;
-    type WeakRef = pyo3::impl_::pyclass::PyClassDummySlot;
-    type BaseNativeType = PyAny;
+impl ::pyo3::PyClass for MyClass { }
+
+impl<'a> ::pyo3::derive_utils::ExtractExt<'a> for &'a mut MyClass {
+    type Target = ::pyo3::PyRefMut<'a, MyClass>;
+}
+
+impl<'a> ::pyo3::derive_utils::ExtractExt<'a> for &'a MyClass {
+    type Target = ::pyo3::PyRef<'a, MyClass>;
 }
 
 impl pyo3::IntoPy<PyObject> for MyClass {
@@ -980,6 +979,11 @@ impl pyo3::impl_::pyclass::PyClassImpl for MyClass {
     type Layout = PyCell<MyClass>;
     type BaseType = PyAny;
     type ThreadChecker = pyo3::impl_::pyclass::ThreadCheckerStub<MyClass>;
+    type Mutability = pyo3::pycell::Mutable;
+    type PyClassMutability = pyo3::pycell::MutableClass;
+    type Dict = ::pyo3::impl_::pyclass::PyClassDummySlot;
+    type WeakRef = ::pyo3::impl_::pyclass::PyClassDummySlot;
+    type BaseNativeType = ::pyo3::PyAny;
 
     fn for_all_items(visitor: &mut dyn FnMut(&pyo3::impl_::pyclass::PyClassItems)) {
         use pyo3::impl_::pyclass::*;
@@ -989,6 +993,7 @@ impl pyo3::impl_::pyclass::PyClassImpl for MyClass {
         visitor(collector.py_methods());
     }
 }
+
 # Python::with_gil(|py| {
 #     let cls = py.get_type::<MyClass>();
 #     pyo3::py_run!(py, cls, "assert cls.__name__ == 'MyClass'")
@@ -999,7 +1004,6 @@ impl pyo3::impl_::pyclass::PyClassImpl for MyClass {
 
 [`GILGuard`]: {{#PYO3_DOCS_URL}}/pyo3/struct.GILGuard.html
 [`PyTypeInfo`]: {{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PyTypeInfo.html
-[`PyTypeObject`]: {{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PyTypeObject.html
 
 [`PyCell`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyCell.html
 [`PyClass`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/trait.PyClass.html

guide/src/class/numeric.md

@@ -7,7 +7,7 @@ Before proceeding, we should think about how we want to handle overflows. There
  be reinventing the wheel.
 - We can raise exceptions whenever `Number` overflows, but that makes the API painful to use.
 - We can wrap around the boundary of `i32`. This is the approach we'll take here. To do that we'll just forward to `i32`'s
- `wrapping_*` methods. 
+ `wrapping_*` methods.
 
 ### Fixing our constructor
 
@@ -124,7 +124,7 @@ impl Number {
 }
 ```
 
-### Unary arithmethic operations
+### Unary arithmetic operations
 
 ```rust
 # use pyo3::prelude::*;
@@ -336,39 +336,39 @@ fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
 # def hash_djb2(s: str):
 #     n = Number(0)
 #     five = Number(5)
-#     
+#
 #     for x in s:
 #         n = Number(ord(x)) + ((n << five) - n)
 #     return n
-# 
+#
 # assert hash_djb2('l50_50') == Number(-1152549421)
 # assert hash_djb2('logo') == Number(3327403)
 # assert hash_djb2('horizon') == Number(1097468315)
-# 
-# 
+#
+#
 # assert Number(2) + Number(2) == Number(4)
 # assert Number(2) + Number(2) != Number(5)
-# 
+#
 # assert Number(13) - Number(7) == Number(6)
 # assert Number(13) - Number(-7) == Number(20)
-# 
+#
 # assert Number(13) / Number(7) == Number(1)
 # assert Number(13) // Number(7) == Number(1)
-# 
+#
 # assert Number(13) * Number(7) == Number(13*7)
-# 
+#
 # assert Number(13) > Number(7)
 # assert Number(13) < Number(20)
 # assert Number(13) == Number(13)
 # assert Number(13) >= Number(7)
 # assert Number(13) <= Number(20)
 # assert Number(13) == Number(13)
-# 
-# 
+#
+#
 # assert (True if Number(1) else False)
 # assert (False if Number(0) else True)
-# 
-# 
+#
+#
 # assert int(Number(13)) == 13
 # assert float(Number(13)) == 13
 # assert Number.__doc__ == "Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not.\nIt's not a story C would tell you. It's a Rust legend."
@@ -383,14 +383,14 @@ fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
 # assert Number(1337).__repr__() == 'Number(1337)'
 "#;
 
-# 
-# use pyo3::type_object::PyTypeObject;
-# 
+#
+# use pyo3::PyTypeInfo;
+#
 # fn main() -> PyResult<()> {
 #     Python::with_gil(|py| -> PyResult<()> {
 #         let globals = PyModule::import(py, "__main__")?.dict();
 #         globals.set_item("Number", Number::type_object(py))?;
-# 
+#
 #         py.run(SCRIPT, Some(globals), None)?;
 #         Ok(())
 #     })
@@ -446,4 +446,4 @@ fn wrap(obj: &PyAny) -> Result<i32, PyErr> {
 [`PyErr::take`]: https://docs.rs/pyo3/latest/pyo3/prelude/struct.PyErr.html#method.take
 [`Python`]: https://docs.rs/pyo3/latest/pyo3/struct.Python.html
 [`FromPyObject`]: https://docs.rs/pyo3/latest/pyo3/conversion/trait.FromPyObject.html
-[`pyo3::ffi::PyLong_AsUnsignedLongMask`]: https://docs.rs/pyo3/latest/pyo3/ffi/fn.PyLong_AsUnsignedLongMask.html
+[`pyo3::ffi::PyLong_AsUnsignedLongMask`]: https://docs.rs/pyo3/latest/pyo3/ffi/fn.PyLong_AsUnsignedLongMask.html

guide/src/class/protocols.md

@@ -4,8 +4,8 @@ Python's object model defines several protocols for different object behavior, s
 
 In the Python C-API which PyO3 is implemented upon, many of these magic methods have to be placed into special "slots" on the class type object, as covered in the previous section. There are two ways in which this can be done:
 
- - [New in PyO3 0.15, recommended in PyO3 0.16] In `#[pymethods]`, if the name of the method is a recognised magic method, PyO3 will place it in the type object automatically.
- - [Deprecated in PyO3 0.16] In special traits combined with the `#[pyproto]` attribute.
+ - In `#[pymethods]`, if the name of the method is a recognised magic method, PyO3 will place it in the type object automatically.
+ - [Deprecated since PyO3 0.16] In special traits combined with the `#[pyproto]` attribute.
 
 (There are also many magic methods which don't have a special slot, such as `__dir__`. These methods can be implemented as normal in `#[pymethods]`.)
 
@@ -404,7 +404,7 @@ impl ClassWithGCSupport {
 
 PyO3 can use the `#[pyproto]` attribute in combination with special traits to implement the magic methods which need slots. The special traits are listed below. See also the [documentation for the `pyo3::class` module]({{#PYO3_DOCS_URL}}/pyo3/class/index.html).
 
-Due to complexity in the implementation and usage, these traits are deprecated in PyO3 0.16 in favour of the `#[pymethods]` solution.
+Due to complexity in the implementation and usage, these traits were deprecated in PyO3 0.16 in favour of the `#[pymethods]` solution.
 
 All `#[pyproto]` methods can return `T` instead of `PyResult<T>` if the method implementation is infallible. In addition, if the return type is `()`, it can be omitted altogether.
 

guide/src/conversions/traits.md

@@ -215,7 +215,7 @@ struct RustyTransparentStruct {
 #### Deriving [`FromPyObject`] for enums
 
 The `FromPyObject` derivation for enums generates code that tries to extract the variants in the
-order of the fields. As soon as a variant can be extracted succesfully, that variant is returned.
+order of the fields. As soon as a variant can be extracted successfully, that variant is returned.
 This makes it possible to extract Python union types like `str | int`.
 
 The same customizations and restrictions described for struct derivations apply to enum variants,

guide/src/faq.md

@@ -8,7 +8,7 @@
 2. The initialization code calls some Python API which temporarily releases the GIL e.g. `Python::import`.
 3. Another thread (thread B) acquires the Python GIL and attempts to access the same `lazy_static` value.
 4. Thread B is blocked, because it waits for `lazy_static`'s initialization to lock to release.
-5. Thread A is blocked, because it waits to re-aquire the GIL which thread B still holds.
+5. Thread A is blocked, because it waits to re-acquire the GIL which thread B still holds.
 6. Deadlock.
 
 PyO3 provides a struct [`GILOnceCell`] which works equivalently to `OnceCell` but relies solely on the Python GIL for thread safety. This means it can be used in place of `lazy_static` or `once_cell` where you are experiencing the deadlock described above. See the documentation for [`GILOnceCell`] for an example how to use it.

guide/src/features.md

@@ -59,7 +59,6 @@ This feature enables a dependency on the `pyo3-macros` crate, which provides the
 - `#[pyfunction]`
 - `#[pyclass]`
 - `#[pymethods]`
-- `#[pyproto]`
 - `#[derive(FromPyObject)]`
 
 It also provides the `py_run!` macro.
@@ -78,9 +77,7 @@ See [the `#[pyclass]` implementation details](class.md#implementation-details) f
 
 ### `pyproto`
 
-This feature enables the `#[pyproto]` macro, which is an alternative (older, soon-to-be-deprecated) to `#[pymethods]` for defining magic methods such as `__eq__`.
-
-> This feature is enabled by default. To disable it, set `default-features = false` for the `pyo3` entry in your Cargo.toml.
+This feature enables the `#[pyproto]` macro, which is a deprecated alternative to `#[pymethods]` for defining magic methods such as `__eq__`.
 
 ### `nightly`
 

guide/src/migration.md

@@ -3,6 +3,43 @@
 This guide can help you upgrade code through breaking changes from one PyO3 version to the next.
 For a detailed list of all changes, see the [CHANGELOG](changelog.md).
 
+## from 0.16.* to 0.17
+
+### The `pyproto` feature is now disabled by default
+
+In preparation for removing the deprecated `#[pyproto]` attribute macro in a future PyO3 version, it is now gated behind an opt-in feature flag. This also gives a slight saving to compile times for code which does not use the deprecated macro.
+
+### `PyTypeObject` trait has been deprecated
+
+The `PyTypeObject` trait already was near-useless; almost all functionality was already on the `PyTypeInfo` trait, which `PyTypeObject` had a blanket implementation based upon. In PyO3 0.17 the final method, `PyTypeObject::type_object` was moved to `PyTypeInfo::type_object`.
+
+To migrate, update trait bounds and imports from `PyTypeObject` to `PyTypeInfo`.
+
+Before:
+
+```rust,ignore
+use pyo3::Python;
+use pyo3::type_object::PyTypeObject;
+use pyo3::types::PyType;
+
+fn get_type_object<T: PyTypeObject>(py: Python<'_>) -> &PyType {
+    T::type_object(py)
+}
+```
+
+After
+
+```rust
+use pyo3::{Python, PyTypeInfo};
+use pyo3::types::PyType;
+
+fn get_type_object<T: PyTypeInfo>(py: Python<'_>) -> &PyType {
+    T::type_object(py)
+}
+
+# Python::with_gil(|py| { get_type_object::<pyo3::types::PyList>(py); });
+```
+
 ## from 0.15.* to 0.16
 
 ### Drop support for older technologies

pyo3-macros-backend/src/attributes.rs

@@ -29,6 +29,7 @@ pub mod kw {
     syn::custom_keyword!(transparent);
     syn::custom_keyword!(unsendable);
     syn::custom_keyword!(weakref);
+    syn::custom_keyword!(immutable);
 }
 
 #[derive(Clone, Debug)]