Merge pull request #1560 from davidhewitt/deprecate-pyproto-pymethods

pyproto: deprecate py_methods

CHANGELOG.md

@@ -29,6 +29,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - Deprecate `PyModule` methods `call`, `call0`, `call1` and `get`. [#1492](https://github.com/PyO3/pyo3/pull/1492)
 - Add length information to `PyBufferError`s raised from `PyBuffer::copy_to_slice` and `PyBuffer::copy_from_slice`. [#1534](https://github.com/PyO3/pyo3/pull/1534)
 - Automatically provide `-undefined` and `dynamic_lookup` linker arguments on macOS with `extension-module` feature. [#1539](https://github.com/PyO3/pyo3/pull/1539)
+- Deprecate `#[pyproto]` methods which are easier to implement as `#[pymethods]`: [#1560](https://github.com/PyO3/pyo3/pull/1560)
+  - `PyBasicProtocol::__bytes__` and `PyBasicProtocol::__format__`
+  - `PyContextProtocol::__enter__` and `PyContextProtocol::__exit__`
+  - `PyDescrProtocol::__delete__` and `PyDescrProtocol::__set_name__`
+  - `PyMappingProtocol::__reversed__`
+  - `PyNumberProtocol::__complex__` and `PyNumberProtocol::__round__`
+  - `PyAsyncProtocol::__aenter__` and `PyAsyncProtocol::__aexit__`
 
 ### Removed
 - Remove deprecated exception names `BaseException` etc. [#1426](https://github.com/PyO3/pyo3/pull/1426)

benches/bench_pyclass.rs

@@ -31,10 +31,6 @@ impl PyObjectProtocol for MyClass {
     fn __str__(&self) -> &'static str {
         "MyClass"
     }
-
-    fn __bytes__(&self) -> &'static [u8] {
-        b"MyClass"
-    }
 }
 
 #[bench]

guide/src/class/protocols.md

@@ -1,12 +1,14 @@
 ## Class customizations
 
-Python's object model defines several protocols for different object behavior, like sequence,
-mapping or number protocols. PyO3 defines separate traits for each of them. To provide specific
-Python object behavior, you need to implement the specific trait for your struct. Important note,
-each protocol implementation block has to be annotated with the `#[pyproto]` attribute.
+PyO3 uses the `#[pyproto]` attribute in combination with special traits to implement certain protocol (aka `__dunder__`) methods of Python classes. The special traits are listed in this chapter of the guide. See also the [documentation for the `pyo3::class` module]({{#PYO3_DOCS_URL}}/pyo3/class/index.html).
 
-All `#[pyproto]` methods which can be defined below can return `T` instead of `PyResult<T>` if the
-method implementation is infallible. In addition, if the return type is `()`, it can be omitted altogether.
+Python's object model defines several protocols for different object behavior, such as the sequence, mapping, and number protocols. You may be familiar with implementing these protocols in Python classes by "dunder" methods, such as `__str__` or `__repr__`.
+
+In the Python C-API which PyO3 is dependent upon, many of these protocol methods have to be provided into special "slots" on the class type object. To fill these slots PyO3 uses the `#[pyproto]` attribute in combination with special traits.
+
+All `#[pyproto]` methods can return `T` instead of `PyResult<T>` if the method implementation is infallible. In addition, if the return type is `()`, it can be omitted altogether.
+
+There are many "dunder" methods which are not included in any of PyO3's protocol traits, such as `__dir__`. These methods can be implemented in `#[pymethods]` as already covered in the previous section.
 
 ### Basic object customization
 
@@ -29,15 +31,6 @@ Each method corresponds to Python's `self.attr`, `self.attr = value` and `del se
 
     Possible return types for `__str__` and `__repr__` are `PyResult<String>` or `PyResult<PyString>`.
 
-  * `fn __bytes__(&self) -> PyResult<PyBytes>`
-
-    Provides the conversion to `bytes`.
-
-  * `fn __format__(&self, format_spec: &str) -> PyResult<impl ToPyObject<ObjectType=PyString>>`
-
-    Special method that is used by the `format()` builtin and the `str.format()` method.
-    Possible return types are `PyResult<String>` or `PyResult<PyString>`.
-
 #### Comparison operators
 
   * `fn __richcmp__(&self, other: impl FromPyObject, op: CompareOp) -> PyResult<impl ToPyObject>`
@@ -132,14 +125,12 @@ The following methods implement the unary arithmetic operations (`-`, `+`, `abs(
 
 Support for coercions:
 
-  * `fn __complex__(&'p self) -> PyResult<impl ToPyObject>`
   * `fn __int__(&'p self) -> PyResult<impl ToPyObject>`
   * `fn __float__(&'p self) -> PyResult<impl ToPyObject>`
 
 Other:
 
   * `fn __index__(&'p self) -> PyResult<impl ToPyObject>`
-  * `fn __round__(&'p self, ndigits: Option<impl FromPyObject>) -> PyResult<impl ToPyObject>`
 
 ### Emulating sequential containers (such as lists or tuples)
 
@@ -237,12 +228,6 @@ For a mapping, the keys may be Python objects of arbitrary type.
     The same exceptions should be raised for improper key values as
     for the `__getitem__()` method.
 
-  * `fn __reversed__(&self) -> PyResult<impl ToPyObject>`
-
-    Called (if present) by the `reversed()` built-in to implement reverse iteration.
-    It should return a new iterator object that iterates over all the objects in
-    the container in reverse order.
-
 ### Garbage Collector Integration
 
 If your type owns references to other Python objects, you will need to

guide/src/migration.md

@@ -15,6 +15,47 @@ For projects embedding Python in Rust, PyO3 no longer automatically initalizes a
 
 The limitation of the new default implementation is that it cannot support multiple `#[pymethods]` blocks for the same `#[pyclass]`. If you need this functionality, you must enable the `multiple-pymethods` feature which will switch `#[pymethods]` to the inventory-based implementation.
 
+### Deprecated `#[pyproto]` methods
+
+Some protocol (aka `__dunder__`) methods such as `__bytes__` and `__format__` have been possible to implement two ways in PyO3 for some time: via a `#[pyproto]` (e.g. `PyBasicProtocol` for the methods listed here), or by writing them directly in `#[pymethods]`. This is only true for a handful of the `#[pyproto]` methods (for technical reasons to do with the way PyO3 currently interacts with the Python C-API).
+
+In the interest of having onle one way to do things, the `#[pyproto]` forms of these methods have been deprecated.
+
+To migrate just move the affected methods from a `#[pyproto]` to a `#[pymethods]` block.
+
+Before:
+
+```rust,ignore
+use pyo3::prelude::*;
+use pyo3::class::basic::PyBasicProtocol;
+
+#[pyclass]
+struct MyClass { }
+
+#[pyproto]
+impl PyBasicProtocol for MyClass {
+    fn __bytes__(&self) -> &'static [u8] {
+        b"hello, world"
+    }
+}
+```
+
+After:
+
+```rust
+use pyo3::prelude::*;
+
+#[pyclass]
+struct MyClass { }
+
+#[pymethods]
+impl MyClass {
+    fn __bytes__(&self) -> &'static [u8] {
+        b"hello, world"
+    }
+}
+```
+
 ## from 0.12.* to 0.13
 
 ### Minimum Rust version increased to Rust 1.45

pyo3-macros-backend/src/defs.rs

@@ -268,15 +268,15 @@ pub const DESCR: Proto = Proto {
     methods: &[
         MethodProto::new("__get__", "PyDescrGetProtocol").args(&["Receiver", "Inst", "Owner"]),
         MethodProto::new("__set__", "PyDescrSetProtocol").args(&["Receiver", "Inst", "Value"]),
-        MethodProto::new("__det__", "PyDescrDelProtocol")
+        MethodProto::new("__delete__", "PyDescrDelProtocol")
             .args(&["Inst"])
             .has_self(),
         MethodProto::new("__set_name__", "PyDescrSetNameProtocol")
             .args(&["Inst"])
             .has_self(),
     ],
     py_methods: &[
-        PyMethod::new("__del__", "PyDescrDelProtocolImpl"),
+        PyMethod::new("__delete__", "PyDescrDelProtocolImpl"),
         PyMethod::new("__set_name__", "PyDescrNameProtocolImpl"),
     ],
     slot_defs: &[

src/class/basic.rs

@@ -61,6 +61,10 @@ pub trait PyObjectProtocol<'p>: PyClass {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__format__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __format__(&'p self, format_spec: Self::Format) -> Self::Result
     where
         Self: PyObjectFormatProtocol<'p>,
@@ -75,6 +79,10 @@ pub trait PyObjectProtocol<'p>: PyClass {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__bytes__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __bytes__(&'p self) -> Self::Result
     where
         Self: PyObjectBytesProtocol<'p>,

src/class/context.rs

@@ -10,13 +10,21 @@ use crate::{PyClass, PyObject};
 /// Context manager interface
 #[allow(unused_variables)]
 pub trait PyContextProtocol<'p>: PyClass {
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__enter__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __enter__(&'p mut self) -> Self::Result
     where
         Self: PyContextEnterProtocol<'p>,
     {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__exit__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __exit__(
         &'p mut self,
         exc_type: Option<Self::ExcType>,

src/class/descr.rs

@@ -31,13 +31,21 @@ pub trait PyDescrProtocol<'p>: PyClass {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__delete__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __delete__(&'p self, instance: &'p PyAny) -> Self::Result
     where
         Self: PyDescrDeleteProtocol<'p>,
     {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__set_name__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __set_name__(&'p self, instance: &'p PyAny) -> Self::Result
     where
         Self: PyDescrSetNameProtocol<'p>,

src/class/mapping.rs

@@ -37,6 +37,10 @@ pub trait PyMappingProtocol<'p>: PyClass {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__reversed__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __reversed__(&'p self) -> Self::Result
     where
         Self: PyMappingReversedProtocol<'p>,

src/class/number.rs

@@ -283,6 +283,10 @@ pub trait PyNumberProtocol<'p>: PyClass {
     {
         unimplemented!()
     }
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__complex__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __complex__(&'p self) -> Self::Result
     where
         Self: PyNumberComplexProtocol<'p>,
@@ -307,6 +311,10 @@ pub trait PyNumberProtocol<'p>: PyClass {
     {
         unimplemented!()
     }
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__round__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __round__(&'p self, ndigits: Option<Self::NDigits>) -> Self::Result
     where
         Self: PyNumberRoundProtocol<'p>,

src/class/pyasync.rs

@@ -39,13 +39,21 @@ pub trait PyAsyncProtocol<'p>: PyClass {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__aenter__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __aenter__(&'p mut self) -> Self::Result
     where
         Self: PyAsyncAenterProtocol<'p>,
     {
         unimplemented!()
     }
 
+    #[deprecated(
+        since = "0.14.0",
+        note = "prefer implementing `__aexit__` in `#[pymethods]` instead of in a protocol"
+    )]
     fn __aexit__(
         &'p mut self,
         exc_type: Option<Self::ExcType>,

tests/test_arithmetics.rs

@@ -1,3 +1,5 @@
+#![allow(deprecated)] // for deprecated protocol methods
+
 use pyo3::class::basic::CompareOp;
 use pyo3::class::*;
 use pyo3::prelude::*;

tests/test_dunder.rs

@@ -1,3 +1,5 @@
+#![allow(deprecated)] // for deprecated protocol methods
+
 use pyo3::class::{
     PyAsyncProtocol, PyContextProtocol, PyDescrProtocol, PyIterProtocol, PyMappingProtocol,
     PyObjectProtocol, PySequenceProtocol,

tests/test_mapping.rs

@@ -1,3 +1,5 @@
+#![allow(deprecated)] // for deprecated protocol methods
+
 use std::collections::HashMap;
 
 use pyo3::exceptions::PyKeyError;