Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow more methods to take interned arguments #2312

Merged
merged 9 commits into from May 2, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Expand Up @@ -14,10 +14,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Several methods of `Py` and `PyAny` now accept `impl IntoPy<Py<PyString>>` rather than just `&str` to allow use of the `intern!` macro. [#2312](https://github.com/PyO3/pyo3/pull/2312)
- Move `PyTypeObject::type_object` method to `PyTypeInfo` trait, and deprecate `PyTypeObject` trait. [#2287](https://github.com/PyO3/pyo3/pull/2287)
- The deprecated `pyproto` feature is now disabled by default. [#2322](https://github.com/PyO3/pyo3/pull/2322)
- Deprecate `ToBorrowedObject` trait (it is only used as a wrapper for `ToPyObject`). [#2333](https://github.com/PyO3/pyo3/pull/2333)


## [0.16.4] - 2022-04-14

### Added
Expand Down
7 changes: 4 additions & 3 deletions src/conversions/path.rs
@@ -1,3 +1,4 @@
use crate::intern;
use crate::types::PyType;
use crate::{FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject};
use std::borrow::Cow;
Expand All @@ -18,10 +19,10 @@ impl FromPyObject<'_> for PathBuf {
Ok(s) => s,
Err(err) => {
let py = ob.py();
let pathlib = py.import("pathlib")?;
let pathlib_path: &PyType = pathlib.getattr("Path")?.downcast()?;
let pathlib = py.import(intern!(py, "pathlib"))?;
let pathlib_path: &PyType = pathlib.getattr(intern!(py, "Path"))?.downcast()?;
if ob.is_instance(pathlib_path)? {
let path_str = ob.call_method0("__str__")?;
let path_str = ob.call_method0(intern!(py, "__str__"))?;
OsString::extract(path_str)?
} else {
return Err(err);
Expand Down
7 changes: 5 additions & 2 deletions src/err/mod.rs
Expand Up @@ -135,7 +135,10 @@ impl PyErr {
///
/// # Examples
/// ```rust
/// use pyo3::{exceptions::PyTypeError, types::PyType, IntoPy, PyErr, Python};
/// use pyo3::prelude::*;
/// use pyo3::exceptions::PyTypeError;
/// use pyo3::types::{PyType, PyString};
///
/// Python::with_gil(|py| {
/// // Case #1: Exception object
/// let err = PyErr::from_value(PyTypeError::new_err("some type error").value(py));
Expand All @@ -146,7 +149,7 @@ impl PyErr {
/// assert_eq!(err.to_string(), "TypeError: ");
///
/// // Case #3: Invalid exception value
/// let err = PyErr::from_value("foo".into_py(py).as_ref(py));
/// let err = PyErr::from_value(PyString::new(py, "foo").into());
/// assert_eq!(
/// err.to_string(),
/// "TypeError: exceptions must derive from BaseException"
Expand Down
60 changes: 38 additions & 22 deletions src/instance.rs
Expand Up @@ -3,7 +3,7 @@ use crate::conversion::PyTryFrom;
use crate::err::{self, PyDowncastError, PyErr, PyResult};
use crate::gil;
use crate::pycell::{PyBorrowError, PyBorrowMutError, PyCell};
use crate::types::{PyDict, PyTuple};
use crate::types::{PyDict, PyString, PyTuple};
use crate::{
ffi, pyclass::MutablePyClass, AsPyPointer, FromPyObject, IntoPy, IntoPyPointer, PyAny, PyClass,
PyClassInitializer, PyRef, PyRefMut, PyTypeInfo, Python, ToPyObject,
Expand Down Expand Up @@ -575,8 +575,8 @@ impl<T> Py<T> {
///
/// This is equivalent to the Python expression `self.attr_name = value`.
///
/// If calling this method becomes performance-critical, the [`intern!`] macro can be used
/// to intern `attr_name`, thereby avoiding repeated temporary allocations of Python strings.
/// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used
/// to intern `attr_name`.
///
/// # Example: `intern!`ing the attribute name
///
Expand Down Expand Up @@ -657,20 +657,28 @@ impl<T> Py<T> {
/// Calls a method on the object.
///
/// This is equivalent to the Python expression `self.name(*args, **kwargs)`.
pub fn call_method(
///
/// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used
/// to intern `name`.
pub fn call_method<N, A>(
&self,
py: Python<'_>,
name: &str,
args: impl IntoPy<Py<PyTuple>>,
name: N,
args: A,
kwargs: Option<&PyDict>,
) -> PyResult<PyObject> {
) -> PyResult<PyObject>
where
N: IntoPy<Py<PyString>>,
A: IntoPy<Py<PyTuple>>,
{
let name: Py<PyString> = name.into_py(py);
let args: Py<PyTuple> = args.into_py(py);

let ptr = self.getattr(py, name)?.into_ptr();
mejrs marked this conversation as resolved.
Show resolved Hide resolved
let args = args.into_ptr();
let kwargs = kwargs.into_ptr();

unsafe {
let args = args.into_py(py).into_ptr();
let kwargs = kwargs.into_ptr();
let ptr = ffi::PyObject_GetAttr(self.as_ptr(), name.to_object(py).as_ptr());
Copy link
Member Author

@mejrs mejrs Apr 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry this is a bit messy since resolving the merge conflict.

How do you feel about:

pyo3::ffi::Something(name.to_object(py).as_ptr());

versus

let ptr = name.to_object(py).into_ptr();
pyo3::ffi::Something(ptr);
ffi::Py_DECREF(ptr);

(or something else?)

Personally I prefer the latter. IMO the former is error prone and gives a bad example.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you feel the former is error prone? I think the second is more error prone because missing the Py_DECREF call is leaking memory just like missing free() in C.

Perhaps a middle ground is to make it obvious we're using RAII through Py?

let name: PyObject = name.to_object(py);
pyo3::ffi::Something(name.as_ptr());

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you feel the former is error prone? I think the second is more error prone because missing the Py_DECREF call is leaking memory just like missing free() in C.

I'm mostly worried about someone reading it and and writing it slightly differently:

let ptr = name.to_object(py).as_ptr();
pyo3::ffi::Something(ptr);

which produces a dangling pointer, as the python object is dropped at the end of the statement (I should add documentation about this).

Perhaps a middle ground is to make it obvious we're using RAII through Py?

let name: PyObject = name.to_object(py);
pyo3::ffi::Something(name.as_ptr());

Sure 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'm mostly worried about someone reading it and and writing it slightly differently:

This is an excellent point which I'd totally missed. IMO the middle ground above is probably the only sensible way to arrange this then. If you like, I'll submit a PR to fix the whole codebase to use this pattern once this PR merged? That way you can just worry about the modified functions here without another merge conflict.

if ptr.is_null() {
return Err(PyErr::fetch(py));
}
let result = PyObject::from_owned_ptr_or_err(py, ffi::PyObject_Call(ptr, args, kwargs));
ffi::Py_DECREF(ptr);
ffi::Py_XDECREF(args);
Expand All @@ -682,24 +690,32 @@ impl<T> Py<T> {
/// Calls a method on the object with only positional arguments.
///
/// This is equivalent to the Python expression `self.name(*args)`.
pub fn call_method1(
&self,
py: Python<'_>,
name: &str,
args: impl IntoPy<Py<PyTuple>>,
) -> PyResult<PyObject> {
///
/// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used
/// to intern `name`.
pub fn call_method1<N, A>(&self, py: Python<'_>, name: N, args: A) -> PyResult<PyObject>
where
N: IntoPy<Py<PyString>>,
A: IntoPy<Py<PyTuple>>,
{
self.call_method(py, name, args, None)
}

/// Calls a method on the object with no arguments.
///
/// This is equivalent to the Python expression `self.name()`.
pub fn call_method0(&self, py: Python<'_>, name: &str) -> PyResult<PyObject> {
///
/// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used
/// to intern `name`.
pub fn call_method0<N>(&self, py: Python<'_>, name: N) -> PyResult<PyObject>
where
N: IntoPy<Py<PyString>>,
{
cfg_if::cfg_if! {
if #[cfg(all(Py_3_9, not(any(Py_LIMITED_API, PyPy))))] {
// Optimized path on python 3.9+
unsafe {
let name = name.into_py(py);
let name: Py<PyString> = name.into_py(py);
PyObject::from_owned_ptr_or_err(py, ffi::PyObject_CallMethodNoArgs(self.as_ptr(), name.as_ptr()))
mejrs marked this conversation as resolved.
Show resolved Hide resolved
}
} else {
Expand Down Expand Up @@ -728,7 +744,7 @@ impl<T> Py<T> {

/// Create a `Py<T>` instance by taking ownership of the given FFI pointer.
///
/// If `ptr` is null then the current Python exception is fetched as a `PyErr`.
/// If `ptr` is null then the current Python exception is fetched as a [`PyErr`].
///
/// # Safety
/// If non-null, `ptr` must be a pointer to a Python object of type T.
Expand Down
10 changes: 7 additions & 3 deletions src/marker.rs
Expand Up @@ -122,10 +122,11 @@
use crate::err::{self, PyDowncastError, PyErr, PyResult};
use crate::gil::{self, GILGuard, GILPool};
use crate::impl_::not_send::NotSend;
use crate::types::{PyAny, PyDict, PyModule, PyType};
use crate::types::{PyAny, PyDict, PyModule, PyString, PyType};
use crate::version::PythonVersionInfo;
use crate::{
ffi, AsPyPointer, FromPyPointer, IntoPyPointer, PyNativeType, PyObject, PyTryFrom, PyTypeInfo,
ffi, AsPyPointer, FromPyPointer, IntoPy, IntoPyPointer, Py, PyNativeType, PyObject, PyTryFrom,
PyTypeInfo,
};
use std::ffi::{CStr, CString};
use std::marker::PhantomData;
Expand Down Expand Up @@ -590,7 +591,10 @@ impl<'py> Python<'py> {
}

/// Imports the Python module with the specified name.
pub fn import(self, name: &str) -> PyResult<&'py PyModule> {
pub fn import<N>(self, name: N) -> PyResult<&'py PyModule>
where
N: IntoPy<Py<PyString>>,
{
PyModule::import(self, name)
}

Expand Down