Drop trait never called on exit

[jimy-byerley]

Hello there, I have an issue on program exit where the Drop trait of a PyClass object doesn't get called when python is exitting. I first encountered it in an extension module I am working on, but it seems to happen also when it is rust initializing python.

Here is minimal example

use pyo3::prelude::*;

fn main() {
    println!("starting");
    let a = Python::with_gil(|py| {
        Py::new(py, A::new(1))
    });
    let b = A::new(2);
    drop(b);
    drop(a);    // commenting this changes nothing
    println!("ending");
//     Python::with_gil(|_| {});   // uncomment it to have a being freed
}

#[pyclass]
struct A {
    a: u32,
}
impl A {
    fn new(a: u32) -> Self {
        println!("creating");
        Self {a}
    }
}
impl Drop for A {
    fn drop(&mut self) {
        println!("dropping {}", self.a);
        self.a = 0;
    }
}

Using pyo3==0.21.2 or lower, and python==3.11.2 I get the following output

starting
creating 1
creating 2
dropping 2
ending

I would have expected to see drop 1 somewhere, likely after ending and main exit, when the started python interpreter stops and reacquire its GIL to drop all its objects. Instead it seems the object is never dropped.

If I reacquire the GIL before leaving main (uncommenting the last line) I finally get what's expected.

starting
creating 1
creating 2
dropping 2
ending
dropping 1

In this example, reacquire the GIL at the end of the main seems to be a workaround, but it is not in the case of an extension module where you do not know when the program will stop and thus cannot insert this GIL acquisition.

In my usecase, I need to properly stop rust threads in drop
Do you have an idea of what to do in order to ensure drop will be called ?

[birkenfeld]

Regardless of if this instance can be fixed, relying on destructors being called at program exit is a recipe for sadness when Python is involved.