How do I decrease the reference count of any PyObject that get passed into a method of my #[pyclass]?

[jymchng]

My question is the same as the title.

I want to decrease the reference count of the value which get passed into a method of my #[pyclass] to disallow the garbage collector from tracking it.

[adamreichold]

You could take Py<PyAny>, use .as_ptr() to get a raw pointer which you can pass into implementation and drop the Py<PyAny>. But that sounds like a recipe for disaster as potentially nothing would keep this pointee behind the raw pointer alive and you have no way of know which pointers are still valid and which are not.

To be honest, this sounds like an XY problem and maybe we could offer better advice if you describe what you are trying to achieve.

[jymchng]

@adamreichold Thanks for your reply.

I am writing to implement something that can remain as 'secretive' as it can be in python.

#[pyclass(name = "SecretNRIC")]
#[pyo3(text_signature = "(string, filepath, key_var)")]
#[derive(Debug)]
pub struct PySecretNRIC {
    pub rust_secret_nric: SecretNRICString,
}

pub struct SecretNRICString {
    pub encrypted_nric: String,
}

#[pymethods]
impl PySecretNRIC {
    #[new]
    pub fn new(string: &PyString, filepath: &str, key_var: &str) -> PyResult<Self> { // constructor to eventually return a `PySecretNRIC` which is a wrapper around `SecretNRICString` and again is a wrapper around `encrypted_nric` which is a `String`
        Python::with_gil( move |_py| {
            let string = string.extract::<String>()?;
            let new_nric = NRIC::new(string); // create a new `NRIC` object
            match new_nric {
                Ok(new_nric) => Ok(PySecretNRIC {
                    rust_secret_nric: SecretNRICString::new(new_nric, filepath, key_var)?,
                }),
                Err(err) => Err(PyValueError::new_err(err)),
            }
        })
    }
}

In python, I did this:

import gc
from nric_do_not_use import SecretNRIC #module name and class name

if __name__ == '__main__':
    w = list()
    for i in ['S1234567D', 'S6364259B']: # two secret strings
        w.append(SecretNRIC(i, '.env.example', 'SECRET_KEY')) # get two `SecretNRIC` objects with via a method I can get its encrypted string
    
    print(gc.get_referents(w)) # get objects referring to `w`.
    print('='*10)
    print(gc.get_referrers(w)) # get objects with reference to `w`.

gc.get_referents(w) returns the following:

[{'__name__': '__main__',
 '__doc__': None,
 '__package__': 'example',
 '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x0000023129F58640>,
 '__spec__': ModuleSpec(name='example.gc_referents', loader=<_frozen_importlib_external.SourceFileLoader object at 0x0000023129F58640>, origin='F:\\...\\crates\\nric\\example\\gc_referents.py'), 
'__annotations__': {},
 '__builtins__': <module 'builtins' (built-in)>,
 '__file__': 'F:\\...\\crates\\nric\\example\\gc_referents.py', 
'__cached__': 'F:\\...\\crates\\nric\\example\\__pycache__\\gc_referents.cpython-310.pyc',
 'gc': <module 'gc' (built-in)>, 
'SecretNRIC': <class 'builtins.SecretNRIC'>, 
'w': [<SECRETNRIC>, <SECRETNRIC>], 
'i': 'S6364259B'}]

Where it clearly references i which is latter secret string. How do I prevent that?

Thank you.

[adamreichold]

I suspect this might actually be the i being kept alive in that scope, e.g. try to del i either within or after the loop?

That said, I would like to re-iterate: There are no security boundaries between Python code and native extensions.

While you may "hide" a value stored inside a native object from trivial inspection, the value itself remains reachable to Python code. Additionally, accidental exposure would also be prevented by just wrapping it in a Python class. So it is unclear to me which threat model is targetted by this approach.

[jymchng]

There are no security boundaries between Python code and native extensions.

Yes with regards to your reiteration, I understand it lol, I'm just wishing to try to enhance the security of some of the python apps.

Additionally, accidental exposure would also be prevented by just wrapping it in a Python class.

Can you explain what do you mean by 'wrapping it in a Python class'?

I suspect this might actually be the i being kept alive in that scope, e.g. try to del i either within or after the loop?

Is there a way to delete i within Rust in the sense of reducing the reference count of string: &PyString in the constructor of PySecretNRIC right at the end of the constructor?

[adamreichold]

Is there a way to delete i within Rust in the sense of reducing the reference count of string: &PyString in the constructor of PySecretNRIC right at the end of the constructor?

My point is that the dictionary in the list returned by get_referents is a scope, probably the scope of local variables when w was defined. I don't think the reference originates with the Rust code at all. Hence the recommendation to place del i in the Python code.

[adamreichold]

Can you explain what do you mean by 'wrapping it in a Python class'?

Write basically the same class you wrote in Rust using Python. Call the field __do_not_access_nirc_string and an accidental direct access can be found by search that name in the source code.

[jymchng]

Oh, then how does this prevent an accidental direct access to the secret value? I am hoping that as much as possible to prevent direct access to the secret value during python runtime.

[adamreichold]

Because no programmer will accidentally write down this name. If you are concerned about debugging logging and such, you can overwrite dunder methods like __str__ and __repr__. The point is that this is also not a security boundary, but it would suffice against the "I accidentally passed around this sensitive piece of data" kind of mistakes.