Description
Describe the bug
PyArrowConvert calls ArrowArray::into_raw
to convert the contents of ArrowArray::empty
into raw pointers, it then passes these to python to populate, before passing them to ArrowArray::try_from_raw
. This copies the memory contents and does not take ownership of the passed pointers, this appears to have been a change made in (#1449) by @viirya. The result is that the allocation of Arc<FFI_ArrowSchema>
and Arc<FFI_ArrowArray>
is never freed.
Some of the memory leaks appear to have been fixed in #1878 but not the ones present in the python interface.
I happen to think the into_raw
method is extremely hard to use correctly, and I would propose removing it in favor of an interface that does not yield ownership of the pointers.
To Reproduce
Expected behavior
Additional context
Activity
Cleanup FFI interface (apache#3684) (apache#3683)
Cleanup FFI interface (apache#3684) (apache#3683)
Cleanup FFI interface (#3684) (#3683) (#3685)
tustvold commentedon Feb 10, 2023
label_issue.py
automatically added labels {'arrow'} from #3685Fix pyarrow memory leak (apache#3683)
Fix pyarrow memory leak (apache#3683)
Fix pyarrow memory leak (apache#3683)
6 remaining items