Skip to content

PyArrowConvert Leaks Memory #3683

@tustvold

Description

@tustvold

Describe the bug

PyArrowConvert calls ArrowArray::into_raw to convert the contents of ArrowArray::empty into raw pointers, it then passes these to python to populate, before passing them to ArrowArray::try_from_raw. This copies the memory contents and does not take ownership of the passed pointers, this appears to have been a change made in (#1449) by @viirya. The result is that the allocation of Arc<FFI_ArrowSchema> and Arc<FFI_ArrowArray> is never freed.

Some of the memory leaks appear to have been fixed in #1878 but not the ones present in the python interface.

I happen to think the into_raw method is extremely hard to use correctly, and I would propose removing it in favor of an interface that does not yield ownership of the pointers.

To Reproduce

Expected behavior

Additional context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arrowChanges to the arrow cratebug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Participants

    @tustvold

    Issue actions

      PyArrowConvert Leaks Memory · Issue #3683 · apache/arrow-rs