Skip to content

Commit

Permalink
BUG: disable garbage collector during memory allocation hook
Browse files Browse the repository at this point in the history
When a cython object contains numpy arrays the pure python allocation
hook can trigger during cythons __dealloc__ method and trigger a second
deletion of the object currently being deleted.
To minimize the probabily that that happens disable the garbage
collector during the hook.
As this involves python calls it is still possible that a double delete
occurs but chances are lowered, a proper solution would be C only hook
like python 3.4 tracemalloc module.

Closes numpygh-4834
  • Loading branch information
juliantaylor committed Jul 5, 2014
1 parent 7a2b14a commit c9c53ea
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions tools/allocation_tracking/track_allocations.py
@@ -1,6 +1,7 @@
from __future__ import division, absolute_import, print_function

import numpy as np
import gc
import inspect
from alloc_hook import NumpyAllocHook

Expand Down Expand Up @@ -35,12 +36,21 @@ def __exit__(self, type, value, traceback):
self.numpy_hook.__exit__()

def hook(self, inptr, outptr, size):
# minimize the chances that the garbage collector kicks in during a
# cython __dealloc__ call and causes a double delete of the current
# object. To avoid this fully the hook would have to avoid all python
# api calls, e.g. by being implemented in C like python 3.4's
# tracemalloc module
gc_on = gc.isenabled()
gc.disable()
if outptr == 0: # it's a free
self.free_cb(inptr)
elif inptr != 0: # realloc
self.realloc_cb(inptr, outptr, size)
else: # malloc
self.alloc_cb(outptr, size)
if gc_on:
gc.enable()

def alloc_cb(self, ptr, size):
if size >= self.threshold:
Expand Down

0 comments on commit c9c53ea

Please sign in to comment.