Memory of copied PIL Images is not released #7935
Comments
|
Pillow's memory allocator doesn't necessarily release the memory in the pool back as soon as an image is destroyed, as it uses that memory pool for future allocations. See Storage.c (https://github.com/python-pillow/Pillow/blob/main/src/libImaging/Storage.c#L310) for the implementation. If you repeatedly open and close an image, you should not see the memory increase, but it won't necessarily drop between destruction and allocation again. |
|
It looks like it caches 0 blocks by default though. Pillow/src/libImaging/Storage.c Lines 260 to 271 in aeeb596 And you can set the number of blocks to cache with the Lines 3624 to 3656 in aeeb596 There's a docs page for this actually: https://pillow.readthedocs.io/en/stable/reference/block_allocator.html |
|
Thanks for the quick answers! Indeed, when I set the |
|
This may or may not help - in your original code you open an image and don't close it. It is recommended instead that you either call Lines 560 to 565 in e8ab564 Edit: I see you've mentioned 'closing images' in your comments, so this remark can just be for reference to others. |
|
I didn't catch this before but what you're doing is basically opening 50 copies of an image and keeping them all. Can you show us a flow where you expect constant memory usage? |
|
Yes, in the first loop I open the image 50 times and hold 50 copies so the memory usage increases which is ok. Manually closing the image copy via import gc
import os
import sys
import time
import PIL.Image
import psutil
FILE = './test_image.png'
def LogMemory():
pid = os.getpid()
rss = 0
for mmap in psutil.Process(pid).memory_maps():
# All memory that this process holds in RAM. RSS = USS + Shared.
rss += mmap.rss
return rss
class Container:
def __init__(self):
self.value = None
def SetValue(self, value):
self.value = self._copyValue(value)
def GetValue(self):
return self._copyValue(self.value)
def _copyValue(self, value):
return value.copy()
if __name__ == '__main__':
print(f'Using Python {sys.version}, PIL {PIL.__version__}')
print(f'PILLOW_ALIGNMENT: {PIL.Image.core.get_alignment()}')
print(f'PILLOW_BLOCK_SIZE: {PIL.Image.core.get_block_size()}')
print(f'PILLOW_BLOCKS_MAX: {PIL.Image.core.get_blocks_max()}')
containers = []
for i in range(50):
before = LogMemory()
img = PIL.Image.open(FILE)
# img = [1] * (1920*1080*3) # this works!
container = Container()
container.SetValue(img)
containers.append(container)
after = LogMemory()
print(f'Loaded image {i} took {after-before} bytes')
for i in range(len(containers)):
before = LogMemory()
containers.pop()
time.sleep(0.1)
after = LogMemory()
print(f'popped container {i} released {before-after} bytes')
print('Delete list')
before = LogMemory()
containers = None
gc.collect()
after = LogMemory()
print(f'Finally released {before-after} bytes')Running the code with The memory usage decreases with every containers.pop() But when I run with and the used memory doesn't decrease while popping the containers from the list. So setting |
What did you do?
Our application works with PIL images and holds a list of containers. Every container has a copy of the last image to track manipulations of the image data. When we delete the containers, the memory reserved by the PIL images is not released. Even closing the image manually via
image.close()in the container's desctructor and calling the garbage collector does not release the memory.If I replace the PIL image with a Python list (line: 55) the memory gets freed when a container is popped from the list.
What did you expect to happen?
The memory, taken by a PIL image copy, should be released after each
containers.pop().What actually happened?
The memory isn't released.
Script output
What are your OS, Python and Pillow versions?
The text was updated successfully, but these errors were encountered: