New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OverflowError: size does not fit in an int #2251
Comments
Can you post a minimal script that reproduces the error? |
@wiredfool Here is the minimal script that reproduces the error: from PIL import Image
import numpy as np
image = np.random.randint(255, size=(31000, 30030, 3))
image = image.astype('uint8')
# next line will cause the OverflowError
img = Image.fromarray(image, 'RGB') |
The underlying issue in #1475 isn't actually fixed, I kept getting memory errors on the randint line, but running this as: from PIL import Image
import numpy as np
a = np.zeros((31000, 30030, 3), dtype=np.uint8)
# next line will cause the OverflowError
img = Image.fromarray(a, 'RGB') Gives me the same issue, it's failing in decode.c:127, the same spot as in #1475. And attempting to mmap this: a = np.zeros((31000, 30030, 4), dtype=np.uint8)
img = Image.fromarray(a, 'RGBX') Fails in a new and interesting way:
which is in this code: map.c:345 if (!PyArg_ParseTuple(args, "O(ii)sOn(sii)", &target, &xsize, &ysize,
&codec, &bbox, &offset, &mode, &stride, &ystep))
return NULL;
if (!PyImaging_CheckBuffer(target)) {
PyErr_SetString(PyExc_TypeError, "expected string or buffer");
return NULL;
}
if (stride <= 0) {
if (!strcmp(mode, "L") || !strcmp(mode, "P"))
stride = xsize;
else if (!strncmp(mode, "I;16", 4))
stride = xsize * 2;
else
stride = xsize * 4;
}
if (ysize > INT_MAX / stride) {
PyErr_SetString(PyExc_MemoryError, "Integer overflow in ysize");
return NULL;
} |
First didn't work in OpenSlide. This test shows the issue, from: python-pillow#2251 (comment)
Is this issue solved? I'm facing same issue in latest version of pillow too. Please help. |
@monjoybme As of Pillow 6.0.0, I would expect you not to be receiving, 'OverflowError: size does not fit in an int', but instead 'ValueError: not enough image data'. I have created PR #3791 to move the problem one step further and solve the issue completely. |
@radarhere Could you tell me how can I solve
I'm using this https://github.com/deroneriksson/python-wsi-preprocessing/blob/c82dcafff48718da041ab99b7534ba697db2cc43/deephistopath/wsi/util.py#L43 |
To solve the error now, you will need to compile Pillow from my PR's branch.
If my PR is merged into Pillow master before July 1, then you should be able to go back to using standard Pillow from 6.1.0 onwards. |
@radarhere Thanks a lot. I compiled and getting new error ValueError: not enough image data `File "WsiToImagePatch.py", line 35, in np_to_pil
File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 2554, in fromarray
File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 2497, in frombuffer
File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 2430, in frombytes
File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 812, in frombytes
ValueError: not enough image data |
Okay. I would have expected that from Pillow 6.0, or from the current Pillow master, but not from my 'int' branch. Could you provide a self-contained example, I presume some form of WsiToImagePatch.py? |
You're linking to a function. I don't know what the input is - 'np_img'. For example, this script uses that function, but it works fine even without my PR. from PIL import Image
import numpy as np
def np_to_pil(np_img):
"""
Convert a NumPy array to a PIL Image.
Args:
np_img: The image represented as a NumPy array.
Returns:
The NumPy array converted to a PIL Image.
"""
if np_img.dtype == "bool":
np_img = np_img.astype("uint8") * 255
elif np_img.dtype == "float64":
np_img = (np_img * 255).astype("uint8")
return Image.fromarray(np_img)
np_img = np.ndarray(shape=(1, 1))
np_to_pil(np_img) The particular input that you are feeding it is triggering the problem. If it's just that your input is too large, I would really think that this should have been solved by my PR. If it's not fixed, then something unexpected is happening, and to investigate further, I would like to be able to trigger the problem myself. To do that, I need to have the specific input. |
Thanks a lot. I've shared (only with you) the code here https://github.com/monjoybme/WSI_to_patch/invitations. |
@monjoybme if you would like this looked at further, please create a new issue with a self-contained example |
Thank you |
@radarhere I am experiencing this issue for Pillow version 5.4.1. A very simple way to reproduce it is: from PIL import Image
import numpy as np
Image.fromarray(np.zeros((29440, 27648, 3),'uint8')) Thanks in advance. |
@lishangqiu testing, I'm able to reproduce the problem with 5.4.1, but it works fine for the latest version of Pillow. Is there a reason you can't upgrade to the latest Pillow? |
Hi,
I am getting an error very similar to #1475 with the
Pillow==3.4.2
, that is supposed to be fixed.Actually I'm getting:
Details about my image are following:
shape = (31000, 30030, 3)
dtype = uint8
My Python version is
Python 2.7.12
The text was updated successfully, but these errors were encountered: