Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to load a compressed TIFF file #69

Closed
kmilos opened this issue Dec 10, 2019 · 15 comments
Closed

Unable to load a compressed TIFF file #69

kmilos opened this issue Dec 10, 2019 · 15 comments

Comments

@kmilos
Copy link
Contributor

kmilos commented Dec 10, 2019

Issue:

When decoding a compressed (e.g. LZW or deflate) TIFF file, an OSError: -2 is thrown. Using e.g. this image:

>>> from PIL import Image
>>> im = Image.open('hopper_lzw.tif')
>>> im.load()
tempfile.tif: Cannot read TIFF header.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\kmilos\.conda\envs\test\lib\site-packages\PIL\TiffImagePlugin.py", line 1097, in load
    return self._load_libtiff()
  File "C:\Users\kmilos\.conda\envs\test\lib\site-packages\PIL\TiffImagePlugin.py", line 1209, in _load_libtiff
    raise IOError(err)
OSError: -2

Seems to be happening only on Windows and only w/ the conda package using the separate libtiff, the statically linked pillow wheel from PyPI works.

See also python-pillow/Pillow#4237 for more details.


Environment (conda list): 
(test) C:\Users\kmilos>conda list
# packages in environment at C:\Users\kmilos\.conda\envs\test:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                     py_0    conda-forge
astroid                   2.3.3                    py37_0    conda-forge
attrs                     19.3.0                     py_0    conda-forge
babel                     2.7.0                      py_0    conda-forge
backcall                  0.1.0                      py_0    conda-forge
blas                      1.0                         mkl    conda-forge
bleach                    3.1.0                      py_0    conda-forge
ca-certificates           2019.11.28           hecc5488_0    conda-forge
certifi                   2019.11.28               py37_0    conda-forge
cffi                      1.13.2           py37hb32ad35_0    conda-forge
chardet                   3.0.4                 py37_1003    conda-forge
cloudpickle               1.2.2                      py_1    conda-forge
colorama                  0.4.1                      py_0    conda-forge
colour-science            0.3.13                     py_0    conda-forge
cryptography              2.8              py37hb32ad35_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
decorator                 4.4.1                      py_0    conda-forge
defusedxml                0.6.0                      py_0    conda-forge
docutils                  0.15.2                   py37_0    conda-forge
entrypoints               0.3                   py37_1000    conda-forge
enum34                    1.1.6                 py37_1002    conda-forge
freetype                  2.10.0               h563cfd7_1    conda-forge
icc_rt                    2019.0.0             h0cc432a_1  
icu                       64.2                 he025d50_1    conda-forge
idna                      2.8                   py37_1000    conda-forge
imagecodecs-lite          2019.12.3        py37hfa6e2cd_0    conda-forge
imageio                   2.6.1                    py37_0    conda-forge
imagesize                 1.1.0                      py_0    conda-forge
importlib_metadata        0.23                     py37_0    conda-forge
intel-openmp              2019.4                      245  
ipykernel                 5.1.3            py37h5ca1d4c_0    conda-forge
ipython                   7.10.1           py37h5ca1d4c_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
isort                     4.3.21                   py37_0    conda-forge
jedi                      0.15.1                   py37_0    conda-forge
jinja2                    2.10.3                     py_0    conda-forge
jpeg                      9c                hfa6e2cd_1001    conda-forge
jsonschema                3.2.0                    py37_0    conda-forge
jupyter_client            5.3.3                    py37_1    conda-forge
jupyter_core              4.6.1                    py37_0    conda-forge
keyring                   19.2.0                   py37_0    conda-forge
kiwisolver                1.1.0            py37he980bc4_0    conda-forge
lazy-object-proxy         1.4.3            py37hfa6e2cd_0    conda-forge
libblas                   3.8.0                    14_mkl    conda-forge
libcblas                  3.8.0                    14_mkl    conda-forge
libclang                  9.0.0           default_hf44288c_4    conda-forge
liblapack                 3.8.0                    14_mkl    conda-forge
liblapacke                3.8.0                    14_mkl    conda-forge
libopencv                 3.4.7                    py37_7    conda-forge
libpng                    1.6.37               h7602738_0    conda-forge
libsodium                 1.0.17               h2fa13f4_0    conda-forge
libtiff                   4.1.0                h21b02b4_1    conda-forge
libwebp                   1.0.2                hfa6e2cd_4    conda-forge
llvmlite                  0.30.0                   py37_0    conda-forge
lz4-c                     1.8.3             he025d50_1001    conda-forge
markupsafe                1.1.1            py37hfa6e2cd_0    conda-forge
matplotlib                3.1.2                    py37_1    conda-forge
matplotlib-base           3.1.2            py37h2981e6d_1    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4           py37hfa6e2cd_1000    conda-forge
mkl                       2019.4                      245  
mkl-service               2.3.0            py37hfa6e2cd_0    conda-forge
more-itertools            7.2.0                      py_0    conda-forge
nbconvert                 5.6.1                    py37_0    conda-forge
nbformat                  4.4.0                      py_1    conda-forge
numba                     0.46.0           py37he350917_1    conda-forge
numpy                     1.17.3           py37hc71023c_0    conda-forge
numpydoc                  0.9.1                      py_0    conda-forge
olefile                   0.46                       py_0    conda-forge
opencv                    3.4.7                    py37_7    conda-forge
openssl                   1.1.1d               hfa6e2cd_0    conda-forge
packaging                 19.2                       py_0    conda-forge
pandoc                    2.8.0.1                       0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
parso                     0.5.1                      py_0    conda-forge
pathlib                   1.0.1                    py37_1    conda-forge
pickleshare               0.7.5                 py37_1000    conda-forge
pillow                    6.2.1            py37h5fcff3f_1    conda-forge
pip                       19.3.1                   py37_0    conda-forge
prompt_toolkit            3.0.2                      py_0    conda-forge
psutil                    5.6.7            py37hfa6e2cd_0    conda-forge
py-opencv                 3.4.7            py37h5ca1d4c_7    conda-forge
pycodestyle               2.5.0                      py_0    conda-forge
pycparser                 2.19                     py37_1    conda-forge
pyflakes                  2.1.1                      py_0    conda-forge
pygments                  2.5.2                      py_0    conda-forge
pylint                    2.4.4                    py37_0    conda-forge
pyopenssl                 19.0.0                   py37_0    conda-forge
pyparsing                 2.4.5                      py_0    conda-forge
pyqt                      5.12.3           py37h6538335_1    conda-forge
pyqt5-sip                 4.19.18                  pypi_0    pypi
pyqtwebengine             5.12.1                   pypi_0    pypi
pyrsistent                0.15.6           py37hfa6e2cd_0    conda-forge
pysocks                   1.7.1                    py37_0    conda-forge
python                    3.7.3                h510b542_1    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
pytz                      2019.3                     py_0    conda-forge
pywin32                   225              py37hfa6e2cd_0    conda-forge
pywin32-ctypes            0.2.0                 py37_1000    conda-forge
pyzmq                     18.1.1           py37h16f9016_0    conda-forge
qt                        5.12.5               h7ef1ec2_0    conda-forge
qtawesome                 0.6.0                      py_0    conda-forge
qtconsole                 4.6.0                      py_0    conda-forge
qtpy                      1.9.0                      py_0    conda-forge
quaternion                2019.10.3.10.26.21  py37hc8d92b1_0    conda-forge
requests                  2.22.0                   py37_1    conda-forge
rope                      0.14.0                     py_0    conda-forge
scipy                     1.3.1            py37h29ff71c_0    conda-forge
setuptools                42.0.2                   py37_0    conda-forge
six                       1.13.0                   py37_0    conda-forge
snowballstemmer           2.0.0                      py_0    conda-forge
sphinx                    2.2.1                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.1                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.1                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.2                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.2                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.3                      py_0    conda-forge
spyder                    3.3.6                    py37_1    conda-forge
spyder-kernels            0.5.2                    py37_0    conda-forge
sqlite                    3.30.1               hfa6e2cd_0    conda-forge
testpath                  0.4.4                      py_0    conda-forge
tifffile                  2019.7.26                py37_2    conda-forge
tk                        8.6.10               hfa6e2cd_0    conda-forge
tornado                   6.0.3            py37hfa6e2cd_0    conda-forge
traitlets                 4.3.3                    py37_0    conda-forge
urllib3                   1.25.7                   py37_0    conda-forge
vc                        14.1                 h0510ff6_4  
vs2015_runtime            14.16.27012          hf0eaf9b_0  
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.33.6                   py37_0    conda-forge
win_inet_pton             1.1.0                    py37_0    conda-forge
wincertstore              0.2                   py37_1003    conda-forge
wrapt                     1.11.2           py37hfa6e2cd_0    conda-forge
xz                        5.2.4             h2fa13f4_1001    conda-forge
zeromq                    4.3.2                h6538335_2    conda-forge
zipp                      0.6.0                      py_0    conda-forge
zlib                      1.2.11            h2fa13f4_1006    conda-forge
zstd                      1.4.4                hd8a0e53_1    conda-forge

Details about conda and system ( conda info ): 
(test) C:\Users\kmilos>conda info

     active environment : test
    active env location : C:\Users\kmilos\.conda\envs\test
            shell level : 2
       user config file : C:\Users\kmilos\.condarc
 populated config files : C:\Users\kmilos\.condarc
          conda version : 4.7.12
    conda-build version : not installed
         python version : 3.7.0.final.0
       virtual packages : 
       base environment : C:\ProgramData\Anaconda3  (read only)
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\ProgramData\Anaconda3\pkgs
                          C:\Users\kmilos\.conda\pkgs
                          C:\Users\kmilos\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\kmilos\.conda\envs
                          C:\ProgramData\Anaconda3\envs
                          C:\Users\kmilos\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.7.12 requests/2.19.1 CPython/3.7.0 Windows/10 Windows/10.0.17763
          administrator : False
             netrc file : None
           offline mode : False
@ocefpaf
Copy link
Member

ocefpaf commented Dec 10, 2019 •
edited

I can reproduce that on Windows. Downgrading libtiff to 4.0.10 (conda install libtiff=4.0.10) works. Investigating it...

PS: it works fine on macOS and Linux, only Windows is failing.

ocefpaf added a commit to ocefpaf/pillow-feedstock that referenced this issue Dec 10, 2019
@ocefpaf
test conda-forge#69
c275c04
@ocefpaf ocefpaf mentioned this issue Dec 10, 2019
Closed
ocefpaf added a commit to ocefpaf/pillow-feedstock that referenced this issue Dec 10, 2019
@ocefpaf
test conda-forge#69
c2f3014
@cmlyldz cmlyldz mentioned this issue Dec 12, 2019
Closed
@hugovk hugovk mentioned this issue Dec 12, 2019
Closed
@kmilos
Copy link
Contributor Author

kmilos commented Dec 27, 2019

Some new info here: python-pillow/Pillow#4237 (comment)

@kmilos kmilos mentioned this issue Jan 3, 2020
Closed
3 tasks
@patricksnape
Copy link
Contributor

Can you double check this was solved by conda-forge/libtiff-feedstock#49?

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Jan 4, 2020

that patch did end up fixing things for pillow, but it definitely broke opencv.

opencv worked correctly before the patch.

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Jan 4, 2020

I think maybe PIL should patch?

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Jan 4, 2020 •
edited

So strange, I don't know opencv does it:

conda install -c conda-forge opencv libtiff=4.0 python=3.7 pillow

Then:

from PIL import Image
im = Image.open('hopper_lzw.tif')
im.load()
import cv2
cv2.imread('hopper_lzw.tif')

both work.

Now, install the unpatched 4.1

conda install libtiff=4.1=*1

PIL fails, opencv works.

Now, install the patched up 4.1

conda install libtiff=4.1=*2

PIL works, opencv doesn't.

@patricksnape
Copy link
Contributor

Looks like @cgohlke has the solution here:

python-pillow/Pillow#4237 (comment)

So re-patch libtiff and try again?

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Jan 4, 2020

Let's upload to our private channels first ;)

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Jan 4, 2020

Ok, I think I setup my azure environment correctly.

Could one of you make a PR to my fork? If I would, it would upload on every push, kinda annoying

I would want to test against:

  1. opencv
  2. tifffile
  3. PIL

My feeling is that we can get PIL to work, but we might break tifffile and opencv. If that is the case, then it is really a PIL issue.

@tdegeus
Copy link
Member

tdegeus commented Jan 8, 2020

Dropping in a bit, I too am getting errors with libtiff.5:

Traceback (most recent call last):
  File "/Users/tdegeus/miniconda3/envs/scripta/lib/python3.8/site-packages/matplotlib/image.py", line 1412, in imread
    from PIL import Image
  File "/Users/tdegeus/miniconda3/envs/scripta/lib/python3.8/site-packages/PIL/Image.py", line 69, in <module>
    from . import _imaging as core
ImportError: dlopen(/Users/tdegeus/miniconda3/envs/scripta/lib/python3.8/site-packages/PIL/_imaging.cpython-38-darwin.so, 2): Library not loaded: @rpath/libwebp.7.dylib
  Referenced from: /Users/tdegeus/miniconda3/envs/scripta/lib/libtiff.5.dylib
  Reason: image not found

@patricksnape
Copy link
Contributor

@tdegeus This isn't related to this issue I'm afraid - can you please open a new issues and provide your full conda environment information?

@patricksnape
Copy link
Contributor

FYI @tdegeus this is likely due to conda-forge/libtiff-feedstock#48 (just me pattern matching on the packages)

@ocefpaf
Copy link
Member

ocefpaf commented Jan 8, 2020

@patricksnape I also noted that in gdal and pulled the packages. We'll have to roll a migrator to get that change in.

@tdegeus
Copy link
Member

tdegeus commented Jan 9, 2020

@patricksnape It seems that my issues were resolve in https://github.com/conda-forge/libtiff-feedstock

@kmilos
Copy link
Contributor Author

kmilos commented Mar 18, 2020

libtiff 4.1.0-h885aae3_4 works for me after conda-forge/libtiff-feedstock#51

@kmilos kmilos closed this as completed Mar 18, 2020
@ocefpaf ocefpaf mentioned this issue Jul 3, 2020
Merged
4 tasks
@kmilos kmilos mentioned this issue Feb 1, 2021
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@hmaarrfk @patricksnape @ocefpaf @kmilos @tdegeus