Unable to build Pillow 5.3.0 on Debian 9

[zopyx]

We are unable to build Pillow 5.3.0 on Debian 9

../bin/python setup.py develop
running develop
running egg_info
writing src/Pillow.egg-info/PKG-INFO
writing top-level names to src/Pillow.egg-info/top_level.txt
writing dependency_links to src/Pillow.egg-info/dependency_links.txt
reading manifest file 'src/Pillow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.c'
warning: no files found matching '*.h'
warning: no files found matching '*.sh'
no previously-included directories found matching 'docs/_static'
warning: no previously-included files found matching '.appveyor.yml'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.codecov.yml'
warning: no previously-included files found matching '.editorconfig'
warning: no previously-included files found matching '.landscape.yaml'
warning: no previously-included files found matching '.travis'
warning: no previously-included files found matching '.travis/*'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
writing manifest file 'src/Pillow.egg-info/SOURCES.txt'
running build_ext


The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.

Please see the install instructions at:
   https://pillow.readthedocs.io/en/latest/installation.html

Traceback (most recent call last):
  File "setup.py", line 814, in <module>
    raise RequiredDependencyException(msg)
__main__.RequiredDependencyException: 

The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.

Please see the install instructions at:
   https://pillow.readthedocs.io/en/latest/installation.html

Building Pillow 4.3.0 on the same system works without issue.

zlib is of course installed:

apt list --installed |grep zlib1

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

zlib1g/stable,now 1:1.2.8.dfsg-5 amd64 [installed]
zlib1g-dev/stable,now 1:1.2.8.dfsg-5 amd64 [installed]

[hugovk]

Do the intermediate releases build? 5.0.0, 5.1.0, 5.2.0.

[zopyx]

Nope

[kkopachev]

probably related to #3103 and #3245 so you might want to pass INCLUDE=/usr/include/x86_64-linux-gnu/

[pslacerda]

I agree with @kkopachev that it is related and that the include will help. Build seems the right tag for this issue.

[zopyx]

Adding INCLUDE manually is not an option, at least not a long term solution. Pillow must build out of the box on any system. Otherwise it is useless for arbitrary usage and deployments.

[hugovk]

Does #3245 fix it?

[pslacerda]

@hugovk it does not fix it. It is related in the sense that both situations need to look for data in order to compile. Source data is even harder because as far as I know there isn't a specialized software artifact that hold such search paths, only standard locations. But is a little less crazy because compiled objects has much more hardware specialized data.

[wiredfool]

Debian has a package for pillow that is kept relatively up to date in Sid. I wonder what they’ve done to it (without upstreaming changes) that gets it to build.

(The last time they removed rather than fixed a bunch of pathfinding code, it wound up breaking when users installed from source on Debian. I wouldn’t be surprised if that repeated.)

[pslacerda]

Great idea to look the Debian build system, maybe also Make and python build tools like SCons.

[pslacerda]

Seems that SCons delegate search paths to the compiler (-I).

Somewhat related code: https://github.com/llvm-mirror/clang/blob/5e1f58d79130478b969bcbb29dd1545852500f7f/lib/Driver/ToolChains/Linux.cpp#L41

There is a clang tool capable of dump "HeaderMap" files. https://github.com/llvm-mirror/clang/blob/e6966fe563654422468e12db1abc3aad69fd78fd/lib/Frontend/InitHeaderSearch.cpp#L210

My cheapware defected mobile phone don't let me research more. Monday I will look further.

[hangya]

I had this issue on Ubuntu 18.04; zlib1g-dev package was installed but was unable to build Pillow.

It turned out that gcc and dpkg-dev packages are also required, after installing them building Pillow was successful.

[marcelolima]

@hangya Thanks for sharing, it worked for me and it's a cleaner solution.

[radarhere]

Here's how the Pillow docker images build for Debian 9 - https://github.com/python-pillow/docker-images/blob/master/archive/debian-9-stretch-x86/Dockerfile

[radarhere]

Debian 9 will no longer be receiving updates - https://www.debian.org/News/2020/20200718

[radarhere]

Reducing the packages used in the Debian 9 Dockerfile to build and test Pillow to those only absolutely necessary, I came up with libjpeg62-turbo-dev, python3-dev, python3-pip (for installing pytest), python3-setuptools and zlib1g-dev - see radarhere/docker-images@b095fb9 and https://github.com/radarhere/docker-images/actions/runs/1759732948.

In that CI run, I was using Pillow 7.2.0, but it also works if I switch to 5.3.0.

[radarhere]

So yes, unable to replicate. My best guess to the original problem here is that the zlib installation was corrupted somehow.

As for #3462 (comment), I wonder if that's not a different problem to being unable to find zlib.

[radarhere]

Debian 9 will end LTS on June 30, before the next release of Pillow.

Closing, unless there are any further comments.