Test is failing with AttributeError for ppc64le

[asellappen]

What did you do?

I am working for IBM to port cpu arch ppc64le for open sources , As a part of continuous integration on this package, We have applied changes , This helps us simplify testing later when distributions are re-building and re-releasing, We typically build applications for customers and ISVs, and while we don't use this package directly,
on the part this work , we are currently working on the package Pillow to support ppc64le.

What did you expect to happen?

We included travis job for ppc64le and it fails in testing instead of 100% pass,The failure happens at test test_subifd with AttributeErrorat ,but expectaion is should be passed.

What actually happened?

Ps refer h=ob https://travis-ci.com/github/asellappen/Pillow/jobs/477131803

Tests/test_file_tiff.py::TestFileTiff::test_subifd FAILED                [ 36%]
=================================== FAILURES ===================================
___________________________ TestFileTiff.test_subifd ___________________________
mode = '1', decoder_name = 'libtiff', args = ('1;I', 'group4', 13, 800)
extra = ()
    def _getdecoder(mode, decoder_name, args, extra=()):
    
        # tweak arguments
        if args is None:
            args = ()
        elif not isinstance(args, tuple):
            args = (args,)
    
        try:
            decoder = DECODERS[decoder_name]
        except KeyError:
            pass
        else:
            return decoder(mode, *args + extra)
    
        try:
            # get decoder
>           decoder = getattr(core, decoder_name + "_decoder")
E           AttributeError: module 'PIL._imaging' has no attribute 'libtiff_decoder'
/home/travis/virtualenv/python3.7.9/lib/python3.7/site-packages/Pillow-8.2.0.dev0-py3.7-linux-ppc64le.egg/PIL/Image.py:436: AttributeError

What are your OS, Python and Pillow versions?

• OS: lunix - ubuntu bionic .
• Python: py 3.7 & 3.8
• Pillow: version Pillow 8.2.0.dev0
• PIL SETUP SUMMARY

---

version Pillow 8.2.0.dev0
platform linux 3.8.7 (default, Jan 9 2021, 06:30:31)
[GCC 5.4.0 20160609]

in travis.yml

added the below to support ppc64le

matrix:
    include:
        - python: 3.7
          env: TOXENV=py37
          sudo: true
        - python: 3.8
          env: TOXENV=py38
          sudo: true
        - python: pypy
          env: TOXENV=pypy
        - python: pypy3
          env: TOXENV=pypy3
        - python: 3.7
          env: TOXENV=py37
          sudo: true
          arch : ppc64le
        - python: 3.8
          env: TOXENV=py38
          sudo: true
          arch : ppc64le
before_install:
        - pip install hypothesis

[nulano]

Hi. You seem to be missing some of Pillow's dependencies: https://travis-ci.com/github/asellappen/Pillow/jobs/477131803#L2605

--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 8.2.0.dev0
*** TKINTER support not installed
--- FREETYPE2 support ok, loaded 2.8.1
*** LITTLECMS2 support not installed
*** WEBP support not installed
*** WEBP Transparency support not installed
*** WEBPMUX support not installed
*** WEBP Animation support not installed
--- JPEG support ok, compiled for libjpeg-turbo 1.5.2
*** OPENJPEG (JPEG2000) support not installed
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
*** LIBTIFF support not installed
--- RAQM (Bidirectional Text) support ok, loaded 0.7.1
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------

I'm not sure why missing libtiff is causing the test to fail; it should probably be skipped when libtiff is not available. (Looking at the Git history, I can see the test was added quite recently, on 2020-12-22, after the last platform without libtiff was removed from CIs. The test was added in #5120.)

---

Note that this repository already runs tests on ppc64le using Github Actions, you can see that all tests except for one known failure (the other two also affect other architectures) are passing here: https://github.com/python-pillow/Pillow/runs/1774902866#step:6:2524

For reference, the source for that Docker image is here: https://github.com/python-pillow/docker-images/tree/qus/ubuntu-20.04-focal-ppc64le

[wiredfool]

Given that travis is not working with open source any more, it appears that there is a chance for IBM to step up and offer open source developers some resources to port to their platforms.

For example, having access to linux VMs on PPC or s390x would be helpful to debug big-endian issues.

[asellappen]

Hi. You seem to be missing some of Pillow's dependencies: https://travis-ci.com/github/asellappen/Pillow/jobs/477131803#L2605

--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 8.2.0.dev0
*** TKINTER support not installed
--- FREETYPE2 support ok, loaded 2.8.1
*** LITTLECMS2 support not installed
*** WEBP support not installed
*** WEBP Transparency support not installed
*** WEBPMUX support not installed
*** WEBP Animation support not installed
--- JPEG support ok, compiled for libjpeg-turbo 1.5.2
*** OPENJPEG (JPEG2000) support not installed
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
*** LIBTIFF support not installed
--- RAQM (Bidirectional Text) support ok, loaded 0.7.1
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------

I'm not sure why missing libtiff is causing the test to fail; it should probably be skipped when libtiff is not available. (Looking at the Git history, I can see the test was added quite recently, on 2020-12-22, after the last platform without libtiff was removed from CIs. The test was added in #5120.)

Note that this repository already runs tests on ppc64le using Github Actions, you can see that all tests except for one known failure (the other two also affect other architectures) are passing here: https://github.com/python-pillow/Pillow/runs/1774902866#step:6:2524

For reference, the source for that Docker image is here: https://github.com/python-pillow/docker-images/tree/qus/ubuntu-20.04-focal-ppc64le

The same lib not installed in amd64 as well,but all the test have been passed successfully.
Pls refer https://travis-ci.com/github/asellappen/Pillow/jobs/477348245

since you are saying it works in github then it should work here too,the test script written in python /PIL/Image.py having issue.

[nulano]

The same lib not installed in amd64 as well,but all the test have been passed successfully.
Pls refer https://travis-ci.com/github/asellappen/Pillow/jobs/477348245

I think you sent the wrong link? That is the ppc64le job, just in a different run. Looking at an amd64 job in that same build, I see that it does have libtiff installed: https://travis-ci.com/github/asellappen/Pillow/jobs/477348242#L2500

--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 8.2.0.dev0
--- TKINTER support ok
--- FREETYPE2 support ok, loaded 2.8.1
--- LITTLECMS2 support ok, loaded 2.9
*** WEBP support not installed
*** WEBP Transparency support not installed
*** WEBPMUX support not installed
*** WEBP Animation support not installed
--- JPEG support ok, compiled for libjpeg-turbo 1.5.2
*** OPENJPEG (JPEG2000) support not installed
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
--- LIBTIFF support ok, loaded 4.0.9
--- RAQM (Bidirectional Text) support ok, loaded 0.7.1
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------

since you are saying it works in github then it should work here too,the test script written in python /PIL/Image.py having issue.

It works in the Github Actions run because that job also has libtiff installed: https://github.com/python-pillow/Pillow/runs/1774902866#step:6:497

--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 8.2.0.dev0
--- TKINTER support ok, loaded 8.6
--- FREETYPE2 support ok, loaded 2.10.1
--- LITTLECMS2 support ok, loaded 2.9
--- WEBP support ok, loaded 0.6.1
--- WEBP Transparency support ok
--- WEBPMUX support ok
--- WEBP Animation support ok
--- JPEG support ok, compiled for libjpeg-turbo 2.0.3
--- OPENJPEG (JPEG2000) support ok, loaded 2.3.1
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
--- LIBTIFF support ok, loaded 4.1.0
--- RAQM (Bidirectional Text) support ok, loaded 0.7.1
--- LIBIMAGEQUANT (Quantization method) support ok, loaded 2.12.0
--- XCB (X protocol) support ok
--------------------------------------------------------------------

[nulano]

since you are saying it works in github then it should work here too,the test script written in python /PIL/Image.py having issue.

To clarify, I'm not saying that there is no issue. I'm just saying that the issue is not ppc64le-specific.

The issue (in Pillow) is either that:

• the test should detect that libtiff is missing and skip itself, or
• the test should not require libtiff.

[radarhere]

I agree with @nulano's analysis. Removing libtiff from our GHA Windows jobs causes a failure from the same test.

That's a problem that's worth fixing by itself - I've created PR #5232 to do so.
Since you might not be convinced though, you can test my change and see if it fixes your situation.

[radarhere]

That PR has now been merged.
If you would like to try again using our current master, you should find that it is fixed. The next release is due out at the beginning of April.
Unless you report otherwise, this issue can be closed shortly.