test_cmyk fails on PPC64 (and PPC32, SPARC)

[thesamesam]

What did you do?

Ran the test suite for Pillow on PPC64 (and PPC32, SPARC) with Ghostscript installed.

What did you expect to happen?

All tests pass.

What actually happened?

(Note that in the past, we had up to 10 failures on PPC64 with 7.0.0, so this is a real improvement).

One test (test_cmyk) failed on PPC64 (and PPC32, SPARC).

=================================== FAILURES ===================================
__________________________________ test_cmyk ___________________________________

    @pytest.mark.skipif(not HAS_GHOSTSCRIPT, reason="Ghostscript not available")
    def test_cmyk():
        with Image.open("Tests/images/pil_sample_cmyk.eps") as cmyk_image:

            assert cmyk_image.mode == "CMYK"
            assert cmyk_image.size == (100, 100)
            assert cmyk_image.format == "EPS"

            cmyk_image.load()
            assert cmyk_image.mode == "RGB"

            if features.check("jpg"):
                with Image.open("Tests/images/pil_sample_rgb.jpg") as target:
>                   assert_image_similar(cmyk_image, target, 10)

Tests/test_file_eps.py:75:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Tests/helper.py:134: in assert_image_similar
    raise e
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = <PIL.EpsImagePlugin.EpsImageFile image mode=RGB size=100x100 at 0x3F266A33DCD0>
b = <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=100x100 at 0x3F266A33D5E0>
epsilon = 10, msg = None

    def assert_image_similar(a, b, epsilon, msg=None):
        assert a.mode == b.mode, msg or f"got mode {repr(a.mode)}, expected {repr(b.mode)}"
        assert a.size == b.size, msg or f"got size {repr(a.size)}, expected {repr(b.size)}"

        a, b = convert_to_comparable(a, b)

        diff = 0
        for ach, bch in zip(a.split(), b.split()):
            chdiff = ImageMath.eval("abs(a - b)", a=ach, b=bch).convert("L")
            diff += sum(i * num for i, num in enumerate(chdiff.histogram()))

        ave_diff = diff / (a.size[0] * a.size[1])
        try:
>           assert epsilon >= ave_diff, (
                (msg or "")
                + f" average pixel value difference {ave_diff:.4f} > epsilon {epsilon:.4f}"
            )
E           AssertionError:  average pixel value difference 119.0294 > epsilon 10.0000
E           assert 10 >= 119.0294

Tests/helper.py:123: AssertionError

This was reported downstream in Gentoo previously for 8.1.0 but I just reproduced it with 8.1.1.

The full build log is here. I've reproduced the error within tox on PPC64 too.

What are your OS, Python and Pillow versions?

• OS: Gentoo GNU/Linux (ppc64, but also ppc32 and sparc)
• Python: Python 3.8.7
• Pillow: Pillow 8.1.1 (also 8.1.0)
• Ghostscript: 9.53.3

[ebuild R ] app-text/ghostscript-gpl-9.53.3-r5:0/9.53::gentoo USE="X cups dbus gtk jpeg2k (unicode) -static-libs" L10N="-de (-ja) -ko -zh-CN -zh-TW" 0 KiB

[nulano]

This test passes on GitHub Actions using QEMU: https://github.com/python-pillow/Pillow/runs/2007403137#step:6:730

That is using ghostscript from the ubuntu repository: https://github.com/python-pillow/docker-images/pull/98/files#diff-9dcb549928488061acac880283c65bb9900666c70a21297152f26d62ae3d2719R5

Edit: But that is using ppc64le, I missed that this issue seems to be about ppc64be.

[thesamesam]

That looks like ppc64le -- this is ppc64be (forgot to say, sorry). The commonality with ppc/ppc64be/sparc seems like it's an endianness issue.

But s390x is big-endian and I have access to that, so I'll try there. Maybe the issue is to do with our (Gentoo)'s ghostscript.

[nulano]

Looking at the linked Gentoo issue with 10 failures, note that it looks like most of those were just turned into xfails, i.e. known failures.

See e.g. the first one:

Pillow/Tests/test_file_libtiff.py

Lines 851 to 852 in 058b8d3

	@pytest.mark.xfail(is_big_endian(), reason="Fails on big-endian")
	def test_old_style_jpeg(self):

[wiredfool]

My recollection is that ghostscript tests have been flakey on bigendian dating back to ubuntu 12.04 on PPC.

[DerDakon]

Still happens in 8.3.0. There also the test_fuzz_images[Tests/images/pil_sample_cmyk.eps] test fails on Sparc.

[DerDakon]

Since this still happens on 8.3.2 for at least PPC and Sparc, what about skipping those tests on big endian as known fails? Would reduce the noise during testing for us.

[hugovk]

Sounds reasonable, would you like to create a PR?

We already have some similar xfail markers:

@pytest.mark.xfail(is_big_endian(), reason="Fails on big-endian")
...
@pytest.mark.xfail(is_ppc64le(), reason="failing on ppc64le on GHA")

[radarhere]

@thesamesam given #3243 (comment), is this resolved now?

[radarhere]

Closing, as #3243 (comment) is from the same user that reported this issue originally in https://bugs.gentoo.org/763309

[thesamesam]

@thesamesam given #3243 (comment), is this resolved now?

Confirmed that 9.0.0 passes on SPARC, PPC, and PPC64.

Thanks so much!

(I've had the tab open for about a week and started testing again earlier when you touched one of the other bugs as I remembered!)

[radarhere]

Thanks!