Testsuite fails on alpine linux s390x

[ncopa]

What did you do?

Tried to rebuild the alpine linux py-pillow package with python 3.7

What did you expect to happen?

Successful build and successful testsuite run

What actually happened?

Tests/test_imagefont.py::TestImageFont::test_multiline_spacing FAILED                                          [ 81%]

====================================================== FAILURES ======================================================
________________________________________ TestImageFont.test_multiline_spacing ________________________________________

self = <test_imagefont.TestImageFont testMethod=test_multiline_spacing>

    def test_multiline_spacing(self):
        ttf = self.get_font()
    
        im = Image.new(mode='RGB', size=(300, 100))
        draw = ImageDraw.Draw(im)
        draw.multiline_text((0, 0), TEST_TEXT, font=ttf, spacing=10)
    
        target = 'Tests/images/multiline_text_spacing.png'
        target_img = Image.open(target)
    
        # Epsilon ~.5 fails with FreeType 2.7
>       self.assert_image_similar(im, target_img, self.metrics['multiline'])

Tests/test_imagefont.py:267: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_imagefont.TestImageFont testMethod=test_multiline_spacing>
a = <PIL.Image.Image image mode=RGB size=300x100 at 0x7F14F42FE9D0>
b = <PIL.PngImagePlugin.PngImageFile image mode=RGB size=300x100 at 0x7F14F42FEC10>, epsilon = 0.5, msg = None

    def assert_image_similar(self, a, b, epsilon, msg=None):
        epsilon = float(epsilon)
        self.assertEqual(
            a.mode, b.mode,
            msg or "got mode %r, expected %r" % (a.mode, b.mode))
        self.assertEqual(
            a.size, b.size,
            msg or "got size %r, expected %r" % (a.size, b.size))
    
        a, b = convert_to_comparable(a, b)
    
        diff = 0
        for ach, bch in zip(a.split(), b.split()):
            chdiff = ImageMath.eval("abs(a - b)", a=ach, b=bch).convert('L')
            diff += sum(i * num for i, num in enumerate(chdiff.histogram()))
    
        ave_diff = float(diff)/(a.size[0]*a.size[1])
        try:
            self.assertGreaterEqual(
                epsilon, ave_diff,
                (msg or '') +
                " average pixel value difference %.4f > epsilon %.4f" % (
                    ave_diff, epsilon))
        except Exception as e:
            if HAS_UPLOADER:
                try:
                    url = test_image_results.upload(a, b)
                    logger.error("Url for test images: %s" % url)
                except Exception:
                    pass
>           raise e
E           AssertionError:  average pixel value difference 6.1854 > epsilon 0.5000

Tests/helper.py:154: AssertionError
================================= 1 failed, 964 passed, 68 skipped in 21.24 seconds ==================================

What are your OS, Python and Pillow versions?

• OS: Alpine linux edge (rolling release) x86, x86_64 and ppc64le
• Python: 3.7.3
• Pillow: 5.4.1

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, Django, or buildout, try to replicate the issue just using Pillow.

code goes here

[radarhere]

Would you be able to update to Pilow 6.0.0, and try again? I would guess that this is failing with freetype 2.10.0, and Pillow 6.0.0 includes an update for that. Our arch build started failing with this error just before updating to Pillow 6.0.0.

[ncopa]

seems like tests fails on s390x: https://build.alpinelinux.org/buildlogs/build-edge-s390x/main/py-pillow/py-pillow-6.0.0-r0.log

[tuan-hoang1]

It seems it fails on s390x in general on both Ubuntu and Fedora.

[tuan-hoang1]

I believe it relates to : #1204

[radarhere]

Interestingly, we now have separate alpine and s390x jobs in our testing matrix, and they pass for this test.

[radarhere]

Note that the linked log in an earlier comment no longer works.

[radarhere]

In Pillow master, we have now dropped support for FreeType 2.7, and updated the test image that is failing here.

If I try and run alpine s390x in Docker, it only has 1 failing test.

That's a timeout failure though, and I don't think it's due to a problem, but just because emulation is slow. Increasing the timeout to 10s allows it to pass.

So unless anyone says that they're still experiencing this problem, I don't think there's anything more to do here.