Do not premultiply alpha when resizing with Image.NEAREST resampling

[nulano]

Fixes #5300.

Test is based on the previous tests that make sure premultiplied alpha is used with BILINEAR resampling.

From https://pillow.readthedocs.io/en/stable/handbook/concepts.html#PIL.Image.NEAREST:

Pick one nearest pixel from the input image. Ignore all other input pixels.

I would interpret that as meaning that each pixel in the output will be equal to one of the original pixels, i.e. not introducing new values.

[radarhere]

So, in what I presume is an extreme situation to allow for ease of testing, your test demonstrates that converting the image to pre-multiplied alpha and then back to normal alpha causes a loss of information when the alpha is zero - because any color with zero alpha becomes black when converted to pre-multiplied alpha.

It may not be related, but I find that if I take one of the values from the issue, and convert it to RGBa and then back again,

>>> from PIL import Image
>>> im = Image.new("RGBA", (1, 1), (2, 2, 0, 64))
>>> im.convert("RGBa").load()[0, 0]
(1, 1, 0, 64)
>>> im.convert("RGBa").convert("RGBA").load()[0, 0]
(3, 3, 0, 64)

the roundtrip changes the value. That's odd.

[nulano]

It may not be related, but I find that if I take one of the values from the issue, and convert it to RGBa and then back again, [...] the roundtrip changes the value. That's odd.

It's not really that odd when you consider what the operation is doing. Premultiplying with alpha 64, you are rescaling each color from 0-255 to 0-64 range, effectively losing two least significant bits. Therefore there are four different values for each RGBA color band that all map to the same value in RGBa. Converting back to RGBA then chooses the "middle" value.

>>> from PIL import Image
>>> Image.new("RGBA", (1,1), (1,1,0,64)).convert("RGBa").load()[0,0]
(0, 0, 0, 64)
>>> Image.new("RGBA", (1,1), (2,2,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (3,3,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (4,4,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (5,5,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (6,6,0,64)).convert("RGBa").load()[0,0]
(2, 2, 0, 64)
>>> Image.new("RGBa", (1,1), (1,1,0,64)).convert("RGBA").load()[0,0]
(3, 3, 0, 64)

[radarhere]

Thanks. So that's why NEAREST should not roundtrip through pre-multiply conversion - since it changes the non-alpha channel values.

But have you figured out what the original purpose of it was? In other words, why it's done for any resampling methods? It's not just a shortcut method, as it actually produces different results. I would like to understand the reason so that I know why it's ok to disregard that reason in this case.

[nulano]

The reason to use RGBa is explained by @wiredfool here: #4516 (comment)

When using NEAREST downsampling, the conversion only discards some pixels keeping the ones nearest to the result image grid, while upsampling duplicates pixels.
All other resampling methods use some sort of averaging of a group of pixels near the output pixel, which can cause the colors of transparent pixels to change the colors of non-transparent pixels. See the tests above this one for an example (test_alpha_premult_resize): without premultiplication the resized image would have a partially-transparent gray band where the two regions were touching originally.

Edit: There is also an awesome blog post with excellent animations demonstrating this issue with BILINEAR resampling: http://www.adriancourreges.com/blog/2017/05/09/beware-of-transparent-pixels/

[radarhere]

All other resampling methods use some sort of averaging of a group of pixels near the output pixel

https://pillow.readthedocs.io/en/stable/handbook/concepts.html#PIL.Image.BOX

PIL.Image.BOX
Each pixel of source image contributes to one pixel of the destination image with identical weights. For upscaling is equivalent of NEAREST.

So that makes BOX an exception, yes?

[nulano]

All other resampling methods use some sort of averaging of a group of pixels near the output pixel

https://pillow.readthedocs.io/en/stable/handbook/concepts.html#PIL.Image.BOX

PIL.Image.BOX
Each pixel of source image contributes to one pixel of the destination image with identical weights. For upscaling is equivalent of NEAREST.

So that makes BOX an exception, yes?

Ah, yes, but only for upsampling. Downsampling produces an average of multiple pixels, with each included pixel having the same weight.