Number of colors in palette should be maintained

[eseifert]

What did you do?

Open an image (PNG or TIFF) with mode P and save it again in the same file format.

What did you expect to happen?

The saved file should have exactly the same colors in the palette.

What actually happened?

The number of colors in the saved palette is larger than in the source palette using more space than necessary.

What are your OS, Python and Pillow versions?

• OS: Linux
• Python: 3.6.6
• Pillow: 5.3.0

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, Django, or buildout, try to replicate the issue just using Pillow.

import PIL.Image
image = PIL.Image.open('source.png')
image.save('result.png', 'PNG')
result = PIL.Image.open('result.png')
# Test
num_colors_source = len(image.palette.getdata()[1])/3
num_colors_result = len(result.palette.getdata()[1])/3
assert num_colors_source == num_colors_result  # fails

Test data

[radarhere]

The supplied PNG image has 8 colours in it. Pillow is providing a palette of the maximum length for the bit depth. Since the nearest valid bit depth is 4, the output palette is 16 (2**4 = 16). You are suggesting that we trim it further, rather than just keeping it within the valid range.

See also #2202

[caron]

I am having a similar problem with gif files. The input gif file has 64 colors in the palette, when opening the file and checking the palette size I get 256 colors. There is data in the palette which I cannot make sense of...

[118, 113, 100, 105, 100, 86, 100, 89, 75, 170, 162, 140, 117, 106, 86, 76, 70, 55, 88, 84, 71, 135, 117, 87, 133, 122, 103, 167, 148, 116, 149, 133, 103, 150, 139, 118, 122, 115, 89, 117, 101, 73, 137, 131, 116, 139, 132, 105, 106, 99, 73, 147, 129, 88, 55, 53, 45, 155, 148, 125, 83, 74, 65, 66, 60, 47, 74, 73, 66, 109, 107, 52, 0, 0, 0, 90, 89, 82, 95, 84, 56, 111, 116, 12, 122, 118, 70, 129, 131, 66, 167, 138, 92, 77, 75, 20, 178, 160, 124, 134, 136, 34, 49, 46, 18, 156, 141, 107, 94, 96, 50, 126, 117, 54, 66, 63, 27, 157, 123, 83, 106, 109, 67, 90, 99, 68, 94, 99, 3, 169, 170, 86, 95, 91, 22, 33, 27, 20, 107, 115, 84, 153, 156, 75, 193, 177, 142, 140, 144, 97, 162, 166, 108, 74, 82, 71, 140, 140, 124, 28, 27, 1, 196, 194, 158, 162, 129, 100, 206, 198, 175, 133, 99, 70, 103, 94, 94, 66, 60, 60, 133, 125, 125, 90, 74, 74, 57, 57, 57, 185, 125, 66, 64, 64, 64, 65, 65, 65,...,...,255,255,255]

I know my test data has 64 colors but I need to know this programmatically. What is the reason for expanding the palette size to the maximum? Is there a workaround?

[radarhere]

#5330 addresses the PNG part of this issue.

[radarhere]

I started putting together a change to fix the TIFF problem -

Date:   Fri Mar 26 23:43:16 2021 +1100

    Limit COLORMAP to number of colors in palette when saving

diff --git a/Tests/test_file_tiff.py b/Tests/test_file_tiff.py
index ba7f9a084..1de91fbba 100644
--- a/Tests/test_file_tiff.py
+++ b/Tests/test_file_tiff.py
@@ -616,6 +616,16 @@ class TestFileTiff:
             im.load()
             assert not fp.closed
 
+    def test_plte_length(self, tmp_path):
+        im = Image.new("P", (1, 1))
+        im.putpalette((1, 1, 1))
+
+        out = str(tmp_path / "temp.tif")
+        im.save(str(tmp_path / "temp.tif"))
+
+        with Image.open(out) as reloaded:
+            assert len(reloaded.palette.getdata()[1]) == 3
+
     # Ignore this UserWarning which triggers for four tags:
     # "Possibly corrupt EXIF data.  Expecting to read 50404352 bytes but..."
     @pytest.mark.filterwarnings("ignore:Possibly corrupt EXIF data")
diff --git a/src/PIL/TiffImagePlugin.py b/src/PIL/TiffImagePlugin.py
index 19bcf4419..0dd5bd0e8 100644
--- a/src/PIL/TiffImagePlugin.py
+++ b/src/PIL/TiffImagePlugin.py
@@ -1520,7 +1520,8 @@ def _save(im, fp, filename):
     ifd[PHOTOMETRIC_INTERPRETATION] = photo
 
     if im.mode in ["P", "PA"]:
-        lut = im.im.getpalette("RGB", "RGB;L")
+        palette_length = len(im.palette.getdata()[1])
+        lut = im.im.getpalette("RGB", "RGB;L")[:palette_length]
         ifd[COLORMAP] = tuple(v * 256 for v in lut)
     # data orientation
     stride = len(bits) * ((im.size[0] * bits[0] + 7) // 8)

But then I found that the TIFF documentation for COLORMAP states that https://www.awaresystems.be/imaging/tiff/tifftags/colormap.html

the ColorMap field for an 8-bit palette-color image would have 3 * 256 values.

It doesn't specify the flexibility that the PNG format has (see #2202). So I would conclude that we shouldn't make this change for TIFF.

[radarhere]

Similarly, the GIF specification - https://www.w3.org/Graphics/GIF/spec-gif87.txt - states that

The number of color map entries following a Screen Descriptor is equal to 2**(# bits per pixel)

So, closing. If anyone disagrees with my reasoning, feel free to comment and this can be re-opened.