Each frame of an animated GIF can have a different palette

[MarkSetchell]

This StackOverflow question refers https://stackoverflow.com/q/64334674/2836621

If you look at the animated GIF in that question and analyse it with ImageMagick, you will see that each frame has a different palette:

$ identify -verbose  anim.gif | grep -A5 "Colormap:"
  Colormap:
    0: (6,2,3,1) #060203FF srgba(6,2,3,1)
    1: (253,135,19,1) #FD8713FF srgba(253,135,19,1)
    2: (29,82,41,1) #1D5229FF srgba(29,82,41,1)
    3: (44,74,95,1) #2C4A5FFF srgba(44,74,95,1)
    4: (101,134,148,1) #658694FF srgba(101,134,148,1)
--
  Colormap:
    0: (5,5,6,1) #050506FF srgba(5,5,6,1)
    1: (206,130,24,1) #CE8218FF srgba(206,130,24,1)
    2: (112,67,11,1) #70430BFF srgba(112,67,11,1)
    3: (12,70,103,1) #0C4667FF srgba(12,70,103,1)
    4: (15,135,146,1) #0F8792FF srgba(15,135,146,1)
--
  Colormap:
    0: (5,5,8,1) #050508FF srgba(5,5,8,1)
    1: (28,132,103,1) #1C8467FF srgba(28,132,103,1)
    2: (15,71,38,1) #0F4726FF srgba(15,71,38,1)
    3: (11,74,99,1) #0B4A63FF srgba(11,74,99,1)
    4: (7,137,148,1) #078994FF srgba(7,137,148,1)
--
  Colormap:
    0: (5,5,8,1) #050508FF srgba(5,5,8,1)
    1: (33,135,106,1) #21876AFF srgba(33,135,106,1)
    2: (21,71,37,1) #154725FF srgba(21,71,37,1)
    3: (11,73,100,1) #0B4964FF srgba(11,73,100,1)
    4: (8,138,145,1) #088A91FF srgba(8,138,145,1)

If you append all the frames side-by-side with ImageMagick, you will get:

convert anim.gif -coalesce +append result.png

If you analyse the same image with PIL:

import numpy as np
from PIL import Image

gif = Image.open('anim.gif')

# Append all frames together into one wide image
for i in range(img.n_frames):
   img.seek(i)
   frame = img.copy()
   print(f'Frame: {i}, palette: {frame.getpalette()[:15]}')
   frame = frame.convert('RGB')
   if i==0:
      combined = np.array(frame)
   else:
      combined = np.hstack((combined,frame))

Image.fromarray(combined).save('result.png')

You get the same palette for all frames:

Frame: 0, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 1, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 2, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 3, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 4, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 5, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 6, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 7, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 8, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 9, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 10, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 11, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 12, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 13, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 14, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 15, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]
Frame: 16, palette: [6, 2, 3, 253, 135, 19, 29, 82, 41, 44, 74, 95, 101, 134, 148]

And a big mess :-)

Using Pillow v7.2.0 on macOS with Python 3.8.

[gofr]

I think this is two separate issues.

I think getpalette() simply always gets you the image's global palette, never the frame's palette. You can get the frame's palette via frame.palette.

The noise in the image on the other hand seems to be using the correct palette, but parts of the image are garbled because those parts use the "do not dispose" "disposal method". That is, those parts of the frames are supposed to be using the pixels from the previous frame.

I think Pillow might not take into account that when copying the previous frame, it's still in palette mode, and it's not possible to have pixels from two different frames that use two different palettes while still maintaining a single palette for the whole frame...

[gofr]

Somewhat illustrative diff:

diff --git a/src/PIL/GifImagePlugin.py b/src/PIL/GifImagePlugin.py
index 4ca5a697..c3a8411d 100644
--- a/src/PIL/GifImagePlugin.py
+++ b/src/PIL/GifImagePlugin.py
@@ -304,8 +304,11 @@ class GifImageFile(ImageFile.ImageFile):
         if self._prev_im and self.disposal_method == 1:
             # we do this by pasting the updated area onto the previous
             # frame which we then use as the current image content
+            tmp = Image.new('P', self.im.size)
+            tmp.putpalette(self.im.getpalette())
             updated = self._crop(self.im, self.dispose_extent)
-            self._prev_im.paste(updated, self.dispose_extent, updated.convert("RGBA"))
+            tmp.im.paste(updated, self.dispose_extent, updated.convert("RGBA"))
+            self._prev_im = tmp.im
             self.im = self._prev_im
         self._prev_im = self.im.copy()

With this diff you'll only see the newly encoded parts of each frame (with their correct colors). Everything that's supposed to be re-used from previous frames is black. E.g. the minion Christmas hats are only encoded in the first frame. The total number of colors used in a single frame can be greater than 256 because each frame can be a composite of multiple images, each potentially using a different palette.

No idea how to fix that, though.

[RLaursen]

I'm working on a fix for this issue in my Pillow fork, here is the result so far:

!! Epilepsy Warning !!

https://user-images.githubusercontent.com/67596484/122627521-d4c63a80-d064-11eb-8f54-a150872bc612.gif

Still a bit of noise being left behind in this particular GIF, and I know why for the "Merry Christmas" part, the decoder is still leaving behind integers in the image8 array because subsequent frames are smaller than the initial frame and it doesn't change them, these integers point to arbitrary colors in the local color tables of these frames.

The total number of colors used in a single frame can be greater than 256 because each frame can be a composite of multiple images

I've decided to handle this by decoding those frames with transparency intact, but layering them on other frames once they are converted to a format that can handle more than 256 colors.

Overlaying that cat image onto existing GIF frames would require making the cat picture the first frame and creating transparency in subsequent frames where you want the cat to appear, this could cause the rest of the first frame to lack colors though and that cat image might be too colorful in itself.

[RLaursen]

update:

Just a bit of a transparency in the first frame coming from somewhere, I'll try to figure out where.

[radarhere]

I've created PR #5857 to resolve this.

Using the code and images from StackOverflow,

from PIL import Image
from io import BytesIO

animated_gif = './img/gif_distort.gif'
transparent_foreground = './img/cat.jpeg'

img = Image.open(animated_gif)
image = Image.open(transparent_foreground)

duration = []
frames = []

for i in range(img.n_frames):
    img.seek(i)
    frame = img.convert('RGBA').copy()
    duration.append(img.info['duration'])
    frame.paste(image)
    frames.append(frame)

# save gif in temp file
temp_gif_path = 'img/output.gif'
frames[0].save(temp_gif_path, format='GIF', save_all=True, append_images=frames[1:], duration=duration, optimise=True)

I get