Strange image shift in .bmp files

[flozirkus]

What did you do?

I opened a *.bmp file which i converted via Dataray WCF File Converter.

What did you expect to happen?

Show the file as it appears in MS Paint, MS Photo App or Affinity Photo.

What actually happened?

The content is shifted by approximately 160 pixels to the left. Missing pixels are added to the right side (seems like the image was cut in two pieces and the left part was added on the right side). The image format matches (width and height). No error was given.
I also tested the files with XnView, Irfanview and Gimp and they all show the same strange behaviour.

What are your OS, Python and Pillow versions?

• OS: Windows 20H2
• Python: Python 3.7.9
• Pillow: Pillow 8.0.1
  testfiles.zip

# -*- coding: utf-8 -*-
"""
Created on Tue Dec  8 13:54:54 2020

@author: HLO
"""
import os
from PIL import Image, ImageShow

demo = Image.open(os.path.join("0.bmp")
ImageShow.show(demo)

Annotation: My python code contains more stuff, but this example is enough to see the misbehaviour.

[radarhere]

Hi. Basic question that will probably not help in the end - but what is 'Dataray WCF File Converter'? Could you link to it?

[flozirkus]

It is a software for converting a laser beam profiling camera file (extension: *.wcf) one gets by Dataray software (when you use their own software for controlling and recording data with their beam profiling camera), you can download it here:
https://dataray-web.s3.amazonaws.com/sw/dataray-wcf-file-converter.zip

[radarhere]

My current theory is that we're not adjusting the PixelDataOffset appropriately

https://medium.com/sysf/bits-to-bitmaps-a-simple-walkthrough-of-bmp-image-format-765dc6857393

We also need to adjust PixelDataOffset the value accordingly by considering the size of the Color Pallet block.

the total size of this block is 4 x N bytes where N is the total entries in the color pallet.

[gofr]

The image is faulty, or unusual at least. The image data offset is supposed to point to the offset where the image data starts. That should've been 1078 in this image, but it instead points to the offset of the color palette at 54. So the 1024 palette values are being treated as pixels too (after Pillow already successfully read those same bytes as the palette first).

Additionally, the size header at offset 2 is supposed to be the full file size, but in this image is actually 1024 less. Perhaps software like MS Paint uses this information to correct it? Doing something like real_file_size - file_size_from_header + image_data_offset would get you to the right place in both situations... assuming this is some sort of non-standard header convention and not just a coincidence or buggy file.

Even after correcting for that I still get 12 pixels on the right that look like they should be on the left, though, and the bottom two lines of the image also look off. This doesn't look like a decoding error. Do you see that in MS Paint etc. too?

[TheDarkTiger]

You are right gofr.
To further investigate, I tried some programs:
Win7 image viewer Good
Paint XP Good
Paint Win7 Good
Fire Alpaca 1.8.11 Good
IrFanView 4.54 21bits Bad
MicrographX Picture Publisher 10.1 Bad

Errors:
BitmapFileHeader.FileSize is 1 228 854 (0x0012c036) instead of 1 229 878 (0x0012c436) -> change the byte from 0x02 to 0x05 to 0x36c41200
BitmapFileHeader.PixelDataOffset is set to 0x36. It lands right on the header -> change the byte at 0x0b to 0x04
BitmapInfoHeader.TotalColors should be 256 -> change the bytes from 0x2e to 0x31 to 0xFF000000

Warnings:
BitmapFileHeader.Reserved bytes in the header should be set to 0 -> the bytes from 0x06 to 0x09 should be 0x00000000
ColorTable.Reserved byte of your colors should be 0 (the 4th byte of each colors)

After applying the correction for the errors, the file looks the same in every program.
So, your original image is broken.
If it can help you, the 1939 first pixels are off (BMP starts at the bottom, so that's the bottom line and half the second to bottom).
In addition to that, even if we manually stitch back the image together, the 4 last pixels of each lines are wrong. More precisely, the 4th and 3rd to last pixels don't match, and the 2nd to last and last pixels are copy of the 4th and 3rd to last (respectively).
Lastly, the first and second lines from the top are copy of each other (and don't seem to perfectly match the image).
All the bad pixels have the same "range" as the image. So it's likely a bad buffer cleaning from the previous image of some overflow than random data.

@flozirkus , did you try the last version of the wcf converter?
Also, do you have the same problem with other converted images?

[radarhere]

I think the image is incorrectly formed, but I've created PR #5899 to handle it.