Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'OSError: cannot identify image file' when opening large TIF file #3815

Closed
anpr opened this issue Apr 29, 2019 · 7 comments
Closed

'OSError: cannot identify image file' when opening large TIF file #3815

anpr opened this issue Apr 29, 2019 · 7 comments
Labels
TIFF

Comments

@anpr
Copy link

anpr commented Apr 29, 2019 •
edited

What did you do?

I tried to open this large TIF file with Pillow 6.0.0 in the following way:
image = Image.open(input_path)

What did you expect to happen?

No exception. And later I want to able to create thumbnails from the image.

What actually happened?

An exception was thrown:
OSError: cannot identify image file '/code/assets/lib/tests/test_data/alp-1978-big-tif.tif'

What are your OS, Python and Pillow versions?

  • OS: Inside a Docker container, Debian stretch 9
  • Python: Python 3.6.7
  • Pillow: 6.0.0

This is possibly related to #3372
@hugovk hugovk added the TIFF label Apr 29, 2019
@hugovk
Copy link
Member

hugovk commented Apr 29, 2019

Debugging like this:

diff --git a/src/PIL/Image.py b/src/PIL/Image.py
index 18121c6f..476187da 100644
--- a/src/PIL/Image.py
+++ b/src/PIL/Image.py
@@ -2680,6 +2680,8 @@ def open(fp, mode="r"):
                 # Leave disabled by default, spams the logs with image
                 # opening failures that are entirely expected.
                 # logger.debug("", exc_info=True)
+                if i == "TIFF":
+                    raise
                 continue
             except BaseException:
                 if exclusive_fp:
diff --git a/src/PIL/TiffImagePlugin.py b/src/PIL/TiffImagePlugin.py
index 44dde199..6f37241e 100644
--- a/src/PIL/TiffImagePlugin.py
+++ b/src/PIL/TiffImagePlugin.py
@@ -69,7 +69,7 @@ except ImportError:
 # __version__ is deprecated and will be removed in a future version. Use
 # PIL.__version__ instead.
 __version__ = "1.3.5"
-DEBUG = False  # Needs to be merged with the new logging approach.
+DEBUG = True  # Needs to be merged with the new logging approach.

 # Set these to true to force use of libtiff for reading or writing.
 READ_LIBTIFF = False

Gives this:

*** TiffImageFile._open ***
- __first: 8
- ifh:  b'II*\x00\x08\x00\x00\x00'
Seeking to frame 0, on frame -1, __next 8, location: 8
Loading tags, location: 8
tag: NewSubfileType (254) - type: long (4) - value: 0
tag: ImageWidth (256) - type: short (3) - value: 1348
tag: ImageLength (257) - type: short (3) - value: 1989
tag: BitsPerSample (258) - type: short (3) Tag Location: 58 - Data Location: 302 - value: (16, 16, 16, 16)
tag: Compression (259) - type: short (3) - value: 5
tag: PhotometricInterpretation (262) - type: short (3) - value: 5
tag: ImageDescription (270) - type: string (2) Tag Location: 94 - Data Location: 310 - value: S_ 063
tag: StripOffsets (273) - type: long (4) Tag Location: 106 - Data Location: 318 - value: <table: 332 bytes>
tag: Orientation (274) - type: short (3) - value: 1
tag: SamplesPerPixel (277) - type: short (3) - value: 4
tag: RowsPerStrip (278) - type: short (3) - value: 24
tag: StripByteCounts (279) - type: long (4) Tag Location: 154 - Data Location: 650 - value: <table: 332 bytes>
tag: XResolution (282) - type: rational (5) Tag Location: 166 - Data Location: 982 - value: 300.0
tag: YResolution (283) - type: rational (5) Tag Location: 178 - Data Location: 990 - value: 300.0
tag: PlanarConfiguration (284) - type: short (3) - value: 1
tag: ResolutionUnit (296) - type: short (3) - value: 2
tag: Software (305) - type: string (2) Tag Location: 214 - Data Location: 998 - value: Adobe Photoshop CC (Macintosh)
tag: DateTime (306) - type: string (2) Tag Location: 226 - Data Location: 1030 - value: 2019:03:25 17:51:38
tag: Artist (315) - type: string (2) Tag Location: 238 - Data Location: 1050 - value: Scanner
tag: XMP (700) - type: undefined (7) Tag Location: 250 - Data Location: 1060 - value: <table: 16780 bytes>
tag: IptcNaaInfo (33723) - type: undefined (7) Tag Location: 262 - Data Location: 17840 - value: <table: 68 bytes>
tag: PhotoshopInfo (34377) - type: undefined (7) Tag Location: 274 - Data Location: 17924 - value: <table: 7786 bytes>
tag: ExifIFD (34665) - type: long (4) - value: 29666400
tag: ICCProfile (34675) - type: undefined (7) Tag Location: 298 - Data Location: 25710 - value: <table: 1829093 bytes>
tag: NewSubfileType (254) - type: long (4) - value: 0
tag: ImageWidth (256) - type: short (3) - value: 1348
tag: ImageLength (257) - type: short (3) - value: 1989
tag: BitsPerSample (258) - type: short (3) Tag Location: 58 - Data Location: 302 - value: (16, 16, 16, 16)
tag: Compression (259) - type: short (3) - value: 5
tag: PhotometricInterpretation (262) - type: short (3) - value: 5
tag: ImageDescription (270) - type: string (2) Tag Location: 94 - Data Location: 310 - value: S_ 063
tag: StripOffsets (273) - type: long (4) Tag Location: 106 - Data Location: 318 - value: <table: 332 bytes>
tag: Orientation (274) - type: short (3) - value: 1
tag: SamplesPerPixel (277) - type: short (3) - value: 4
tag: RowsPerStrip (278) - type: short (3) - value: 24
tag: StripByteCounts (279) - type: long (4) Tag Location: 154 - Data Location: 650 - value: <table: 332 bytes>
tag: XResolution (282) - type: rational (5) Tag Location: 166 - Data Location: 982 - value: 300.0
tag: YResolution (283) - type: rational (5) Tag Location: 178 - Data Location: 990 - value: 300.0
tag: PlanarConfiguration (284) - type: short (3) - value: 1
tag: ResolutionUnit (296) - type: short (3) - value: 2
tag: Software (305) - type: string (2) Tag Location: 214 - Data Location: 998 - value: Adobe Photoshop CC (Macintosh)
tag: DateTime (306) - type: string (2) Tag Location: 226 - Data Location: 1030 - value: 2019:03:25 17:51:38
tag: Artist (315) - type: string (2) Tag Location: 238 - Data Location: 1050 - value: Scanner
tag: XMP (700) - type: undefined (7) Tag Location: 250 - Data Location: 1060 - value: <table: 16780 bytes>
tag: IptcNaaInfo (33723) - type: undefined (7) Tag Location: 262 - Data Location: 17840 - value: <table: 68 bytes>
tag: PhotoshopInfo (34377) - type: undefined (7) Tag Location: 274 - Data Location: 17924 - value: <table: 7786 bytes>
tag: ExifIFD (34665) - type: long (4) - value: 29666400
tag: ICCProfile (34675) - type: undefined (7) Tag Location: 298 - Data Location: 25710 - value: <table: 1829093 bytes>
*** Summary ***
- compression: tiff_lzw
- photometric_interpretation: 5
- planar_configuration: 1
- fill_order: 1
- YCbCr subsampling: None
- size: (1348, 1989)
format key: (b'II', 5, (1,), 1, (16, 16, 16, 16), ())
- unsupported format
Traceback (most recent call last):
  File "/Users/hugo/github/Pillow/src/PIL/TiffImagePlugin.py", line 1238, in _setup
    self.mode, rawmode = OPEN_INFO[key]
KeyError: (b'II', 5, (1,), 1, (16, 16, 16, 16), ())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "3815.py", line 3, in <module>
    image = Image.open(input_path)
  File "/Users/hugo/github/Pillow/src/PIL/Image.py", line 2692, in open
    im = _open_core(fp, filename, prefix)
  File "/Users/hugo/github/Pillow/src/PIL/Image.py", line 2676, in _open_core
    im = factory(fp, filename)
  File "/Users/hugo/github/Pillow/src/PIL/ImageFile.py", line 103, in __init__
    self._open()
  File "/Users/hugo/github/Pillow/src/PIL/TiffImagePlugin.py", line 1001, in _open
    self._seek(0)
  File "/Users/hugo/github/Pillow/src/PIL/TiffImagePlugin.py", line 1056, in _seek
    self._setup()
  File "/Users/hugo/github/Pillow/src/PIL/TiffImagePlugin.py", line 1242, in _setup
    raise SyntaxError("unknown pixel mode")
SyntaxError: unknown pixel mode

So (b'II', 5, (1,), 1, (16, 16, 16, 16), ()) isn't supported.

The supported list:

Pillow/src/PIL/TiffImagePlugin.py

Lines 145 to 273 in 7d378a8

OPEN_INFO = {
# (ByteOrder, PhotoInterpretation, SampleFormat, FillOrder, BitsPerSample,
# ExtraSamples) => mode, rawmode
(II, 0, (1,), 1, (1,), ()): ("1", "1;I"),
(MM, 0, (1,), 1, (1,), ()): ("1", "1;I"),
(II, 0, (1,), 2, (1,), ()): ("1", "1;IR"),
(MM, 0, (1,), 2, (1,), ()): ("1", "1;IR"),
(II, 1, (1,), 1, (1,), ()): ("1", "1"),
(MM, 1, (1,), 1, (1,), ()): ("1", "1"),
(II, 1, (1,), 2, (1,), ()): ("1", "1;R"),
(MM, 1, (1,), 2, (1,), ()): ("1", "1;R"),
(II, 0, (1,), 1, (2,), ()): ("L", "L;2I"),
(MM, 0, (1,), 1, (2,), ()): ("L", "L;2I"),
(II, 0, (1,), 2, (2,), ()): ("L", "L;2IR"),
(MM, 0, (1,), 2, (2,), ()): ("L", "L;2IR"),
(II, 1, (1,), 1, (2,), ()): ("L", "L;2"),
(MM, 1, (1,), 1, (2,), ()): ("L", "L;2"),
(II, 1, (1,), 2, (2,), ()): ("L", "L;2R"),
(MM, 1, (1,), 2, (2,), ()): ("L", "L;2R"),
(II, 0, (1,), 1, (4,), ()): ("L", "L;4I"),
(MM, 0, (1,), 1, (4,), ()): ("L", "L;4I"),
(II, 0, (1,), 2, (4,), ()): ("L", "L;4IR"),
(MM, 0, (1,), 2, (4,), ()): ("L", "L;4IR"),
(II, 1, (1,), 1, (4,), ()): ("L", "L;4"),
(MM, 1, (1,), 1, (4,), ()): ("L", "L;4"),
(II, 1, (1,), 2, (4,), ()): ("L", "L;4R"),
(MM, 1, (1,), 2, (4,), ()): ("L", "L;4R"),
(II, 0, (1,), 1, (8,), ()): ("L", "L;I"),
(MM, 0, (1,), 1, (8,), ()): ("L", "L;I"),
(II, 0, (1,), 2, (8,), ()): ("L", "L;IR"),
(MM, 0, (1,), 2, (8,), ()): ("L", "L;IR"),
(II, 1, (1,), 1, (8,), ()): ("L", "L"),
(MM, 1, (1,), 1, (8,), ()): ("L", "L"),
(II, 1, (1,), 2, (8,), ()): ("L", "L;R"),
(MM, 1, (1,), 2, (8,), ()): ("L", "L;R"),
(II, 1, (1,), 1, (12,), ()): ("I;16", "I;12"),
(II, 1, (1,), 1, (16,), ()): ("I;16", "I;16"),
(MM, 1, (1,), 1, (16,), ()): ("I;16B", "I;16B"),
(II, 1, (2,), 1, (16,), ()): ("I", "I;16S"),
(MM, 1, (2,), 1, (16,), ()): ("I", "I;16BS"),
(II, 0, (3,), 1, (32,), ()): ("F", "F;32F"),
(MM, 0, (3,), 1, (32,), ()): ("F", "F;32BF"),
(II, 1, (1,), 1, (32,), ()): ("I", "I;32N"),
(II, 1, (2,), 1, (32,), ()): ("I", "I;32S"),
(MM, 1, (2,), 1, (32,), ()): ("I", "I;32BS"),
(II, 1, (3,), 1, (32,), ()): ("F", "F;32F"),
(MM, 1, (3,), 1, (32,), ()): ("F", "F;32BF"),
(II, 1, (1,), 1, (8, 8), (2,)): ("LA", "LA"),
(MM, 1, (1,), 1, (8, 8), (2,)): ("LA", "LA"),
(II, 2, (1,), 1, (8, 8, 8), ()): ("RGB", "RGB"),
(MM, 2, (1,), 1, (8, 8, 8), ()): ("RGB", "RGB"),
(II, 2, (1,), 2, (8, 8, 8), ()): ("RGB", "RGB;R"),
(MM, 2, (1,), 2, (8, 8, 8), ()): ("RGB", "RGB;R"),
(II, 2, (1,), 1, (8, 8, 8, 8), ()): ("RGBA", "RGBA"), # missing ExtraSamples
(MM, 2, (1,), 1, (8, 8, 8, 8), ()): ("RGBA", "RGBA"), # missing ExtraSamples
(II, 2, (1,), 1, (8, 8, 8, 8), (0,)): ("RGBX", "RGBX"),
(MM, 2, (1,), 1, (8, 8, 8, 8), (0,)): ("RGBX", "RGBX"),
(II, 2, (1,), 1, (8, 8, 8, 8, 8), (0, 0)): ("RGBX", "RGBXX"),
(MM, 2, (1,), 1, (8, 8, 8, 8, 8), (0, 0)): ("RGBX", "RGBXX"),
(II, 2, (1,), 1, (8, 8, 8, 8, 8, 8), (0, 0, 0)): ("RGBX", "RGBXXX"),
(MM, 2, (1,), 1, (8, 8, 8, 8, 8, 8), (0, 0, 0)): ("RGBX", "RGBXXX"),
(II, 2, (1,), 1, (8, 8, 8, 8), (1,)): ("RGBA", "RGBa"),
(MM, 2, (1,), 1, (8, 8, 8, 8), (1,)): ("RGBA", "RGBa"),
(II, 2, (1,), 1, (8, 8, 8, 8, 8), (1, 0)): ("RGBA", "RGBaX"),
(MM, 2, (1,), 1, (8, 8, 8, 8, 8), (1, 0)): ("RGBA", "RGBaX"),
(II, 2, (1,), 1, (8, 8, 8, 8, 8, 8), (1, 0, 0)): ("RGBA", "RGBaXX"),
(MM, 2, (1,), 1, (8, 8, 8, 8, 8, 8), (1, 0, 0)): ("RGBA", "RGBaXX"),
(II, 2, (1,), 1, (8, 8, 8, 8), (2,)): ("RGBA", "RGBA"),
(MM, 2, (1,), 1, (8, 8, 8, 8), (2,)): ("RGBA", "RGBA"),
(II, 2, (1,), 1, (8, 8, 8, 8, 8), (2, 0)): ("RGBA", "RGBAX"),
(MM, 2, (1,), 1, (8, 8, 8, 8, 8), (2, 0)): ("RGBA", "RGBAX"),
(II, 2, (1,), 1, (8, 8, 8, 8, 8, 8), (2, 0, 0)): ("RGBA", "RGBAXX"),
(MM, 2, (1,), 1, (8, 8, 8, 8, 8, 8), (2, 0, 0)): ("RGBA", "RGBAXX"),
(II, 2, (1,), 1, (8, 8, 8, 8), (999,)): ("RGBA", "RGBA"), # Corel Draw 10
(MM, 2, (1,), 1, (8, 8, 8, 8), (999,)): ("RGBA", "RGBA"), # Corel Draw 10
(II, 2, (1,), 1, (16, 16, 16), ()): ("RGB", "RGB;16L"),
(MM, 2, (1,), 1, (16, 16, 16), ()): ("RGB", "RGB;16B"),
(II, 2, (1,), 1, (16, 16, 16, 16), ()): ("RGBA", "RGBA;16L"),
(MM, 2, (1,), 1, (16, 16, 16, 16), ()): ("RGBA", "RGBA;16B"),
(II, 2, (1,), 1, (16, 16, 16, 16), (0,)): ("RGBX", "RGBX;16L"),
(MM, 2, (1,), 1, (16, 16, 16, 16), (0,)): ("RGBX", "RGBX;16B"),
(II, 2, (1,), 1, (16, 16, 16, 16), (1,)): ("RGBA", "RGBa;16L"),
(MM, 2, (1,), 1, (16, 16, 16, 16), (1,)): ("RGBA", "RGBa;16B"),
(II, 2, (1,), 1, (16, 16, 16, 16), (2,)): ("RGBA", "RGBA;16L"),
(MM, 2, (1,), 1, (16, 16, 16, 16), (2,)): ("RGBA", "RGBA;16B"),
(II, 3, (1,), 1, (1,), ()): ("P", "P;1"),
(MM, 3, (1,), 1, (1,), ()): ("P", "P;1"),
(II, 3, (1,), 2, (1,), ()): ("P", "P;1R"),
(MM, 3, (1,), 2, (1,), ()): ("P", "P;1R"),
(II, 3, (1,), 1, (2,), ()): ("P", "P;2"),
(MM, 3, (1,), 1, (2,), ()): ("P", "P;2"),
(II, 3, (1,), 2, (2,), ()): ("P", "P;2R"),
(MM, 3, (1,), 2, (2,), ()): ("P", "P;2R"),
(II, 3, (1,), 1, (4,), ()): ("P", "P;4"),
(MM, 3, (1,), 1, (4,), ()): ("P", "P;4"),
(II, 3, (1,), 2, (4,), ()): ("P", "P;4R"),
(MM, 3, (1,), 2, (4,), ()): ("P", "P;4R"),
(II, 3, (1,), 1, (8,), ()): ("P", "P"),
(MM, 3, (1,), 1, (8,), ()): ("P", "P"),
(II, 3, (1,), 1, (8, 8), (2,)): ("PA", "PA"),
(MM, 3, (1,), 1, (8, 8), (2,)): ("PA", "PA"),
(II, 3, (1,), 2, (8,), ()): ("P", "P;R"),
(MM, 3, (1,), 2, (8,), ()): ("P", "P;R"),
(II, 5, (1,), 1, (8, 8, 8, 8), ()): ("CMYK", "CMYK"),
(MM, 5, (1,), 1, (8, 8, 8, 8), ()): ("CMYK", "CMYK"),
(II, 5, (1,), 1, (8, 8, 8, 8, 8), (0,)): ("CMYK", "CMYKX"),
(MM, 5, (1,), 1, (8, 8, 8, 8, 8), (0,)): ("CMYK", "CMYKX"),
(II, 5, (1,), 1, (8, 8, 8, 8, 8, 8), (0, 0)): ("CMYK", "CMYKXX"),
(MM, 5, (1,), 1, (8, 8, 8, 8, 8, 8), (0, 0)): ("CMYK", "CMYKXX"),
# JPEG compressed images handled by LibTiff and auto-converted to RGBX
# Minimal Baseline TIFF requires YCbCr images to have 3 SamplesPerPixel
(II, 6, (1,), 1, (8, 8, 8), ()): ("RGB", "RGBX"),
(MM, 6, (1,), 1, (8, 8, 8), ()): ("RGB", "RGBX"),
(II, 8, (1,), 1, (8, 8, 8), ()): ("LAB", "LAB"),
(MM, 8, (1,), 1, (8, 8, 8), ()): ("LAB", "LAB"),
}

@radarhere
Copy link
Member

Do you have an image that we are able to include in the test suite, under the Pillow license?

@anpr
Copy link
Author

anpr commented Apr 30, 2019

Hi,

Thanks for the investigation. I'm sorry, you can use the image to debug this, but I does not have the Pillow license. Would it be possible to add support for this anyway?

@radarhere radarhere mentioned this issue Apr 30, 2019
Merged
@radarhere
Copy link
Member

I've created PR #3817. If you could test it, that would be good.

@radarhere
Copy link
Member

Let me ask this - I presume that you're not just interested in opening a file and creating a thumbnail from it, you'd also like to then do something with that data. Save the images as files? Convert them to a different mode? Get the individual pixel values?

@anpr
Copy link
Author

anpr commented May 4, 2019

I'd like to save the thumbnail as file, exactly.

@radarhere
Copy link
Member

Closing with the merge of #3817

@anpr anpr mentioned this issue May 6, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
TIFF
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hugovk @anpr @radarhere