How to exhaust memory with one reputably sourced image

[JackDandy]

Using hachoir commit #5b9e05a on Windows 10 x64.

Steps...

1. Save the code below as test.py
2. Downloaded the only image in section BACKGROUND from the reputable source
   https://fanart.tv/series/331821/the-looming-tower/
   (direct image link is https://fanart.tv/api/download.php?type=download&image=88183&section=1)
3. Save image file to same folder as the test code as fanart.jpg (the jpg image crc is BA866C09)
4. Run Python -V # output Python 3.7.4 (32 bit)
5. Run python.exe test.py

Result:

infinite loop exhausting memory until Python crashes with MemoryError.

[warn] [/exif/content/ifd[0]] [Autofix] Fix parser error: stop parser, found unparsed segment: start 1408, length 8, found unparsed segment: start 1480, length 8, found unparsed segment: start 1552, length 8, found unparsed segment: start 1712, length 16
[warn] [/exif/content/ifd[1]] [Autofix] Fix parser error: stop parser, found unparsed segment: start 1408, length 8, found unparsed segment: start 1480, length 8, found unparsed segment: start 1552, length 8, found unparsed segment: start 1712, length 16
[warn] [/exif/content/ifd[2]] [Autofix] Fix parser error: stop parser, found unparsed segment: start 1408, length 8, found unparsed segment: start 1480, length 8, found unparsed segment: start 1552, length 8, found unparsed segment: start 1712, length 16
...
and so on...
...
[warn] [/exif/content/ifd[231]] [Autofix] Fix parser error: stop parser, found unparsed segment: start 1408, length 8, found unparsed segment: start 1480, length 8, found unparsed segment: start 1552, length 8, found unparsed segment: start 1712, length 16
...
and so on...

Test code

test.py script located where cloned hachoir3 (or named whatever) folder is located.

import os
import sys
  
HACHOIR_CLONE_PATH = 'hachoir3'  
  
sys.path.insert(1, os.path.abspath(os.path.join(os.path.dirname(__file__),
                                   HACHOIR_CLONE_PATH)))  
  
from hachoir import parser
from hachoir import metadata

path = os.path.abspath(os.path.join(os.path.dirname(__file__), 'fanart.jpg'))

try:
    parser = parser.createParser(path)
    metadata = metadata.extractMetadata(parser)
except Exception as e:
    print('Unable to extract metadata %r' % e)

[vstinner]

Yeah, Hachoir has bugs, but it's basically no longer maintained: http://unmaintained.tech/

If you propose a fix, I can try to review it.

[nneonneo]

Thanks for the bug report. It’s an infinite loop in the IFD generator caused by a bogus next pointer in IFD0 which points back to IFD0. The easy fix is probably just to ban recursion with a simple visited-set check, but I’m a little worried you could still DOS this by specifying “slid” offsets. In theory the number of IFDs should be finite - a cursory reading of the spec suggests the limit is 3 (IFD0, IFD1, IFD2). I’m a teensy bit hesitant to set this limit in case arbitrary linked IFDs was an intentional feature (although, for example, libexif only parses the first two). I can put together a simple patch that fixes this issue pretty quickly - I already have it tested and working locally. Robert

…

On Aug 27, 2019, at 12:50 AM, Victor Stinner ***@***.***> wrote: Yeah, Hachoir has bugs, but it's basically no longer maintained: http://unmaintained.tech/ If you propose a fix, I can try to review it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[vstinner]

The problem is that Hachoir allows to create 2 fields at the same address:

 + 0) start_image: Start of image (SOI) (2 bytes)
 - 2) exif: EXIF (292 bytes)
      0) header= 0xff: Header (1 bytes)
      1) type= 0xe1: Type (1 bytes)
      2) size= 290: Size (2 bytes)
    - 4) content: Chunk content (288 bytes)
         0) header= "Exif\0\0": Header (Exif\0\0) (6 bytes)
         6) endian= "MM": Endian ('II' or 'MM') (2 bytes)
         8) version= 42: TIFF version number (2 bytes)
         10) img_dir_ofs= 8: Next image directory offset (4 bytes)
       + 14) ifd[0] (216 bytes)
       + 234) exif[0] (54 bytes)
       + 14) ifd[1] (216 bytes)
       + 234) exif[1] (54 bytes)
       + 14) ifd[2] (216 bytes)
       + 234) exif[2] (54 bytes)
       + 14) ifd[3] (216 bytes)
       ...

It should raise an error.

[nneonneo]

Yes, that would be a problem with the implementation of RootSeekableFieldSet. Perhaps the right solution then is to ban the creation of duplicate fields there. Keep in mind that RSFS’s design allows you to skip around in the file, but right now we don’t ban overlapping fields (maybe we should). Robert

…

On Aug 28, 2019, at 1:55 PM, Victor Stinner ***@***.***> wrote: The problem is that Hachoir allows to create 2 fields at the same address: + 0) start_image: Start of image (SOI) (2 bytes) - 2) exif: EXIF (292 bytes) 0) header= 0xff: Header (1 bytes) 1) type= 0xe1: Type (1 bytes) 2) size= 290: Size (2 bytes) - 4) content: Chunk content (288 bytes) 0) header= "Exif\0\0": Header (Exif\0\0) (6 bytes) 6) endian= "MM": Endian ('II' or 'MM') (2 bytes) 8) version= 42: TIFF version number (2 bytes) 10) img_dir_ofs= 8: Next image directory offset (4 bytes) + 14) ifd[0] (216 bytes) + 234) exif[0] (54 bytes) + 14) ifd[1] (216 bytes) + 234) exif[1] (54 bytes) + 14) ifd[2] (216 bytes) + 234) exif[2] (54 bytes) + 14) ifd[3] (216 bytes) ... It should raise an error. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[mengyyy]

maybe the same problem LonamiWebs/Telethon#1385