Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed reading APP13 marker without Photoshop data #3771

Merged
merged 1 commit into from May 4, 2019

Conversation

radarhere
Copy link
Member

@radarhere radarhere commented Apr 4, 2019

Resolves #3769

https://dev.exiv2.org/projects/exiv2/wiki/The_Metadata_in_JPEG_files

The content of an APP13 segment is formed by APP1 marker (0xFFE1), an identifier string (usually "Photoshop 3.0\000", but also 'Adobe_Photoshop2.5:', used by earlier versions) followed by a sequence of resource data blocks.

The reported issue has found an image with 'Adobe_Photoshop2.5:'. This PR fixes the logic error that results in this issue with header strings that are not 'Photoshop 3.0\x00'. The test image is from #3772 (comment)

@Meshael-S
Copy link

Thank you for sharing the solution
Does this solution included in last version of PIllow.
If not how can I update these files using juipter nootbook ?

@radarhere
Copy link
Member Author

At the moment, this is just a suggestion. It has not yet been accepted and become a part of Pillow. All things going well, it should be a part of Pillow 6.1.0 when it is released on July 1.

If you would like to include this code in your installation before that, run this code to determine the path of your Pillow's JpegImagePlugin.

>>> from PIL import JpegImagePlugin
>>> JpegImagePlugin
<module 'PIL.JpegImagePlugin' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/PIL/JpegImagePlugin.py'>

Open that file path, and change the code as seen in the 'Files changed' tab of this PR. You only need to change that file, you don't need to change the test files.

@yvesnadjarian
Copy link

I tried this update, but still getting the same error:

UnboundLocalError                         Traceback (most recent call last)
<ipython-input-10-b4302131cb35> in <module>
      1 results = model.fit_generator(train_image_gen, epochs=200, steps_per_epoch=20, 
----> 2                     validation_data=test_image_gen, validation_steps=12)

~\Anaconda3\envs\cvision\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~\Anaconda3\envs\cvision\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

~\Anaconda3\envs\cvision\lib\site-packages\keras\engine\training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    232                             val_enqueuer_gen,
    233                             validation_steps,
--> 234                             workers=0)
    235                     else:
    236                         # No need for try/except because

~\Anaconda3\envs\cvision\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~\Anaconda3\envs\cvision\lib\site-packages\keras\engine\training.py in evaluate_generator(self, generator, steps, max_queue_size, workers, use_multiprocessing, verbose)
   1470             workers=workers,
   1471             use_multiprocessing=use_multiprocessing,
-> 1472             verbose=verbose)
   1473 
   1474     @interfaces.legacy_generator_methods_support

~\Anaconda3\envs\cvision\lib\site-packages\keras\engine\training_generator.py in evaluate_generator(model, generator, steps, max_queue_size, workers, use_multiprocessing, verbose)
    328 
    329         while steps_done < steps:
--> 330             generator_output = next(output_generator)
    331             if not hasattr(generator_output, '__len__'):
    332                 raise ValueError('Output of generator should be a tuple '

~\Anaconda3\envs\cvision\lib\site-packages\keras\utils\data_utils.py in get(self)
    707                     "`use_multiprocessing=False, workers > 1`."
    708                     "For more information see issue #1638.")
--> 709             six.reraise(*sys.exc_info())

~\Anaconda3\envs\cvision\lib\site-packages\six.py in reraise(tp, value, tb)
    691             if value.__traceback__ is not tb:
    692                 raise value.with_traceback(tb)
--> 693             raise value
    694         finally:
    695             value = None

~\Anaconda3\envs\cvision\lib\site-packages\keras\utils\data_utils.py in get(self)
    683         try:
    684             while self.is_running():
--> 685                 inputs = self.queue.get(block=True).get()
    686                 self.queue.task_done()
    687                 if inputs is not None:

~\Anaconda3\envs\cvision\lib\multiprocessing\pool.py in get(self, timeout)
    642             return self._value
    643         else:
--> 644             raise self._value
    645 
    646     def _set(self, i, obj):

~\Anaconda3\envs\cvision\lib\multiprocessing\pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
    117         job, i, func, args, kwds = task
    118         try:
--> 119             result = (True, func(*args, **kwds))
    120         except Exception as e:
    121             if wrap_exception and func is not _helper_reraises_exception:

~\Anaconda3\envs\cvision\lib\site-packages\keras\utils\data_utils.py in next_sample(uid)
    624         The next value of generator `uid`.
    625     """
--> 626     return six.next(_SHARED_SEQUENCES[uid])
    627 
    628 

~\Anaconda3\envs\cvision\lib\site-packages\keras_preprocessing\image\iterator.py in __next__(self, *args, **kwargs)
     98 
     99     def __next__(self, *args, **kwargs):
--> 100         return self.next(*args, **kwargs)
    101 
    102     def next(self):

~\Anaconda3\envs\cvision\lib\site-packages\keras_preprocessing\image\iterator.py in next(self)
    110         # The transformation of images is not under thread lock
    111         # so it can be done in parallel
--> 112         return self._get_batches_of_transformed_samples(index_array)
    113 
    114     def _get_batches_of_transformed_samples(self, index_array):

~\Anaconda3\envs\cvision\lib\site-packages\keras_preprocessing\image\iterator.py in _get_batches_of_transformed_samples(self, index_array)
    224                            color_mode=self.color_mode,
    225                            target_size=self.target_size,
--> 226                            interpolation=self.interpolation)
    227             x = img_to_array(img, data_format=self.data_format)
    228             # Pillow images should be closed after `load_img`,

~\Anaconda3\envs\cvision\lib\site-packages\keras_preprocessing\image\utils.py in load_img(path, grayscale, color_mode, target_size, interpolation)
    102         raise ImportError('Could not import PIL.Image. '
    103                           'The use of `array_to_img` requires PIL.')
--> 104     img = pil_image.open(path)
    105     if color_mode == 'grayscale':
    106         if img.mode != 'L':

~\Anaconda3\envs\cvision\lib\site-packages\PIL\Image.py in open(fp, mode)
   2688         return None
   2689 
-> 2690     im = _open_core(fp, filename, prefix)
   2691 
   2692     if im is None:

~\Anaconda3\envs\cvision\lib\site-packages\PIL\Image.py in _open_core(fp, filename, prefix)
   2674                 elif result:
   2675                     fp.seek(0)
-> 2676                     im = factory(fp, filename)
   2677                     _decompression_bomb_check(im.size)
   2678                     return im

~\Anaconda3\envs\cvision\lib\site-packages\PIL\JpegImagePlugin.py in jpeg_factory(fp, filename)
    781 # Factory for making JPEG and MPO instances
    782 def jpeg_factory(fp=None, filename=None):
--> 783     im = JpegImageFile(fp, filename)
    784     try:
    785         mpheader = im._getmp()

~\Anaconda3\envs\cvision\lib\site-packages\PIL\ImageFile.py in __init__(self, fp, filename)
    101 
    102         try:
--> 103             self._open()
    104         except (IndexError,  # end of data
    105                 TypeError,  # end of data (ord)

~\Anaconda3\envs\cvision\lib\site-packages\PIL\JpegImagePlugin.py in _open(self)
    371                 name, description, handler = MARKER[i]
    372                 if handler is not None:
--> 373                     handler(self, i)
    374                 if i == 0xFFDA:  # start of scan
    375                     rawmode = self.mode

~\Anaconda3\envs\cvision\lib\site-packages\PIL\JpegImagePlugin.py in APP(self, marker)
    137                 if offset & 1:
    138                     offset += 1
--> 139         self.info["photoshop"] = photoshop
    140     elif marker == 0xFFEE and s[:5] == b"Adobe":
    141         self.info["adobe"] = i16(s, 5)

UnboundLocalError: local variable 'photoshop' referenced before assignment

@radarhere
Copy link
Member Author

Thanks for the traceback.

~\Anaconda3\envs\cvision\lib\site-packages\PIL\JpegImagePlugin.py in APP(self, marker)
    137                 if offset & 1:
    138                     offset += 1
--> 139         self.info["photoshop"] = photoshop
    140     elif marker == 0xFFEE and s[:5] == b"Adobe":

This shows that the code hasn't been updated, so you may want to try again. Alternatively, you could downgrade to Pillow 5.4.1 for the moment.

@lifez
Copy link

lifez commented Apr 25, 2019

When will we merge this fixed?

@hugovk
Copy link
Member

hugovk commented Apr 25, 2019

It'll most likely be merged before the next release on 1st July.

@koleror
Copy link

koleror commented Apr 30, 2019

Is there any chance it may be merged earlier as a patch release?

@hugovk
Copy link
Member

hugovk commented Apr 30, 2019

Patch releases are reserved for security, installation or critical bug fixes and I'm not sure this meets those criteria.

@hugovk hugovk merged commit 0ccd887 into python-pillow:master May 4, 2019
@radarhere radarhere deleted the app13 branch May 4, 2019 13:11
@Yuranus
Copy link

Yuranus commented May 5, 2019

Thanks for the traceback.

~\Anaconda3\envs\cvision\lib\site-packages\PIL\JpegImagePlugin.py in APP(self, marker)
    137                 if offset & 1:
    138                     offset += 1
--> 139         self.info["photoshop"] = photoshop
    140     elif marker == 0xFFEE and s[:5] == b"Adobe":

This shows that the code hasn't been updated, so you may want to try again. Alternatively, you could downgrade to Pillow 5.4.1 for the moment.

Thanks, downgrading Pillow to version 5.4.1 solves my problem!

@hugovk
Copy link
Member

hugovk commented May 5, 2019

This had been merged and will be in the next 6.1.0 release on 1st July.

@SaikiranKannaiah436
Copy link

At the moment, this is just a suggestion. It has not yet been accepted and become a part of Pillow. All things going well, it should be a part of Pillow 6.1.0 when it is released on July 1.

If you would like to include this code in your installation before that, run this code to determine the path of your Pillow's JpegImagePlugin.

>>> from PIL import JpegImagePlugin
>>> JpegImagePlugin
<module 'PIL.JpegImagePlugin' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/PIL/JpegImagePlugin.py'>

Open that file path, and change the code as seen in the 'Files changed' tab of this PR. You only need to change that file, you don't need to change the test files.

Changed the file accordingly and now the code runs. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

UnboundLocalError in PIL/JpegImagePlugin.py
8 participants