Unzipping with OS X's Archive Utility raises bomCopierFileCopyError

[ddefrenne]

We receive zips from our clients, which we then check to make sure only specific extensions are allowed. We also remove any hidden files and move entries when they're not in the correct place.
The client can afterwards also download his own zip (for example a colleague of him/her wants it as well).

The problem we're having is that when the zip gets extracted with OS X's Archive Utility (the default being used by Finder), it raises "Error 2 - No such file or directory". In Console.app we see "Archive Utility: bomCopierFileCopyError, error 2". We have no issues with any other extracting software.

Trying to 'fix' it wih the zip-program on cli gives us:

$ zip -FF solr_configuration.zip --out test.zip

Fix archive (-FF) - salvage what can  
 Found end record (EOCDR) - says expect single disk archive  
Scanning for entries...  
 copying: directory_under_directory/  (0 bytes)  
 copying: directory_under_directory/nl.aff  (0 bytes)  
 copying: directory_under_directory/nl.dic  (0 bytes)  
 copying: protwords.txt  
    zip warning: no end of stream entry found: protwords.txt  
    zip warning: rewinding and scanning for later entries

The code that's changing the uploaded zip:

Zip::File.open(@file.file) do |zip_file|
  zip_file.dup.each do |entry|
    if @valid_entries.include?(entry.name)
      if @single_directory.present? && entry_name_contains_directory?(entry.name)
        zip_file.rename(entry.name, entry_name_without_single_directory_part(entry))
      end
    else
      zip_file.remove(entry.name)
    end

    zip_file.commit
  end
end

[hainesr]

I've seen this problem too, and specifically with Mac Archive Utility. From what I could tell, from my minimal debugging, the issue arises when changing a zip file that already exists. We had to workaround the problem by creating a new archive and copying stuff in.

It's far from ideal, but so far it works fine: https://github.com/BioVeL/seek/blob/master/lib/taverna_player_callbacks.rb#L21

[russ]

I'm getting the same issue but I'm not modifying the zip itself, just extracting data out of it.

Zip.sort_entries = true
  Zip::ZipFile.open(full_path_with_file) do |file|
    file.each do |entry|
      next if entry.name =~ /__MACOSX/
      next if entry.name =~ /^\./

      file_path = File.join(full_path_with_group, "#{sprintf('%03d', i)}#{File.extname(entry.name)}")
      File.unlink(file_path) if File.exist?(file_path)
      entry.extract(file_path)
      save_image_metadata(file_path, entry.name, i)

      i += 1
    end
end

[alicederyn]

Whatever is producing the Zip files may be missing the data descriptor signature. It's supposed to be optional, but Archive Utility fails with error 2 if it is missing (or incorrect). See https://codingantihero.wordpress.com/2020/10/29/mac-os-x-zip-issue/ for more.

[hainesr]

I suppose that what is happening in these cases might be that the original archive has a data descriptor, then if rubyzip is used to edit the archive in some way, as it knows nothing of the data descriptor then it might be incorrect after the edits and cause these issues. That would explain the problem I had above which I solved by re-writing the archive from scratch. I guess it's time that rubyzip learned about data descriptors...

[hainesr]

@russ:

I'm getting the same issue but I'm not modifying the zip itself, just extracting data out of it.

There is an issue with Zip.sort_entries that was silently altering files even if only read from (#329). The fix has been merged but not released yet (will be in 3.0). I guess that this will have changed things in the file such that it broke it when data descriptors were present, and that triggered Archive Utility to complain.

[sandstrom]

I'm doing some issue gardening 🌱🌿 🌷 and came upon this issue. Since it's quite old I just wanted to ask if this is still relevant? If it isn't, maybe we can close this issue?

By closing some old issues we reduce the list of open issues to a more manageable set.

[hainesr]

I think this can be closed as we've got a record of needing to handle data descriptors in #460.