Add ability to not store extra attributes

[TikiTDO]

I have a scenario where I would like to create a zip that resolves to the same hash each time, similar to how the *nix zip -X flag works, but ideally more cross-platform. To that effect it would be nice to have an option to disable the storage of additional fields.

Currently I can hack it in by doing something like this, but it would be nice to just have a flag I could set.

require "zip"
require "digest"

File.write("a", "asdf")
Zip.default_compression = Zlib::NO_COMPRESSION

Zip::File.open("a.zip", Zip::File::CREATE) do |zipfile|
  a = zipfile.add("a", "a")
  a.time = ::Zip::DOSTime.new(2000)
  a.extra = ::Zip::ExtraField.new
  puts a.class
end
puts Digest::SHA2.hexdigest(File.read("a.zip"))
File.unlink("a.zip")

[jdleesmiller]

Does this (not yet merged) example help? #365

[TikiTDO]

That's pretty similar to the snippet I had above, but my main worry with such an approach is that it depends on internal APIs that may end up changing at some point in the future. Ideally this could be done as an option something like Zip::File.open(name, Zip::File::CREATE, false, reproducible: true), with a test case.

I can make a PR to make the Zip::File#add function respect an option like that using either my example code above, or the example from the PR you referenced, but I don't want to assume your preferred implementation.

[GrimmDaddy]

Zip::File.open(name, Zip::File::CREATE, false, reproducible: true), with a test case.

[GrimmDaddy]

Zip::File.open("a.zip", Zip::File::CREATE) do |zipfile|
a = zipfile.add("a", "a")
a.time = ::Zip::DOSTime.new(2000)
a.extra = ::Zip::ExtraField.new
puts a.class

[hainesr]

In terms of locking down the timestamp of a file, we're going through making sure this works correctly in #413, which may actually do what you need here once it's complete.

[hainesr]

I do think that the ability to not store extra attributes by default is a good feature to consider. I'm going to look into sensible strategies for implementing this.

I'd be interested in people's thoughts on whether this aught to be a setting applied at the archive, or individual file level, or some combination of both.

[hainesr]

While working on #421 I have convinced myself that there should be some sort of control over which extra fields are stored. I am looking into this now.

[jdleesmiller]

Thanks for the PRs, which are both now landed. I am wondering whether we are at a good place for a new release, probably 2.1.0, or whether to hold a bit longer and try to get this in, given that you've been working on it @hainesr ?

depends on internal APIs

For better or (more likely) for worse, I am not sure there is a very clear distinction between internal and external APIs in this library, at present. So, if we did change the signature of Zip::Entry.new, I think we'd have to call it a major version bump in any case.

whether this ought to be a setting applied at the archive, or individual file level, or some combination of both.

I think there are several possible implementations of this idea:

• Use mtime for each file, as suggested in Added reproducible ZIP example #365
• Use an arbitrary constant time, as suggested in Add ability to not store extra attributes #398 (comment)

and I am not sure that one is universally preferred (do you care about mtimes when you're doing the hash, or just the content?), so TBH I'm leaning toward just documenting how to do it, as in #365, rather than trying to implement one of the above. I could be swayed, however.

[jdleesmiller]

Note: I've now gone ahead and released 2.1.0.

[hainesr]

I'm working on a wholesale replacement of the extra fields code which will include the option to save none and/or strip out, etc. No guarantees about timescales - it's a gnarly area of the codebase - but I am working on it. And it will require a major version bump.