Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zip without compression extracts to empty files #236

Closed
ericcj opened this issue Jun 15, 2015 · 7 comments
Closed

zip without compression extracts to empty files #236

ericcj opened this issue Jun 15, 2015 · 7 comments

Comments

@ericcj
Copy link

ericcj commented Jun 15, 2015

i created a zip using https://github.com/pixelglow/zipzap with compress:NO on the entries and it unzips fine using the normal unzip command line utility (Info-ZIP) and Archive Utility on my os x laptop. however, when i try to extract it with rubyzip it creates zero-byte files for all the entries. here's what one of the entries and its input stream looks like. sysread on that stream always returns empty string.

#<Zip::Entry:0x007f9abcab60e0 @local_header_offset=99599183, @local_header_size=nil, @internal_file_attributes=0, @external_file_attributes=2175008768, @header_signature=33639248, @version_needed_to_extract=10, @version=30, @ftype=:file, @filepath=nil, @gp_flags=2056, @follow_symlinks=false, @restore_times=true, @restore_permissions=false, @restore_ownership=false, @unix_uid=nil, @unix_gid=nil, @unix_perms=420, @dirty=false, @fstype=3, @zipfile="/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip", @name="C45664C1-ACC5-413E-9534-C7BF05DC5338.jpg", @comment="", @extra={}, @compressed_size=2239128, @crc=375996240, @compression_method=0, @size=2239128, @time=2015-06-15 16:42:50 -0400, @last_mod_time=34137, @last_mod_date=18127, @name_length=40, @extra_length=0, @comment_length=0>

#<Zip::InputStream:0x007f9ac2549a98 @lineno=0, @pos=0, @output_buffer="", @archive_io=#<File:/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip>, @decompressor=#<Zip::PassThruDecompressor:0x007f9ac25495e8 @input_stream=#<File:/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip>, @chars_to_read=0, @read_so_far=0, @has_returned_empty_string=false>, @decrypter=#<Zip::NullDecrypter:0x007f9ac25499d0>, @current_entry=#<Zip::Entry:0x007f9ac25499a8 @local_header_offset=99599183, @local_header_size=70, @internal_file_attributes=1, @external_file_attributes=0, @header_signature=67324752, @version_needed_to_extract=20, @version=10, @ftype=:file, @filepath=nil, @gp_flags=2056, @follow_symlinks=false, @restore_times=true, @restore_permissions=false, @restore_ownership=false, @unix_uid=nil, @unix_gid=nil, @unix_perms=nil, @dirty=false, @fstype=0, @zipfile=#<File:/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip>, @name="C45664C1-ACC5-413E-9534-C7BF05DC5338.jpg", @comment="", @extra={}, @compressed_size=0, @crc=0, @compression_method=0, @size=0, @time=2015-06-15 16:42:50 -0400, @last_mod_time=34137, @last_mod_date=18127, @name_length=40, @extra_length=0>>
@simonoff
Copy link
Member

Need to look on zip file structure what this library making.

@jlhonora
Copy link

I'm facing the same issue. Basically the problem is that in lib/zip/input_stream.rb:147

when @current_entry.compression_method == ::Zip::Entry::STORED
   ::Zip::PassThruDecompressor.new(@archive_io, @current_entry.size)

@current_entry.size == 0. When unpacking the values from the buffer, the size is indeed zero, lib/zip/entry.rb:

def unpack_local_entry(buf)
  puts "unpack local entry buf: #{buf.bytes.to_a.map {|c| '%02x' % c}.join(' ')}"

prints 50 4b 03 04 0a 00 08 08 00 00 96 bb fe 46 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00, so effectively size = 0 (and compressed_size = 0).

I'm using this zip to replicate it. The test case is:

  def test_extract_zipzap
    filename = 'test/data/zipzap.zip'
    ::Zip::File.open(filename) {
      |zf|
      assert_operator zf.read('evaluation.json').size, :>, 0
    }
  end

Which of course fails.

I've been trying to follow rubyzip's code in order to fix it, but this is my first time dealing with zip formats and the like. Any help is appreciated.

@pharmazone
Copy link

Is there any updates or solutions?
thnx

@herdst
Copy link

herdst commented Dec 7, 2017

I'm having the same issue. Does anyone have a solution? Thanks

@rlgomes
Copy link

rlgomes commented Mar 23, 2018

Any update here ? I have zip files with NO_COMPRESSION and basically rubzip says there are no entries when uncompressing

@ameyer-pivotal
Copy link
Contributor

@ericcj Take a look at PR #358. It looks like your general purpose bit 3 is set (2056 & 8 != 0), and this may be a fix for you too.

idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 4, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Sam Coward <scoward@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 4, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Sam Coward <scoward@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 4, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Sam Coward <scoward@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 5, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Sam Coward <scoward@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 5, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Sam Coward <scoward@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 6, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Sam Coward <scoward@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 23, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 25, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue Apr 26, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
lisamburns pushed a commit to cloudfoundry/cloud_controller_ng that referenced this issue May 3, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue May 3, 2018
- Add "stack" to buildpack model
- At migration time, buildpacks will be assigned default stack from
stacks.yml file if its path is set in the STACKS_YML env var
- hwc_buildpack is assigned the newer of windows2016,windows2012R2
stacks if present in stacks.yml
- if STACKS_YML is not set, buildpacks' stacks will be set to nil
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
idoru added a commit to idoru/cloud_controller_ng that referenced this issue May 3, 2018
[#153256959]
[#154527662]

- Add "stack" to buildpack model, which will be nil by default
- Buildpacks are now unique over name AND stack
- Sets buildpack stack from manifest.yml in buildpack zip on creation
- Validate buildpack model stack against stack in buildpack zip manifest.yml
- Validate stack exists upon buildpack bits upload
- Include stack name in serialized buildpack filename
- Only provide buildpacks for the relevant stack (or those with a nil stack) to the staging container
- Handle buildpack stacks appropriately in the buildpack installer
- Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236

NOTE: The API checkshum has changed due to adding stack as an input

Signed-off-by: Dave Goddard <dave@goddard.id.au>
Signed-off-by: Victoria Henry <vhenry@pivotal.io>
Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com>
Signed-off-by: Tyler Phelan <tphelan@pivotal.io>
Signed-off-by: Andrew Meyer <ameyer@pivotal.io>
Signed-off-by: Leah Hanson <lhanson@pivotal.io>
Signed-off-by: Eric Promislow <eric.promislow@suse.com>
Signed-off-by: Lisa Cho <lcho@pivotal.io>
@jdleesmiller
Copy link
Member

(I'm helping Alex with maintenance.) I see a 👍 and no further reports since #358 landed in 1.2.2, so I will close the ticket. Please comment if it is still a problem.

Thanks for the fix @ameyer-pivotal and @idoru.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants