Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker 18.06: copyArchiveFromContainerCmd fails due to presence of gzip encoding #1079

Closed
rnorth opened this issue Aug 4, 2018 · 1 comment

Comments

@rnorth
Copy link
Contributor

rnorth commented Aug 4, 2018

We've observed an issue affecting users of Testcontainers and Docker 18.06 on Mac, whereby our attempts to copy a file from a container would fail (producing zero-length or corrupt data).

We believe that we've traced it back to the following causes:

  1. docker-java’s NettyInvocationBuilder sets an outgoing Accept-Encoding: gzip header. Until very recently docker was not actually doing anything with this header - i.e. not producing a compressed response
  2. Content negotiation to produce compressed responses were added to Docker in Content encoding negotiation added to archive request. moby/moby#36164, meaning that TAR archives will now come back with gzip/deflate compression according to the request header
  3. AFAICT, despite the Accept-Encoding request header, there is no code in docker-java to actually decompress responses

It seems to me that a simple fix is to add channel.pipeline().addLast(new HttpContentDecompressor()) in the right places in NettyInvocationBuilder. If this works for us (and our forked version of this class), I'll contribute a PR back to docker-java.

rnorth added a commit to rnorth/docker-java that referenced this issue Aug 14, 2018
ivankelly added a commit to ivankelly/pulsar that referenced this issue Jan 9, 2019
The previous version was using a version of docker-java which didn't
handle gzip compression well. So if the docker api returned an
response that was gzip compressed, it wouldn't be decompressed and
then log dumping would gzip it again, and then tar zxvf wouldn't
handle it.

See docker-java/docker-java#1079.
merlimat pushed a commit to apache/pulsar that referenced this issue Jan 9, 2019
The previous version was using a version of docker-java which didn't
handle gzip compression well. So if the docker api returned an
response that was gzip compressed, it wouldn't be decompressed and
then log dumping would gzip it again, and then tar zxvf wouldn't
handle it.

See docker-java/docker-java#1079.
@tgeorgiev
Copy link

tgeorgiev commented Oct 30, 2019

Great investigation by @rnorth ! I also started hitting this when I switched to NettyDockerCmdExecFactory. Then I realized it is time to update docker-java :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants