[Jib core] Tar archives with same contents are not reproducible

[davidtron]

Environment:

• Jib version: v0.18.0-core
• Build tool: Gradle 6.01
• OS: macOS 10.15.7

Description of the issue:
I create a tar file from a registry image.
Jib.from(RegistryImage.named(image)).containerize(Containerizer.to(TarImage.at(outputPath).named(image)));

This is being executed as part of a bazel genrule, with the tar file as the output. We discovered that if this is rerun, the tar file created is different when using the same inputs, run on the same machine but at a later time.

Expected behavior:
We expect the tar file to be identical if the inputs have not changed.

Steps to reproduce:

1. Create tar file using Jib as above
2. Rename the tar file output.tar
3. Wait 5 seconds, and create another tar file using Jib as above, rename the tar output2.tar
4. shassum -a 256 output.tar
5. shassum -a 256 output2.tar

Additional Information:
The issue is due to TarStreamBuilder creating blobs using new TarArchiveEntry(name)
By default this sets the mod time of the blob to be current time, thus making the resulting Tar image not reproducible.

this.modTime = new Date().getTime() / MILLIS_PER_SECOND;

#3159 3159

[chanseokoh]

Thanks for the PR!

So a "tar image" is just a local archive (for whatever purpose), simply grabbing necessary files (such as image layer tars, container config JSON, etc) that can constitute a complete image when thought conceptually. AFAIK, there's no standard about how one should create this archive. I believe this is out of the scope of Docker or OCI specifications, because it's simply irrelevant to them. All that matters in the end is how an image can actually be stored in a registry or a Docker daemon; it's irrelevant how one can temporarily carry necessary files from one machine to another until you finally store them in a registry.

IMO, it's debatable whether Jib should force a specific timestamp (e.g., epoch) for file entries when creating a "tar image" archive. For building a reproducible container image (not a "tar image"), it's unfortunate that file timestamps in a container image affect reproducibilty according to the current Docker and OCI specifications, and that tools (including Jib) have to make a painful compromise to force a specific timestamp for files inside a container; there's no other choice, and you can't achieve image reproducibilty otherwise. If the specifications had a way to achieve reproducibility without resetting file timestamps in a container image, we (and all other tools) would certainly have kept original file timestamps. OTOH, for a "tar image" archive, nothing forces us to reset timestamps of tar entries; why erase useful information (i.e., timestamp, which many people think is important) for no good reason?

However, I know the Bazel philosophy where ensuring reproducibility of every build artifact is so fundamental to the build system, so I understand where this is coming from. AFAIK, for example, when you create a zip file in Bazel (using the general rules_pkg mechanism), the Bazel rule resets the timestamp of all the entries in the zip file to be like 1999-12-31, because that's the only way to achieve reproducibility of a zip file. This kind of behavior is really unique to the Bazel build system.

But I think it's not unreasonable to reset timestamps of tar entries when creating a "tar image." I don't think people will complain if we do so.

[mpeddada1]

@davidtron thanks again for your contribution! Jib-Core 0.19.0 has been release your fix (#3158).