Skip to content

Commit

Permalink
copy: compute blob compression on reused blobs based on source MediaType
Browse files Browse the repository at this point in the history
When copying blobs, if we end up reusing a blob from the destination
with the same digest as that given in the srcInfo, but don't know how
that reused blob was compressed, compute the compression information
from the MediaType included in the srcInfo, if one was supplied.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
  • Loading branch information
nalind committed Feb 15, 2021
1 parent c6e5805 commit f990c32
Showing 1 changed file with 33 additions and 0 deletions.
33 changes: 33 additions & 0 deletions copy/copy.go
Original file line number Diff line number Diff line change
Expand Up @@ -1069,6 +1069,26 @@ type diffIDResult struct {
// copyLayer copies a layer with srcInfo (with known Digest and Annotations and possibly known Size) in src to dest, perhaps (de/re/)compressing it,
// and returns a complete blobInfo of the copied layer, and a value for LayerDiffIDs if diffIDIsNeeded
func (ic *imageCopier) copyLayer(ctx context.Context, srcInfo types.BlobInfo, toEncrypt bool, pool *mpb.Progress) (types.BlobInfo, digest.Digest, error) {
// If the srcInfo doesn't contain compression information, try to compute it from the
// MediaType, which was either read from a manifest by way of LayerInfos() or constructed
// by LayerInfosForCopy(), if it was supplied at all. If we succeed in copying the blob,
// the BlobInfo we return will be passed to UpdatedImage() and then to UpdateLayerInfos(),
// which uses the compression information to compute the updated MediaType values.
// (Sadly UpdatedImage() is documented to not update MediaTypes from
// ManifestUpdateOptions.LayerInfos[].MediaType, so we are doing it indirectly.)
//
// This MIME type → compression mapping belongs in manifest-specific code in our manifest
// package (but we should preferably replace/change UpdatedImage instead of productizing
// this workaround).
if srcInfo.CompressionAlgorithm == nil {
switch srcInfo.MediaType {
case manifest.DockerV2Schema2LayerMediaType, imgspecv1.MediaTypeImageLayerGzip:
srcInfo.CompressionAlgorithm = &compression.Gzip
case imgspecv1.MediaTypeImageLayerZstd:
srcInfo.CompressionAlgorithm = &compression.Zstd
}
}

cachedDiffID := ic.c.blobInfoCache.UncompressedDigest(srcInfo.Digest) // May be ""
// Diffs are needed if we are encrypting an image or trying to decrypt an image
diffIDIsNeeded := ic.diffIDsAreNeeded && cachedDiffID == "" || toEncrypt || (isOciEncrypted(srcInfo.MediaType) && ic.c.ociDecryptConfig != nil)
Expand Down Expand Up @@ -1097,6 +1117,19 @@ func (ic *imageCopier) copyLayer(ctx context.Context, srcInfo types.BlobInfo, to
Artifact: srcInfo,
}
}

// If the reused blob has the same digest as the one we asked for, but
// the transport didn't/couldn't supply compression info, fill it in based
// on what we know from the srcInfos we were given.
// If the srcInfos came from LayerInfosForCopy(), then UpdatedImage() will
// call UpdateLayerInfos(), which uses this information to compute the
// MediaType value for the updated layer infos, and it the transport
// didn't pass the information along from its input to its output, then
// it can derive the MediaType incorrectly.
if blobInfo.Digest == srcInfo.Digest && blobInfo.CompressionAlgorithm == nil {
blobInfo.CompressionOperation = srcInfo.CompressionOperation
blobInfo.CompressionAlgorithm = srcInfo.CompressionAlgorithm
}
return blobInfo, cachedDiffID, nil
}
}
Expand Down

0 comments on commit f990c32

Please sign in to comment.