Generalize signature copy support #1593

mtrmac · 2022-07-05T18:06:46Z

OK, let’s start cashing in the infrastructure work.

This introduces internal.Signature, as a way to represent multiple signature formats, and GetSignaturesWithFormat and PutSignaturesWithFormat, and teaches copy.Image to use it.

Note that it doesn’t actually implement Cosign signatures yet.

Note: This makes implementation choices, which is why I’m filling this PR early.

Defines a long-term, on-disk, generic format for storing various signature formats.
dir, docker *lookaside, ostree and storage are going to store the generic-format blobs, and can represent any future formats of signatures.
docker X-R-S-S and OpenShift only accept simple signatures, and fail when copying the other kind.

~~Depends on unmerged #1592.~~

mtrmac · 2022-07-06T14:23:20Z

Encapsulated the signature data to make sure underlying slices are not shared/modified
Moved the new signature.FromBlob to this PR.
Added unit tests for internal/signature (but did not check if the transport changes would benefit from test expansion)
Actually fixed signature.Blob/signature.FromBlob to work with the format-tagged representation.

I think this is now ready for review. The transport test coverage review is a known deficit.

vrothberg

LGTM, merge at will

internal/signature/signature.go

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

We want the "signatures" package name to be visible. Signed-off-by: Miloslav Trmač <mitr@redhat.com>

err is necessarily nil at that location, so this was returning no error indication. Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

We need something to wrap (format ID, actual data). And a shared helper for a file/blob representation for things like dir or c/storage. This looks quite immature. Tests of the non-simple signing will only be added with support for that format. Signed-off-by: Miloslav Trmač <mitr@redhat.com>

NOTE: This makes implementation choices: - dir, docker LOOKASIDE, ostree and storage are going to store the generic-format blobs, and can represent any future formats of signatures. - docker X-R-S-S and OpenShift only accept simple signatures, and fail when copying the other kind. Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

vrothberg

LGTM

copy.Image can now copy non-image OCI artifacts. Added support for sigstore signatures: they (and related cosign attachments) can be copied along with images after opt-in in registries.d. Signatures can be created by copy.Image and enforced via policy.json (currently with public/private key pairs only). Now requires Go 1.17. GPGMe now must be new enough to be visible via pkg-config. github.com/pkg/errors is no longer used; that might affect caller-observable error types (in particular, errors.{As,Is} might need to be used instead of pkg/errors.Cause). Changes default paths on FreeBSD. - Remove unused Makefile variables - Config files should live in /usr/local on FreeBSD - docker: validate received parts - Use go env to fetch the go path - docker: add workaround for CloudFront - Improve errors messages when image missing from list - Stop calling gpgme-config - Fix codespell errors - Make sure github.com/opencontainers/runc >= 1.1.2 is used - Cirrus: use Ubuntu 22.04 LTS - Merge pull request containers#1576 from mtrmac/private-image - Merge pull request containers#1577 from mtrmac/mocks - Merge pull request containers#1571 from mtrmac/go1.17 - Merge pull request containers#1578 from mtrmac/sourced-image-struct - Fix error on parallel multiple image pullings with additionallayerstore - Merge pull request containers#1579 from mtrmac/copy-layers-refactor - Reject OCI artifacts in manifest.OCI1.ImageID - Reject OCI artifacts in manifest.OCI1.Inspect - Refuse to convert non-image OCI artifacts to Docker formats - Reject OCI artifacts in image.manifestOCI1.OCIConfig - Introduce SourcedImage.CanChangeLayerCompression, use it in copy.Image - Use an updated CI image - Use strings.ReplaceAll instead of strings.Replace(..., -1) - Move the main helper removal case to the main path on RemoveAllAuthentication - Merge pull request containers#1588 from mtrmac/pkg_errors - Merge pull request containers#1589 from mtrmac/private-dest-impls - Merge pull request containers#1590 from mtrmac/private-src-impls - Merge pull request containers#1592 from mtrmac/blobcache-wrap-private - Use "io.ReadAll" instead of "os.ReadAll" - Merge pull request containers#1596 from mtrmac/cosign-payload - Generalize copy.Image to be able to copy signatures with any format - Merge pull request containers#1593 from mtrmac/cosign-sigs - Introduce signature.Cosign as a format - Add use-cosign-attachments to registries.d/*.yaml - Add support for reading and writing Cosign attachments, incl. signatures - Merge pull request containers#1595 from mtrmac/cosign-docker - Add support for creating Cosign signatures - Fix a long-standing incorrect comment - Fix JSON syntax in the policy.json(5) man page - Correctly decode Cosign-generated payloads - Add Cosign verification support - s/sigstore/lookaside/g in comments and documentation - Refer to lookasideStorage instead of signatureStorage in code - Add lookaside and lookaside-staging, hide sigstore and sigstore-staging - Merge pull request containers#1605 from mtrmac/sigstore - Fix a typo in error messages - Remove a copy&pasted test entry - Add context to some test failures - Use more valid data in TestPRSignedByIsSignatureAuthorAccepted - Generalize keyPath/keyData exclusivity checks - Remove repetition in tests - Accept multiple keyrings in newEphemeralGPGSigningMechanism - Allow accepting multiple GPG keyrings via signedBy.keyPaths - Switch to golang native error wrapping - Point out use-sigstore-registries in sigstoreSigned documentation - Use .pub extension for public keys in sigstoreSigned examples - copy: print copy info once when writer==io.Discard - Silence a "potentially unused parameter" warning - Read signatures from UnparsedImage instead of ImageSource directly - Consolidate reading messages, and checking for support, into a helper - build(deps): bump github.com/containers/storage from 1.40.0 to 1.40.2 - build(deps): bump github.com/docker/docker - build(deps): bump github.com/klauspost/compress from 1.15.2 to 1.15.3 - build(deps): bump github.com/klauspost/compress from 1.15.3 to 1.15.4 - build(deps): bump github.com/docker/docker - build(deps): bump github.com/proglottis/gpgme from 0.1.1 to 0.1.2 - build(deps): bump github.com/vbauerster/mpb/v7 from 7.4.1 to 7.4.2 - build(deps): bump github.com/imdario/mergo from 0.3.12 to 0.3.13 - build(deps): bump github.com/klauspost/compress from 1.15.4 to 1.15.5 - build(deps): bump github.com/sylabs/sif/v2 from 2.7.0 to 2.7.1 - build(deps): bump github.com/klauspost/compress from 1.15.5 to 1.15.6 - build(deps): bump github.com/stretchr/testify from 1.7.1 to 1.7.2 - build(deps): bump github.com/docker/docker - build(deps): bump github.com/stretchr/testify from 1.7.2 to 1.7.4 - build(deps): bump github.com/stretchr/testify from 1.7.4 to 1.7.5 - build(deps): bump github.com/stretchr/testify from 1.7.5 to 1.8.0 - build(deps): bump github.com/klauspost/compress from 1.15.6 to 1.15.7 - build(deps): bump github.com/proglottis/gpgme from 0.1.2 to 0.1.3 - build(deps): bump github.com/klauspost/compress from 1.15.7 to 1.15.8 - build(deps): bump github.com/sirupsen/logrus from 1.8.1 to 1.9.0 - build(deps): bump github.com/theupdateframework/go-tuf - build(deps): bump github.com/BurntSushi/toml from 1.1.0 to 1.2.0 Signed-off-by: Miloslav Trmač <mitr@redhat.com>

mtrmac force-pushed the cosign-sigs branch 2 times, most recently from 031a819 to 287c3cd Compare July 5, 2022 21:21

This was referenced Jul 5, 2022

Add a Cosign signature format carrier #1594

Merged

DO NOT MERGE IRRESPONSIBLE: Cosign integration #1599

Closed

mtrmac force-pushed the cosign-sigs branch 2 times, most recently from 6be75d0 to 0bbad4a Compare July 6, 2022 14:17

mtrmac marked this pull request as ready for review July 6, 2022 14:23

vrothberg reviewed Jul 7, 2022

View reviewed changes

internal/signature/signature.go Outdated Show resolved Hide resolved

mtrmac added 8 commits July 7, 2022 13:30

Introduce ImageSourceInternalOnly

43bd794

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Rename local variables in storageImageSource.GetSignatures

1b6a4bf

We want the "signatures" package name to be visible. Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Fix error handling

ba768be

err is necessarily nil at that location, so this was returning no error indication. Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Rename a variable to be consistent

f1f6bbc

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Introduce ImageSource.GetSignaturesWithFormat

ad2c293

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Generalize copy.Image to be able to copy signatures with any format

4b4a5be

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

mtrmac force-pushed the cosign-sigs branch from 0bbad4a to 4b4a5be Compare July 7, 2022 11:46

vrothberg approved these changes Jul 7, 2022

View reviewed changes

mtrmac merged commit cae5559 into containers:main Jul 7, 2022

mtrmac deleted the cosign-sigs branch July 7, 2022 12:02

mtrmac mentioned this pull request Jul 21, 2022

v5.22.0 #1619

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalize signature copy support #1593

Generalize signature copy support #1593

mtrmac commented Jul 5, 2022 •

edited

mtrmac commented Jul 6, 2022

vrothberg left a comment

vrothberg left a comment

Generalize signature copy support #1593

Generalize signature copy support #1593

Conversation

mtrmac commented Jul 5, 2022 • edited

mtrmac commented Jul 6, 2022

vrothberg left a comment

Choose a reason for hiding this comment

vrothberg left a comment

Choose a reason for hiding this comment

mtrmac commented Jul 5, 2022 •

edited