Intoto v0.0.2 #973

pxp928 · 2022-08-15T15:26:48Z

Summary

The current implementation of the intoto type within Rekor does not persist the signatures from the wrapping DSSE envelope into the log entry stored by Trillian. This makes it impossible to independently verify the cryptographic validity of the entry without possession of the original DSSE envelope.

Based on the discussion and design doc, it was decided to create a v0.0.2 of the intoto Rekor type with changes from v0.0.1. In addition also persisting multiple public keys and signatures.

Design Doc - https://docs.google.com/document/d/17gB598uEkoxx8j9sDhrvfhuNFHW5BSsWEiMP8POn8xo/edit?resourcekey=0-1H4eG4-4-UQYIEXZgj6AKQ#heading=h.6dq5va2kfzsw

Fixes #582

Release Note

Adds new v0.0.2 for intoto type into rekor for support of DSSE envelope with multi signature and public key

Documentation

pkg/types/intoto/v0.0.2/entry.go

bobcallaway · 2022-08-16T13:14:54Z

pkg/types/intoto/v0.0.2/entry.go

+// verifyEnvelope takes in an array of possible key bytes and attempts to parse them as x509 public keys.
+// it then uses these to verify the envelope and makes sure that every signature on the envelope is verified.
+// it returns a map of verifiers indexed by the signature the verifier corresponds to.
+func verifyEnvelope(allPubKeyBytes [][]byte, env *dsse.Envelope) (map[string]*x509.PublicKey, error) {


Is there a way we can use the DSSE signer/verifier implementation from https://pkg.go.dev/github.com/sigstore/sigstore@v1.4.0/pkg/signature/dsse instead of re-implementing it here?

Let me look to use this instead.

Using the implementation provided by sigstore does not provide the functionality in mapping the keys to the valid signature. The sigstore implementation just validates all the signatures with the pub keys but we further do mapping of the keys to sigs to create the log entry. We could create a issue to track this and update sigstore to add this functionality there?

pkg/types/intoto/v0.0.2/intoto_v0_0_2_schema.json

pkg/types/intoto/v0.0.2/entry.go

pkg/types/entries.go

pkg/types/intoto/v0.0.2/entry.go

pkg/types/intoto/v0.0.2/intoto_v0_0_2_schema.json

pxp928 · 2022-08-17T18:13:41Z

@bobcallaway Ready for another review.

codecov-commenter · 2022-08-17T18:17:34Z

Codecov Report

Merging #973 (fd253b5) into main (828f689) will increase coverage by 0.11%.
The diff coverage is 47.85%.

❗ Current head fd253b5 differs from pull request most recent head d855eea. Consider uploading reports for the commit d855eea to get more accurate results

@@            Coverage Diff             @@
##             main     #973      +/-   ##
==========================================
+ Coverage   48.06%   48.17%   +0.11%     
==========================================
  Files          61       62       +1     
  Lines        5413     5741     +328     
==========================================
+ Hits         2602     2766     +164     
- Misses       2528     2676     +148     
- Partials      283      299      +16

Impacted Files	Coverage Δ
cmd/rekor-cli/app/root.go	`20.68% <ø> (ø)`
pkg/types/entries.go	`6.25% <ø> (ø)`
pkg/types/intoto/intoto.go	`37.03% <0.00%> (ø)`
pkg/types/intoto/v0.0.2/entry.go	`43.75% <43.75%> (ø)`
cmd/rekor-cli/app/pflag_groups.go	`85.91% <76.47%> (-1.29%)`	⬇️
cmd/rekor-cli/app/pflags.go	`85.29% <85.00%> (-0.04%)`	⬇️
pkg/sharding/ranges.go	`46.01% <0.00%> (-1.26%)`	⬇️
pkg/types/rekord/v0.0.1/entry.go	`49.00% <0.00%> (+0.66%)`	⬆️
pkg/types/alpine/v0.0.1/entry.go	`56.01% <0.00%> (+2.48%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

dlorenc · 2022-08-18T13:37:04Z

Looks like something is up with codegen here....

bobcallaway · 2022-08-18T13:42:57Z

looks like some of the path names in Makefile.swagger have an extra / in them for some reason.

pxp928 · 2022-08-18T13:47:19Z

Looks like the Makefile is doing unwanted substitution?

@echo "SWAGGER_GEN=`find pkg/generated/client pkg/generated/models/ pkg/generated/restapi/ -iname '*.go' | grep -v 'configure_rekor_server' | sort -d | tr '\n' ' ' | sed 's/ $$//'`" >> Makefile.swagger;

bobcallaway · 2022-08-18T14:16:34Z

Looks like the Makefile is doing unwanted substitution?

@echo "SWAGGER_GEN=`find pkg/generated/client pkg/generated/models/ pkg/generated/restapi/ -iname '*.go' | grep -v 'configure_rekor_server' | sort -d | tr '\n' ' ' | sed 's/ $$//'`" >> Makefile.swagger;

#984 pushed to fix this

cmd/rekor-server/app/serve.go

pkg/types/intoto/intoto_schema.json

cmd/rekor-cli/app/pflag_groups.go

pkg/types/intoto/v0.0.2/intoto_v0_0_2_schema.json

dlorenc · 2022-08-18T17:32:09Z

Tests seem stuck here.

dlorenc · 2022-08-18T20:46:43Z

Github is back, you might need to force push/rebase to kick the tests though.

pxp928 · 2022-08-19T02:40:21Z

Updating the tests.

pxp928 · 2022-08-19T19:21:01Z

Any thoughts on why the e2e test fail on [GET /api/v1/log/entries/{entryUUID}][404] getLogEntryByUuidNotFound. The entry seems to get created successfully.

priyawadhwa · 2022-08-22T20:23:43Z

Any thoughts on why the e2e test fail on [GET /api/v1/log/entries/{entryUUID}][404] getLogEntryByUuidNotFound. The entry seems to get created successfully.

I ran it locally and got this error:

    e2e_test.go:527: [POST /api/v1/log/entries][400] createLogEntryBadRequest  &{Code:400 Message:Error processing entry: intoto implementation for version '0.0.1' not found: unable to locate entry for version 0.0.1}

probably related to this comment https://github.com/sigstore/rekor/pull/973/files#r948283591

pxp928 · 2022-08-23T22:21:36Z

Any thoughts on why the e2e test fail on [GET /api/v1/log/entries/{entryUUID}][404] getLogEntryByUuidNotFound. The entry seems to get created successfully.

I ran it locally and got this error:
    e2e_test.go:527: [POST /api/v1/log/entries][400] createLogEntryBadRequest  &{Code:400 Message:Error processing entry: intoto implementation for version '0.0.1' not found: unable to locate entry for version 0.0.1}
probably related to this comment https://github.com/sigstore/rekor/pull/973/files#r948283591

Still getting the same error on the e2e test: [GET /api/v1/log/entries/{entryUUID}][404] getLogEntryByUuidNotFound. Did some testing and it looks like the entry does get created. When I run it twice, it does say an entry is present already.

asraa

Thanks! This may be a very late drive-by on the schema: but I have a major question about the schema's payload and payloadHash

cmd/rekor-cli/app/search.go

asraa · 2022-08-24T17:18:56Z

pkg/types/alpine/v0.0.1/entry.go

-		if err != nil {
-			return nil, fmt.Errorf("error reading public key file: %w", err)
+	if len(publicKeyBytes) == 0 {
+		if len(props.PublicKeyPath) > 1 {


nit: for simplicity just use if len(props.PublicKeyPath) != 1? then no need for the second if clause

asraa · 2022-08-24T17:20:09Z

pkg/types/cose/v0.0.1/entry.go

-		publicKeyBytes, err = ioutil.ReadFile(filepath.Clean(props.PublicKeyPath.Path))
-		if err != nil {
-			return nil, fmt.Errorf("error reading public key file: %w", err)
+		if len(props.PublicKeyPath) > 1 {


again, could consolidate the >1 and ==1 check

asraa · 2022-08-24T17:21:20Z

pkg/types/alpine/v0.0.1/entry.go

 	} else {
-		re.AlpineModel.PublicKey.Content = (*strfmt.Base64)(&publicKeyBytes)
+		re.AlpineModel.PublicKey.Content = (*strfmt.Base64)(&publicKeyBytes[0])


should there be a warning when more than one public key byte files are provided?

yup will add.

pkg/types/intoto/v0.0.2/entry.go

Adds a DSSE envelope type to rekor. If the DSSE envelope's payload is an in-toto statement the in-toto subjects will be used as indices for the envelope's rekord. If the envelope's payload is within the server's configured attestation size the payload will be stored as an attestation. Signed-off-by: Mikhail Swift <mikhail@testifysec.com> Signed-off-by: pxp928 <parth.psu@gmail.com>

Signed-off-by: pxp928 <parth.psu@gmail.com>

dlorenc · 2022-08-28T17:30:15Z

Everything is passing! Is this on track to merge this week @bobcallaway?

bobcallaway · 2022-08-29T13:35:07Z

Everything is passing! Is this on track to merge this week @bobcallaway?

@pxp928 and I are debugging hopefully the final issue before merge now

tests/e2e_test.go

pkg/types/alpine/v0.0.1/entry.go

pkg/types/cose/v0.0.1/entry.go

pkg/types/hashedrekord/v0.0.1/entry.go

pkg/types/helm/v0.0.1/entry.go

pkg/types/intoto/v0.0.1/entry.go

pkg/types/intoto/v0.0.2/intoto_v0_0_2_schema.json

pkg/types/rekord/v0.0.1/entry.go

pkg/types/rpm/v0.0.1/entry.go

pkg/types/tuf/v0.0.1/entry.go

Signed-off-by: pxp928 <parth.psu@gmail.com>

bobcallaway

LGTM, thanks for the hard work!

priyawadhwa

nice!!

dlorenc · 2022-08-29T18:43:59Z

YESSS!

The intoto v001 type does not persist signatures of the DSSE envelope, as noted in sigstore/rekor#973. We introduced an intoto v002 type shortly after to fix this, but since then, we've introduced another newer type, DSSE v001, which also does not persist the attestation in Rekor (as we discourage using Rekor as storage). I also updated the verifier in slsa-framework/slsa-verifier#742 to search for both Rekor entry types. Signed-off-by: Hayden Blauzvern <hblauzvern@google.com>

The intoto v001 type does not persist signatures of the DSSE envelope, as noted in sigstore/rekor#973. We introduced an intoto v002 type shortly after to fix this, but since then, we've introduced another newer type, DSSE v001, which also does not persist the attestation in Rekor (as we discourage using Rekor as storage). I also updated the verifier in slsa-framework/slsa-verifier#742 to search for both Rekor entry types. # Summary ... ## Testing Process ... ## Checklist - [ ] Review the contributing [guidelines](./../CONTRIBUTING.md) - [ ] Add a reference to related issues in the PR description. - [ ] Update documentation if applicable. - [ ] Add unit tests if applicable. - [ ] Add changes to the [CHANGELOG](./../CHANGELOG.md) if applicable. --------- Signed-off-by: Hayden Blauzvern <hblauzvern@google.com> Signed-off-by: Hayden B <hblauzvern@google.com> Signed-off-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com> Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com>

pxp928 requested review from bobcallaway and a team as code owners August 15, 2022 15:26

pxp928 force-pushed the intoto_v0.0.2 branch 2 times, most recently from 3c47eb1 to 404d041 Compare August 15, 2022 15:30

bobcallaway reviewed Aug 16, 2022

View reviewed changes

pkg/types/intoto/v0.0.2/intoto_v0_0_2_schema.json Outdated Show resolved Hide resolved

pxp928 force-pushed the intoto_v0.0.2 branch 2 times, most recently from c4bc858 to d855eea Compare August 17, 2022 18:16

dlorenc mentioned this pull request Aug 18, 2022

Allow uploading detached signatures over in-toto statements with --type=intoto #865

Open

bobcallaway mentioned this pull request Aug 18, 2022

Makefile adds extra forward slashes in Makefile.Swagger #983

Closed

bobcallaway reviewed Aug 18, 2022

View reviewed changes

pxp928 force-pushed the intoto_v0.0.2 branch from d855eea to 8d465c5 Compare August 18, 2022 14:59

pxp928 force-pushed the intoto_v0.0.2 branch 2 times, most recently from 994694d to f0417be Compare August 18, 2022 21:42

pxp928 force-pushed the intoto_v0.0.2 branch 2 times, most recently from 9ddbccb to c2cc210 Compare August 19, 2022 12:55

pxp928 force-pushed the intoto_v0.0.2 branch from c2cc210 to 6449311 Compare August 23, 2022 20:18

asraa reviewed Aug 24, 2022

View reviewed changes

bobcallaway mentioned this pull request Aug 26, 2022

enable blocking specific pluggable type versions from being inserted into the log #1004

Merged

mikhailswift and others added 6 commits August 26, 2022 17:13

added intoto v0.0.2

b40583f

Signed-off-by: pxp928 <parth.psu@gmail.com>

implemented fixes and added new cli flag

d7b45ea

Signed-off-by: pxp928 <parth.psu@gmail.com>

changed public-key to slice

4811d79

Signed-off-by: pxp928 <parth.psu@gmail.com>

changes to if statements

3070576

Signed-off-by: pxp928 <parth.psu@gmail.com>

changed to PublicKeyPaths

4c10b1a

Signed-off-by: pxp928 <parth.psu@gmail.com>

pxp928 force-pushed the intoto_v0.0.2 branch from 1794b90 to 47d33ac Compare August 26, 2022 21:26

added updated based on linked PRs

77a7cde

Signed-off-by: pxp928 <parth.psu@gmail.com>

pxp928 force-pushed the intoto_v0.0.2 branch from 47d33ac to 77a7cde Compare August 26, 2022 21:59

pxp928 force-pushed the intoto_v0.0.2 branch from c0ba290 to a7010d9 Compare August 29, 2022 15:44

bobcallaway reviewed Aug 29, 2022

View reviewed changes

pxp928 force-pushed the intoto_v0.0.2 branch 2 times, most recently from 2fa4649 to 0826315 Compare August 29, 2022 16:07

added test to check for v001 blocking

390abd7

Signed-off-by: pxp928 <parth.psu@gmail.com>

pxp928 force-pushed the intoto_v0.0.2 branch from 0826315 to 390abd7 Compare August 29, 2022 16:08

bobcallaway approved these changes Aug 29, 2022

View reviewed changes

bobcallaway mentioned this pull request Aug 29, 2022

in-toto records don't contain signatures #582

Closed

priyawadhwa approved these changes Aug 29, 2022

View reviewed changes

bobcallaway merged commit 568e31a into sigstore:main Aug 29, 2022

github-actions bot added this to the v1.0.0 milestone Aug 29, 2022

asraa mentioned this pull request Sep 1, 2022

[intoto] Support intoto v0.0.2 type in rekor for upload and verification sigstore/cosign#2219

Closed

csullivanupgrade mentioned this pull request Sep 13, 2022

fix: adds envelope hash to in-toto entries in tlog entry creation sigstore/cosign#2118

Merged

pxp928 mentioned this pull request Sep 19, 2022

dsse type #596

Closed

haydentherapper mentioned this pull request Feb 29, 2024

fix: Switch to newer DSSE rekor type slsa-framework/slsa-github-generator#3299

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Intoto v0.0.2 #973

Intoto v0.0.2 #973

pxp928 commented Aug 15, 2022

bobcallaway Aug 16, 2022

pxp928 Aug 16, 2022

pxp928 Aug 17, 2022

pxp928 commented Aug 17, 2022

codecov-commenter commented Aug 17, 2022

dlorenc commented Aug 18, 2022

bobcallaway commented Aug 18, 2022

pxp928 commented Aug 18, 2022 •

edited

bobcallaway commented Aug 18, 2022

dlorenc commented Aug 18, 2022

dlorenc commented Aug 18, 2022

pxp928 commented Aug 19, 2022 •

edited

pxp928 commented Aug 19, 2022

priyawadhwa commented Aug 22, 2022

pxp928 commented Aug 23, 2022

asraa left a comment

asraa Aug 24, 2022

pxp928 Aug 25, 2022

asraa Aug 24, 2022

pxp928 Aug 25, 2022

asraa Aug 24, 2022

pxp928 Aug 24, 2022

dlorenc commented Aug 28, 2022

bobcallaway commented Aug 29, 2022

bobcallaway left a comment

priyawadhwa left a comment

dlorenc commented Aug 29, 2022

Intoto v0.0.2 #973

Intoto v0.0.2 #973

Conversation

pxp928 commented Aug 15, 2022

Summary

Release Note

Documentation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pxp928 commented Aug 17, 2022

codecov-commenter commented Aug 17, 2022

Codecov Report

dlorenc commented Aug 18, 2022

bobcallaway commented Aug 18, 2022

pxp928 commented Aug 18, 2022 • edited

bobcallaway commented Aug 18, 2022

dlorenc commented Aug 18, 2022

dlorenc commented Aug 18, 2022

pxp928 commented Aug 19, 2022 • edited

pxp928 commented Aug 19, 2022

priyawadhwa commented Aug 22, 2022

pxp928 commented Aug 23, 2022

asraa left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dlorenc commented Aug 28, 2022

bobcallaway commented Aug 29, 2022

bobcallaway left a comment

Choose a reason for hiding this comment

priyawadhwa left a comment

Choose a reason for hiding this comment

dlorenc commented Aug 29, 2022

pxp928 commented Aug 18, 2022 •

edited

pxp928 commented Aug 19, 2022 •

edited