Verification: Return an STH in a Verification response

[asraa]

Description

I propose adding an STH in a rekor entry Verification.

Right now, a full verification on an entry would do the following:

1. Verify the inclusion proof using the entry Body as the leaf hash
2. Verify the root hash against an STH
3. Verify the SET

See impl here:

rekor/pkg/verify/verify.go

Line 192 in 3ee0810

if err := VerifyRootHash(ctx, rClient, verifier,

Note that most of this can be done with the returned info in an entry LogEntry EXCEPT 2. That requires a Rekor client to fetch LogInfo to retrieved an STH and then also a consistency proof from the root hash of the inclusion proof to the STH. Instead, we should return the STH for the root hash of the inclusion proof. Overall, the struct would look something like

Entry
  Verification:
    - InclusionProof
    - SignedTreeHead (correspnding to root hash of inclusion proof)
    - SignedEntryTimestamp

alternatively we can also place the SignedTreeHead checkpoint inside the inclusion proof, and remove the roothash/treesize fields and consume them from the sth.

Related #986

cc @haydentherapper @bobcallaway

Version

[haydentherapper]

Wrote up a doc with more details on offline and online verification - https://docs.google.com/document/d/1_Ykbb1n_rXQjZIairCBKj6ZTpvw5s19fjpwGTksGCbI/edit# (sigstore-dev has access)

[haydentherapper]

Task: Update verification functions once the STH is returned

[haydentherapper]

The plan for changes:

• Sign a tree head on successful inclusion
• Return the tree head in the response - I'd propose that a checkpoint field be added to InclusionProof instead of alongside InclusionProof. Then verification is either an SET or inclusion proof (A checkpoint doesn't need to be included with an SET)
• Persist the checkpoint in a database (including the current tree, size, etc)
• Change GetLogInfo to fetch the latest checkpoint from the db

The last step is necessary for gossiping. We can't sign a new tree head - Otherwise the tree head signatures would differ between an inclusion proof's checkpoint and GetLogInfo's checkpoint.

@asraa @bobcallaway @priyawadhwa Please review

[asraa]

Then verification is either an SET or inclusion proof (A checkpoint doesn't need to be included with an SET)

That makes logical sense to me. Note that I think some languages JSON decoders may have more strict validation than Go, and that it would be a breaking change and we should publish a release for it.

Persist the checkpoint in a database (including the current tree, size, etc)

Like Rekor server should do that? Can we table that for later to make this more incremental?

Change GetLogInfo to fetch the latest checkpoint from the db

Also, can we table that for later? Let's just make this issue on changing return response

[haydentherapper]

Yes, we can fix these incrementally, but I think to fully close this issue, we will need to persist the latest checkpoint. Otherwise, there will be no way for a client to get quorum with other witnesses, given the STH from an inclusion proof.

[haydentherapper]

Persistance can be punted. A client with a checkpoint can't ask a witness if it's seen the checkpoint with the current model for Rekor anyways, since the checkpoints update so frequently.

Just some thoughts for if we do this, a way to avoid writing to a database: We only need to persist the latest, so we can just store it on the global api object, with appropriate locking and checks that we aren't writing an old version. We'd need to figure out how to handle startup though.

[priyawadhwa]

@asraa I believe this can be closed now?

[asraa]

Agree!

[haydentherapper]

Thanks! Going to open up another non-blocking issue for figuring out the story around returning a consistent checkpoint.