How to sign and verify with certificates from another CA?

[peer-jslater]

Background

We have an external root CA. It has a root certificate. Let's say the root CA signs an intermediate cert. Then that intermediate cert is used to sign a leaf cert. This leaf cert is specifically for the signing process.

root ->
intermediate ->
leaf

I want to sign a docker image using the leaf cert's private key. And I want to verify the docker image signature using the root CA's public key. The idea being that the root certificate is distributed to the users with minimal effort and the leaf cert is given to the sigstore tools but can be revoked if needed without having to issue a new root cert.

I've read as much as I could find on this. And I'm still very confused. Below is my process so far. The problem is that I have to supply the (cosign imported) public key for the LEAF cert instead of the root cert. Ideally, I wouldn't even have to import root cert to cosign since that requires access to the root cert's private key.

Approach

1. Generate the CA certificates using openssl based on Millie Smith's answer here. Additionally, make the leaf cert a CA so that Fulcio can use it to sign.
2. Start/Host Rekor locally
3. Start/Host Fulcio locally with docker compose options:

   ...
   fulcio-server:
     ...
     "--ca=fileca",
     "--fileca-cert=/etc/fulcio-config/ca_files/leaf.pem",
     "--fileca-key=/etc/fulcio-config/ca_files/.private/leaf.key",
     "--fileca-key-passwd=WRONG_PASSWORD_TO_PROVE_ITS_NOT_BEING_USED",
   

   (done according to: Setting up a fulcio instance)
   (I also need to: $Env:FULCIO_METRICS_PORT=2113 since it conflicts with rekor)
4. Follow TUF setup (I've even tried putting the public root cert in the targets folder)
5. Clone cosign and apply changes from https://github.com/haydentherapper/cosign/tree/fix-key-bytes as a patch. I know this is a work in progress but I was getting the errors on see issue bug: x509 verification broken #2632. I can elaborate if needed.
6. Import the leaf cert as a cosign key:

   cosign import-key-pair --key ~/certs/ca_files/.private/leaf.key --output-key-prefix "leaf-key-import-cosign"
   

7. Sign the key:

   cosign sign
     --key leaf-key-import-cosign.key 
     --certificate ~/certs/ca_files/without_metadata/leaf.pem 
     --certificate-chain ~/certs/ca_files/without_metadata/intermediate.pem 
     --fulcio-url http://localhost:5555 
     --rekor-url http://localhost:3000 
     $IMAGE_DIGEST
   

8. Verify the key:

   cosign verify
     --key leaf-key-import-cosign.pub 
     --rekor-url http://localhost:3000
     $IMAGE_DIGEST
   

   Verification succeeds but my main question is point 1 below. Output:

   Verification for pg-proget.peergroup.com/ifs_test/peer_ifs_poc_cosign@sha256:924fb58387ef8bf779bdc87dda7483b4aaeee23d178f7355691108495f040f08 --
   The following checks were performed on each of these signatures:
     - The cosign claims were validated
     - Existence of the claims in the transparency log was verified offline
     - The signatures were verified against the specified public key
   

Questions

1. point 8: How do I change this workflow to something like cosign verify --ca-roots root-ca.crt.pem --rekor-url http://localhost:3000 $IMAGE_DIGEST? Is this only supported once feature: 'cosign verify' add flags --ca-roots and --ca-intermediates to allow multiple CA roots #3462 gets in?
2. Does my approach achieve signing and verifying against the leaf certificate? Or does it not even achieve that?
3. point 6: Why is it okay that the password I type doesn't have to match the certificate's private key password?
4. point 6: It took me forever to find that https://docs.sigstore.dev/signing/signing_with_containers/#sign-and-attach-a-certificate-and-certificate-chain was referring to importing the key as documented here: https://docs.sigstore.dev/key_management/import-keypair/. I spent a long time trying to supply pem files as the --key for sign. Could a link to import keys be added in the sign-and-attach-a-certificate-and-certificate-chain section? (I'm asking because certificates don't seem very well supported yet and a full guide on them would be ideal but I don't know how to best change the documentation as I'm new to certs and signing).
5. point 3: Why does Fulcio need a private key but doesn't seem to use the password (since it's wrong)? I tried not using the imported key but fulcio switched to ephemeral keys (which imo is for testing according to Certificate Issuing).

[haydentherapper]

1. You can provide a chain already via --certificate-chain. The linked PR is to support multiple trusted roots and to differentiate between roots and chain building CA certificates. In the example you gave, the existing --certificate-chain flag should be sufficient. - https://docs.sigstore.dev/verifying/verify/#verify-image-with-user-provided-trusted-chain
2. Yes it would verify a certificate provided via the --certificate flag along with --certificate-chain. That will extract the key after verification.
3. I don't think you are ever hitting Fulcio, because when you provided a --key on signing, that took priority over contacting the CA to get a cert. If you want a cert for a provided key, you need to add --issue-certificate. However, given you are spinning a Fulcio instance already, do you need to be generated your own keys, or can you simply configure the client to talk to your local instance of Fulcio with --fulcio-url? Basically just drop --key from the sign command, or drop --fulcio-url if you want to just use a key. (See https://blog.sigstore.dev/adopting-sigstore-incrementally-1b56a69b8c15/ for more info)
4. Please propose a PR on the docs repo.
5. The password is used to encrypt the on-disk CA signing key. I think you've mixed up leaf and intermediate keys here. The file CA key should be the intermediate key which is certified by the root CA. The password simply encrypts that CA key. The leaf key should never be known by Fulcio, that leaf key is used to sign artifacts (or, you can let Cosign generate an ephemeral signing key for you).

[peer-jslater]

1. Regarding question 3 when I drop the --key it generates ephemeral keys.

   COSIGN_PASSWORD="" cosign sign --certificate ~/certs/ca_files/without_metadata/leaf.pem --certificate-chain ~/certs/ca_files/without_metadata/intermediate.pem --fulcio-url http://localhost:5555 --rekor-url http://localhost:3000 $IMAGE_DIGEST
   

2. I also want to know how I might avoid the oauth2.sigstore.dev as the signing authority. Can I skip the oidc process?

Edit: I've also tried adding --issue-certificate to the end of the signing command and it still only signing using ephemeral keys.

[haydentherapper]

If you want to get a certificate for a managed key, you need both --key and --issue-certificate. Without --key, an ephemeral key is used. Note that is the recommended way to use Sigstore, as removing key management is one of the goals of the project.

A token can be fetched out of band and provided with --identity-token - https://docs.sigstore.dev/signing/overview/#identity-tokens

[peer-jslater]

So far, the best workflow was:

1. Have fulcio use --ca=fileca and specify the intermediate ca cert there. This way the root trusts the certificate.
2. Use siging command:

   cosign sign
       --fulcio-url http://localhost:5555
       --rekor-url http://localhost:3000
       $IMAGE_DIGEST
   

   This does generate ephemeral keys but the intermediate certificate is included in the signature as the chain. So that's good.
3. Use verification command:

   cosign verify
       $IMAGE_DIGEST
       --certificate-identity='email you signed with'
       --certificate-oidc-issuer='https://oauth2.sigstore.dev/auth'
       --certificate-chain=path/to/intermediate.crt
   

Thanks for your help. I'll be adding the identity token you mentioned.

[haydentherapper]

One thing to mention is that certificate-chain should contain the full CA certificate chain, starting with intermediates and ending with the root. Otherwise the trust chain is built up from the intermediate.

Lemme know if you have any other questions!