chain-of-trust.md

Chain of Trust

Chain of Trust is one of our safeguards for security-sensitive tasks. It exists to provide a second factor, in addition to taskcluster scopes, to verify that:

1. Requests to run security sensitive tasks can be traced back to commits in a trusted repository.
2. The environment in which those sensitive tasks run is trusted.
3. The artifacts generated from those tasks have not been modified at rest.

If a security sensitive task depends on the artifacts generated by another task, that upstream task is also considered a security sensitive task.

Worker requirements

Currently, docker-worker and generic-worker generate artifacts used in Chain of Trust verification. These are the requirements for Chain of Trust generation.

certified.log

At the end of a task, but before uploading artifacts, copy the task log to a static log file named certified.log. We will generate checksums for this file, and upload it to public/logs/certified.log.

The purpose of certified.log is for human auditing of the logfile of the task. Because the checksum is baked into the Chain of Trust artifact, we have assurances that no one has altered this log post-upload. (If we generate a checksum for live_backing.log but continue appending to it after generating the checksum but before upload, we will have an invalid checksum for an otherwise valid artifact. If we stop logging to live_backing.log after generating its checksum but before upload, we may lose error messages or warnings post-checksum-generation.)

Chain of Trust artifact

At the end of the task, after generating certified.log, we create a Chain of Trust artifact with metadata about the task and its artifacts that allow us to verify the task, its inputs, and its outputs.

This artifact will be uploaded to public/chain-of-trust.json. We are using Chain of Trust artifact schema v1.

Details:

• chainOfTrustVersion refers to the CoT artifact schema version.

• artifacts contains a dictionary of artifacts and their checksums:

  "artifacts": {
      "public/logs/certified.log": {
          "sha256": "9877e6e5c27a859eed2c8f5b2b17e01aea9d0187bd828e603e7fa77e58b70285"
      },
      ...
  }
  

  We currently support sha256 and sha512 as valid algorithms. We're considering adding both checksums, as well as the filesize, to the Chain of Trust artifact.

  Because we expect live.log and live_backing.log to change after checksum generation, we exclude them from the artifacts dictionary.

• task contains the task definition.

• taskId contains the taskID.

• runId contains the runID.

• workerGroup, workerId, and environment contain metadata about the worker that ran the task, to allow for auditing.

  • In docker-worker tasks, we use environment.imageHash to specify the hash of the docker image that ran. There are three hashes for docker images:

    1. the artifact hash, e.g. the sha256 of image.tar.xz

    2. the image layer hash, which was used historically but allowed for image content tampering without detection, and

    3. content addressable ids, which hash up the contents of the image. We can get this via

       docker inspect --format='{{.RepoDigests}}' $IMAGE

       or

       docker images --digests

    We currently use the third option, the content addressable id / digest, as the imageHash.

  • We have included publicIpAddress, privateIpAddress, instanceId, instanceType, and region to docker-worker and generic-worker environment metadata, as metadata that could be helpful in auditing or debugging issues. There is no strict schema here at the moment, should the set of useful metadata change.

We currently prefer if the chain-of-trust.json artifact is indented for easier human readability. (We also previously hit a gpg line length limit issue, but we no longer use gpg to sign the CoT artifact.)

Chain of Trust signature

In a subset of cases, we generate a detached ed25519 signature of the Chain of Trust artifact. This signature is uploaded as public/chain-of-trust.json.sig .

Chain of Trust artifacts are not a mandatory behavior of workers, and can be configured off. Furthermore, the signature is an optional piece of the Chain of Trust feature. The signature is only needed to verify that artifacts at rest, including the Chain of Trust artifact itself, have not been tampered with. There may be a class of lower-security-sensitive tasks which can skip signature verification.

Security of the private key

Chain of Trust is only as secure as the private key. (The Chain of Trust signature shows the Chain of Trust artifact has not been tampered with.)

• This is a second factor to help secure Taskcluster scopes. Therefore, maintainers should not be able to access this secret without some credentials other than Taskcluster scopes.
• The file should be read-only, and only readable by the worker user.
• The task user must not be able to read the file.
• Restricting access to, and usage of, workers with the private key will reduce the attack surface area.
• Restricting access to the key by humans will also reduce the attack surface area.