Digest output missing for buildx

[JensRantil]

Troubleshooting

Before submitting a bug report please read the Troubleshooting doc. ✅

Behaviour

I have a Github actions workflow that previously has been using build-push-action@v4 without setup-buildx-action and relying on outputs.digest from the build-push-action (to know which Docker image to deploy). When adding

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

the digest is no longer output from build-push-action@v4 and instead empty.

Steps to reproduce this issue

Unfortunately what I am working on is closed source so I can't share this.

1. Create a workflow with
   1. First step calling setup-buildx-action@v2.
   2. Second step building a Docker image using build-push-action@v4. In my case I am also pushing the image & have GHA caching enabled (not sure that's relevant here, though). Give the step id: dockerbuild.
   3. Third step, echo Digest: ${{ steps.dockerbuild.outputs.digest }}.
2. Execute the workflow.
3. Notice how the echo output is empty.

Expected behaviour

The echo output should contain the digest, not an empty string.

Actual behaviour

The output is Digest: with no digest.

Configuration

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Build and push Docker image
        id: build-docker
        uses: docker/build-push-action@v4
        with:
          file: Dockerfile
          tags: "my-service:latest"
          cache-from: type=gha
          cache-to: type=gha,mode=max
      - name: Summarize the Docker digest
        run: |
          echo 'Digest: ${{ steps.build-docker.outputs.digest }}'

Logs

Not available.

[crazy-max]

Would need logs from your workflow and/or link to your repo to help you out.

[miguelvr]

I'm having the same problem!

[miguelvr]

solved by passing in load: true

this should be documented, because both imageid and digest outputs show up as empty, and the metadata json is incomplete

[ryzr]

Currently having the same issue. In my case, we can't use load and output together.

name: Publish Image (8.2)

on:
  push:
    branches: [main]
    paths:
      - 8.2/**

env:
  REGISTRY_IMAGE: public.ecr.aws/luminsports/php-runtime

jobs:
  build:
    name: Build

    runs-on: ${{ matrix.runs-on }}

    strategy:
      fail-fast: false
      matrix:
        platform: [linux/amd64, linux/arm64]
        include:
          - platform: linux/amd64
            runs-on: [self-hosted, linux, x64]
          - platform: linux/arm64
            runs-on: [self-hosted, linux, arm64]

    steps:
      - name: Check out the repo
        uses: actions/checkout@v3

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ env.REGISTRY_IMAGE }}
          tags: |
            type=ref,event=branch,prefix=8.2-
            type=sha,prefix=8.2-
            type=raw,value=8.2

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Login to Public ECR
        uses: docker/login-action@v2
        with:
          registry: public.ecr.aws
          username: ${{ secrets.AWS_ACCESS_KEY_ID }}
          password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        env:
          AWS_REGION: 'us-east-1'

      - name: Build and push by digest
        id: build
        uses: docker/build-push-action@v4
        with:
          context: .
          file: './8.2/Dockerfile'
          platforms: ${{ matrix.platform }}
          labels: ${{ steps.meta.outputs.labels }}
          outputs: type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,name-canonical=true,push=true

      - name: Export digest
        run: |
          mkdir -p /tmp/digests
          digest="${{ steps.build.outputs.digest }}"
          touch "/tmp/digests/${digest#sha256:}"

      - name: Upload digest
        uses: actions/upload-artifact@v3
        with:
          name: digests
          path: /tmp/digests/*
          if-no-files-found: error
          retention-days: 1

  merge:
    name: Merge and Push image manifest
    runs-on: ubuntu-latest
    needs:
      - build
    steps:
      - name: Download digests
        uses: actions/download-artifact@v3
        with:
          name: digests
          path: /tmp/digests

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@v4
        with:
          tags: |
            type=ref,event=branch,prefix=8.2-
            type=sha,prefix=8.2-
            type=raw,value=8.2
          images: ${{ env.REGISTRY_IMAGE }}

      - name: Login to Public ECR
        uses: docker/login-action@v2
        with:
          registry: public.ecr.aws
          username: ${{ secrets.AWS_ACCESS_KEY_ID }}
          password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        env:
          AWS_REGION: 'us-east-1'

      - name: Create manifest list and push
        working-directory: /tmp/digests
        run: |
          docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
            $(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)

      - name: Inspect image
        run: |
          docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}

Basically a copy-paste of what is documented, except I have added tags, and use AWS ECR as a registry.

Run docker/setup-buildx-action@v2
  with:
    driver: docker-container
    buildkitd-flags: --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host
    install: false
    use: true
    cleanup: true
  env:
    REGISTRY_IMAGE: public.ecr.aws/luminsports/php-runtime
    DOCKER_METADATA_OUTPUT_VERSION: 8.2-main
    DOCKER_METADATA_OUTPUT_TAGS: public.ecr.aws/luminsports/php-runtime:8.2-main
  public.ecr.aws/luminsports/php-runtime:8.2
  public.ecr.aws/luminsports/php-runtime:8.2-55ee41e
    DOCKER_METADATA_OUTPUT_LABELS: org.opencontainers.image.created=2023-08-01T07:03:50.110Z
  org.opencontainers.image.description=
  org.opencontainers.image.licenses=
  org.opencontainers.image.revision=55ee41e49966302b5739eb0a054d6d6698d03316
  org.opencontainers.image.source=https://github.com/luminsports/docker-php-runtime
  org.opencontainers.image.title=docker-php-runtime
  org.opencontainers.image.url=https://github.com/luminsports/docker-php-runtime
  org.opencontainers.image.version=8.2-main
    DOCKER_METADATA_OUTPUT_JSON: {"tags":["public.ecr.aws/luminsports/php-runtime:8.2-main","public.ecr.aws/luminsports/php-runtime:8.2","public.ecr.aws/luminsports/php-runtime:8.2-55ee41e"],"labels":{"org.opencontainers.image.created":"2023-08-01T07:03:50.110Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"55ee41e49966302b5739eb0a054d6d6698d03316","org.opencontainers.image.source":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.title":"docker-php-runtime","org.opencontainers.image.url":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.version":"8.2-main"}}
    DOCKER_METADATA_OUTPUT_BAKE_FILE: /tmp/docker-actions-toolkit-KZi5wP/docker-metadata-action-bake.json
Docker info
  /bin/docker version
  Client:
   Version:           20.10.23
   API version:       1.41
   Go version:        go1.18.9
   Git commit:        7155243
   Built:             Tue Apr 11 22:56:36 2023
   OS/Arch:           linux/amd64
   Context:           default
   Experimental:      true
  
  Server:
   Engine:
    Version:          20.10.23
    API version:      1.41 (minimum version 1.12)
    Go version:       go1.18.9
    Git commit:       6051f14
    Built:            Tue Apr 11 22:57:17 2023
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          1.6.19
    GitCommit:        1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
   runc:
    Version:          1.1.7
    GitCommit:        f19387a6bec4944c770f7668ab51c4348d9c2f38
   docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0
  /bin/docker info
  Client:
   Context:    default
   Debug Mode: false
   Plugins:
    buildx: Docker Buildx (Docker Inc., 0.0.0+unknown)
  
  Server:
   Containers: 0
    Running: 0
    Paused: 0
    Stopped: 0
   Images: 2
   Server Version: 20.10.23
   Storage Driver: overlay2
    Backing Filesystem: xfs
    Supports d_type: true
    Native Overlay Diff: true
    userxattr: false
   Logging Driver: json-file
   Cgroup Driver: cgroupfs
   Cgroup Version: 1
   Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
   Swarm: inactive
   Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
   runc version: f19387a6bec4944c770f7668ab51c4348d9c2f38
   init version: de40ad0
   Security Options:
    seccomp
     Profile: default
   Kernel Version: 5.10.184-175.749.amzn2.x86_64
   Operating System: Amazon Linux 2
   OSType: linux
   Architecture: x86_64
   CPUs: 2
   Total Memory: 7.612GiB
   Name: ip-10-10-4-207.us-east-2.compute.internal
   ID: IGJ7:R2LK:GTXA:RLRD:BBEA:7G4G:RJBX:II6O:G2HG:YYRW:23SJ:KAZT
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Registry: https://index.docker.io/v1/
   Labels:
   Experimental: false
   Insecure Registries:
    127.0.0.0/8
   Live Restore Enabled: false
  
Buildx version
  /bin/docker buildx version
  github.com/docker/buildx 0.0.0+unknown 
Creating a new builder instance
  /bin/docker buildx create --name builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31 --driver docker-container --use
  builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31
Booting builder
  /bin/docker buildx inspect --bootstrap
  #1 [internal] booting buildkit
  #1 pulling image moby/buildkit:buildx-stable-1
  #1 pulling image moby/buildkit:buildx-stable-1 0.2s done
  #1 creating container buildx_buildkit_builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310
  #1 creating container buildx_buildkit_builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310 0.5s done
  #1 DONE 0.7s
  Name:          builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31
  Driver:        docker-container
  Last Activity: 2023-08-01 07:03:51 +0000 UTC
  
  Nodes:
  Name:      builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310
  Endpoint:  unix:///var/run/docker.sock
  Status:    running
  Buildkit:  v0.11.6
  Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Inspect builder
  {
    "nodes": [
      {
        "name": "builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310",
        "endpoint": "unix:///var/run/docker.sock",
        "status": "running",
        "buildkit": "v0.11.6",
        "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/amd64/v4,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6"
      }
    ],
    "name": "builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d31",
    "driver": "docker-container",
    "lastActivity": "2023-08-01T07:03:51.000Z"
  }
BuildKit version
  builder-c7c8f7e5-dfdc-47a8-8876-620c8cfc2d310: v0.11.6
  
  
  
.....
  
  
  
#14 exporting to image
#14 exporting layers
#14 exporting layers 14.9s done
#14 exporting manifest sha256:1728bb3d8f7d1b84edb2ec52388e51092ad0d36d12fef52a008b4429f5b22c34 0.0s done
#14 exporting config sha256:c3539ea4a70dde4ea0e01454c88599fc0262b8b6a55685a119f847da6128ffb7 done
#14 exporting attestation manifest sha256:102072b2361b36773c1ca495e7a0c49ad88aecff2d000224ae9378dda901c045
#14 ...

#15 [auth] aws:: luminsports/php-runtime:pull,push token for public.ecr.aws
#15 DONE 0.0s

#14 exporting to image
#14 exporting attestation manifest sha256:102072b2361b36773c1ca495e7a0c49ad88aecff2d000224ae9378dda901c045 0.0s done
#14 exporting manifest list sha256:cfd1d14f1596d8de02256ead61815688d180676940d81d3fa4feb5a5902c0fa2 done
#14 pushing layers
#14 pushing layers 8.2s done
#14 pushing manifest for public.ecr.aws/luminsports/php-runtime
#14 pushing manifest for public.ecr.aws/luminsports/php-runtime 0.6s done
#14 DONE 23.7s



.....



Run mkdir -p /tmp/digests
  mkdir -p /tmp/digests
  digest=""
  touch "/tmp/digests/${digest#sha256:}"
  shell: /bin/bash -e {0}
  env:
    REGISTRY_IMAGE: public.ecr.aws/luminsports/php-runtime
    DOCKER_METADATA_OUTPUT_VERSION: 8.2-main
    DOCKER_METADATA_OUTPUT_TAGS: public.ecr.aws/luminsports/php-runtime:8.2-main
  public.ecr.aws/luminsports/php-runtime:8.2
  public.ecr.aws/luminsports/php-runtime:8.2-55ee41e
    DOCKER_METADATA_OUTPUT_LABELS: org.opencontainers.image.created=2023-08-01T07:03:50.110Z
  org.opencontainers.image.description=
  org.opencontainers.image.licenses=
  org.opencontainers.image.revision=55ee41e49966302b5739eb0a054d6d6698d03316
  org.opencontainers.image.source=https://github.com/luminsports/docker-php-runtime
  org.opencontainers.image.title=docker-php-runtime
  org.opencontainers.image.url=https://github.com/luminsports/docker-php-runtime
  org.opencontainers.image.version=8.2-main
    DOCKER_METADATA_OUTPUT_JSON: {"tags":["public.ecr.aws/luminsports/php-runtime:8.2-main","public.ecr.aws/luminsports/php-runtime:8.2","public.ecr.aws/luminsports/php-runtime:8.2-55ee41e"],"labels":{"org.opencontainers.image.created":"2023-08-01T07:03:50.110Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"55ee41e49966302b5739eb0a054d6d6698d03316","org.opencontainers.image.source":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.title":"docker-php-runtime","org.opencontainers.image.url":"https://github.com/luminsports/docker-php-runtime","org.opencontainers.image.version":"8.2-main"}}
    DOCKER_METADATA_OUTPUT_BAKE_FILE: /tmp/docker-actions-toolkit-KZi5wP/docker-metadata-action-bake.json

Will work on getting this repo public to make it easier to share logs.

[crazy-max]

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Build and push Docker image
        id: build-docker
        uses: docker/build-push-action@v4
        with:
          file: Dockerfile
          tags: "my-service:latest"
          cache-from: type=gha
          cache-to: type=gha,mode=max
      - name: Summarize the Docker digest
        run: |
          echo 'Digest: ${{ steps.build-docker.outputs.digest }}'

Looking again at your workflow this is expected. When using the docker/setup-buildx-action@v2 action without specifying the driver, a container builder will be created. As you don't export the build result (load or push), then no digest is available in result metadata.

Depending on your use case you can either use the docker driver in the setup buildx action or set load: true in the build push action.

I have updated our workflow in #928 to check this behavior. Here is a summary when digest and imageid is available following your setup:

	digest	imageid
docker (no export)	✅	✅
docker-container (no export)	❌	❌
docker (load)	✅	✅
docker-container (load)	⚠️	✅
docker (push)	✅	✅
docker-container (push)	✅	⚠️

⚠️ means the value is set for backward compatibility. imageid should be used if you want to check an image from docker store (docker image ls) and digest for a pushed image. More info: #569

[JensRantil]

Hello! Original author here. 👋 I can confirm that adding load: true made outputs.digest be populated for me. I will leave it up to the maintainers to decide if this issue should be closed now, closed when this behaviour is documented, or when a new ticket for documenting the behaviour has been created.

[JensRantil]

Hello - it's me again! load: true worked without push: true, but am getting

buildx failed with: ERROR: push and load may not be set together at the moment

with push: true. Is there a workaround for this? I saw --output ...,push=true above, so maybe --output=type=docker,push=true?

[crazy-max]

Hello - it's me again! load: true worked without push: true, but am getting

buildx failed with: ERROR: push and load may not be set together at the moment

--load is a shorthand for --output=type=docker like --push is a shorthand for --output=type=registry but multiple exporters are currently not supported: moby/buildkit#1555

with push: true. Is there a workaround for this?

You need to call the action twice as shown in https://docs.docker.com/build/ci/github-actions/test-before-push/ if this is to test your image before pushing it. Otherwise no sure why you would like to load and push.

[ryzr]

In case it helps anyone:

My runners were on Amazon Linux 2 (also tried Amazon Linux 2023), which are stuck on Docker Server 20.x.

I updated my runners to Ubuntu 22.04, which comes with Docker Server 24.x. The digest is returned correctly now.

[AndreasBergmeier6176]

Be aware, that docker.io in version 24.x, which comes with Ubuntu 22.04.1+ is actually missing buildx up until at least 2023-09-07.

[crazy-max]

Hello! Original author here. 👋 I can confirm that adding load: true made outputs.digest be populated for me. I will leave it up to the maintainers to decide if this issue should be closed now, closed when this behaviour is documented, or when a new ticket for documenting the behaviour has been created.

Will look to document this case

--load is a shorthand for --output=type=docker like --push is a shorthand for --output=type=registry but multiple exporters are currently not supported: moby/buildkit#1555

Since Buildx 0.13 multi exporters can be used but we need to handle that correctly in the exporter response. Atm only the first response is returned, see docker/buildx#2181

[dorschmeister]

Posting here in case someone else has this edge case....

I ran into this problem as well and traced the problem to the Amazon Linux docker package.

When running docker info the following output is returned

Client:
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.0.0+unknown
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
...

Buildx is definitely installed with this package and I believe it is 0.13.x as the provenance param works with the cli on the server. This is a current open issue on the Amazon Linux 2023 repo.

When using this action on an Amazon Linux VM hosted runner this invalid version prevents the provenance flag from being passed to the cli and therefore no digest is generated. Manually setting provenance:true does not fix it and generates a warning in attestations stating that provenance is not allow for versions under 0.10.x

The current options are to not use this action and write the CLI command or switch to Ubuntu. I ended up terminating all our runners and recreating them on Ubuntu.

This is sonewhat edge case however it will affect attempting to use Amazon Linux as a runner when using the multi runner script referenced in the docker docs. it would be nice to have an option in the action to manually append cli arguments or bypass version checks to force through cli args as workarounds for this or future similar cases.