Cannot use the `buildx` with the docker compose v2 (cache_to feature not supported) #9832

kaznovac · 2022-09-12T11:28:35Z

Description

I'm trying to use <service>.build.cache_to property in docker-compose.yml file to cache/restore the buildx image cache layers in subsequent GHA job runs (the same behavior is observable in local docker for mac distribution).
<service>.build.cache_from property seems to work correctly.

Steps to reproduce the issue:

create docker-compose.yml with cache_to instruction (e.g. cache_to: [ type=local,dest=/tmp/.buildx-cache-new,mode=max ])
try to use docker compose build

Describe the results you received:
the following error is displayed

cache export feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")

I've tried using the docker buildx create --use --bootstrap, and still cannot instruct docker compose to use the buildx. docker buildx bake works as expected on the same docker-compose.yml

Describe the results you expected:
I expect docker compose to work with default build driver, or to have an argument to select the desired build driver.

Additional information you deem important (e.g. issue happens only occasionally):
reproducible always

Output of docker compose version:

Docker Compose version v2.10.2

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.10.2)
  extension: Manages Docker extensions (Docker Inc., v0.2.9)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.19.0)

Server:
 Containers: 20
  Running: 4
  Paused: 0
  Stopped: 16
 Images: 45
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: 89srq4nbppz7i7dc8emycrcod
  Is Manager: true
  ClusterID: psekb5sp5curg03awy23ql26i
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8  
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.65.3
  Manager Addresses:
   192.168.65.3:2377
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.124-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 2.428GiB
 Name: docker-desktop
 ID: 6Q54:J23F:JM2Q:ROXM:CD2C:NUMJ:6XY3:FEFW:O37S:KH7Y:6P6F:VKIG
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

The text was updated successfully, but these errors were encountered:

laurazard · 2022-09-15T19:57:28Z

Hi @kaznovac, thanks for the report.

I think this might be addressed by the changes in #9729. Could you see if this is still an issue with a more recent version?

dee-kryvenko · 2022-09-16T00:43:09Z

Had the same problem. Using Docker 20.10.18 (which includes Compose 2.11.0 (which includes #9729)) fixed it for me. Released 2 days ago. I can't describe how happy I am right now. I was two days away from getting my week ruined. Thanks for the fix!

laurazard · 2022-09-16T14:05:08Z

@dee-kryvenko Awesome! Glad to hear that. I'm closing this issue @kaznovac, feel free to comment/reopen if it's still an issue.

witsch · 2022-10-04T14:58:01Z

Hi, we're basically trying to tackle the same thing:

steps:
  - uses: actions/checkout@v3
  - name: Build images
    uses: docker/bake-action@master
    with:
      load: true
      targets: backend
      set: |
        backend.cache-from=type=gha
        backend.cache-to=type=gha
  - name: Run tests
    run: docker compose run backend

gives us ERROR: cache export feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use"), but we should supposedly have a recent enough version, i.e. including #9729:

[command]/usr/bin/docker version
Client:
Version:           20.10.18+azure-1
API version:       1.41
Go version:        go1.18.6
Git commit:        b40c2f6b5deeb11ac6c485c940865ee40664f0f0
Built:             Thu Sep  8 08:19:02 UTC 2022
OS/Arch:           linux/amd64
Context:           default
Experimental:      true

Server:
Engine:
Version:          20.10.18+azure-1
API version:      1.41 (minimum version 1.12)
Go version:       go1.18.6
Git commit:       e42327a6d3c55ceda3bd5475be7aae6036d02db3
Built:            Thu Sep  8 22:50:10 2022
OS/Arch:          linux/amd64
Experimental:     false
containerd:
Version:          1.5.13+azure-1
GitCommit:        a17ec496a95e55601607ca50828147e8ccaeebf1
runc:
Version:          1.1.4
GitCommit:        5fd4c4d144137e991c4acebb2146ab1483a97925
docker-init:
Version:          0.19.0
GitCommit:        

[command]/usr/bin/docker info
Client:
Context:    default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., 0.9.1+azure-1)
compose: Docker Compose (Docker Inc., 2.11.2+azure-1)

Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 20
Server Version: 20.10.18+azure-1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: a17ec496a95e55601607ca50828147e8ccaeebf1
runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925
init version: 
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.15.0-1020-azure
Operating System: Ubuntu 20.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 6.781GiB
Name: fv-az186-20
ID: COJK:DGIK:OKWD:REMT:QGYK:H4LG:3S2R:VCG3:VWYD:4DOU:YCMW:U3BY
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: githubactions
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

Are we missing anything? 😊

PS: the error is also the same using

backend.cache-to=type=local,dest=/tmp/.buildx-cache-new,mode=max

glours · 2022-10-04T15:15:56Z

@witsch Take a look to the Compose specification, you should be able to do it by defining cache_from and cache_to in the build section of your service

norman-zon · 2022-10-04T15:25:43Z

@glours defining the caching in the service like this

 backend:
    build:
      target: backend-testing
      cache_from:
      - type=gha
      cache_to:
      - type=gha

yields the same error: buildx bake failed with: ERROR: cache export feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")

glours · 2022-10-04T15:28:31Z

Ho, ok, sorry, you should setup buildx action before using bake to be able to use a driver which support this option

-
  name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v2

norman-zon · 2022-10-04T15:31:59Z

Buildx is setup. It just complains that the cache-export feature is not supportet. This is the relevant snippet of the workflow:

 steps:
      - uses: actions/checkout@v3
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          driver: docker

      - name: Build images
        uses: docker/bake-action@master
        with:
          load: true
          files: testing.yml
          targets: backend

Had the same problem. Using Docker 20.10.18 (which includes Compose 2.11.0 (which includes #9729)) fixed it for me. Released 2 days ago. I can't describe how happy I am right now. I was two days away from getting my week ruined. Thanks for the fix!

We are running docker 20.10.18 and compose 2.11.2, so I expected caching to work now with the builx docker driver.

glours · 2022-10-04T15:36:16Z

right it should, anyway regarding the error message, the best place to ask directly in the action repository

witsch · 2022-10-04T15:48:51Z

right it should, anyway regarding the error message, the best place to ask directly in the action repository

Ok, we'll try there… Thanks for your help! 😊

dee-kryvenko · 2022-10-04T17:47:45Z

The error message makes it clear - you are using docker driver and it does not support cache. You should use create command to set up docker-container driver. Before 20.10.18 it was broken with docker-container driver too, now it is fixed.

norman-zon · 2022-10-05T07:25:11Z

Thank you for the clarification @dee-kryvenko. I misinterpreted the discussion in this issue, thinking it would work now with the docker driver.
docker-container driver works for me too, confirmed!

laurazard self-assigned this Sep 12, 2022

laurazard closed this as completed Sep 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot use the `buildx` with the docker compose v2 (cache_to feature not supported) #9832

Cannot use the `buildx` with the docker compose v2 (cache_to feature not supported) #9832

kaznovac commented Sep 12, 2022

laurazard commented Sep 15, 2022

dee-kryvenko commented Sep 16, 2022

laurazard commented Sep 16, 2022

witsch commented Oct 4, 2022

glours commented Oct 4, 2022

norman-zon commented Oct 4, 2022 •

edited

glours commented Oct 4, 2022 •

edited

norman-zon commented Oct 4, 2022

glours commented Oct 4, 2022

witsch commented Oct 4, 2022

dee-kryvenko commented Oct 4, 2022

norman-zon commented Oct 5, 2022

Cannot use the buildx with the docker compose v2 (cache_to feature not supported) #9832

Cannot use the buildx with the docker compose v2 (cache_to feature not supported) #9832

Comments

kaznovac commented Sep 12, 2022

laurazard commented Sep 15, 2022

dee-kryvenko commented Sep 16, 2022

laurazard commented Sep 16, 2022

witsch commented Oct 4, 2022

glours commented Oct 4, 2022

norman-zon commented Oct 4, 2022 • edited

glours commented Oct 4, 2022 • edited

norman-zon commented Oct 4, 2022

glours commented Oct 4, 2022

witsch commented Oct 4, 2022

dee-kryvenko commented Oct 4, 2022

norman-zon commented Oct 5, 2022

Cannot use the `buildx` with the docker compose v2 (cache_to feature not supported) #9832

Cannot use the `buildx` with the docker compose v2 (cache_to feature not supported) #9832

norman-zon commented Oct 4, 2022 •

edited

glours commented Oct 4, 2022 •

edited