Support for docker build --secret for build-time docker secrets.

[mikhail-khodorovskiy]

Is your feature request related to a problem? Please describe.
https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066 described how to provide secret mounts available at build times.

docker build --secret id=mysite.key,src=path/to/mysite.key .

Can be referenced during docker build:

RUN --mount=type=secret,id=mysite.key,required

docker compose build command: https://docs.docker.com/compose/compose-file/#build currently has no support for the secrets section

Describe the solution you'd like
build command should allow referencing the secrets defined in the same docker-compose file

Describe alternatives you've considered
The alternatives is to use secret-containing file contents as build arguments. The solution is cumbersome since the secret has to be splatted so it can become available for Dockerfile commands which may cause it to become part of a docker layer, i.e it may leak the secret value.

Additional context
Add any other context or screenshots about the feature request here.

[ciaranmcnulty]

This would be really useful - right now we're doing a docker build using the new option and matching the image name to the one that compose will generate, as part of a shell wrapper.

Native support would be great.

[ciaranmcnulty]

As a future-proofing measure it might be interesting to have a key that allows arbitrary CLI flags to be passed to build?

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile
      target: secure
      flags: '--secret id=mysite.key,src=path/to/mysite.key'

[yinzara]

Agree! SUPER useful especially in a setting where you're building artifacts that have to access private repositories of some sort. Nearly impossible to do without this.

While I would support the idea of a 'flags' attribute, I think because of the specifics of this feature, a real language element would be better. Maybe make a separate feature request specifically for this.

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile
      target: secure
      secrets:
        -  id: mysite.key
           src: path/to/mysite.key

or possibly assuming there will never be other attributes to the command:

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile
      target: secure
      secrets:
        'mysite.key': path/to/mysite.key

[ciaranmcnulty]

@yinzara flags attribute was to address next time this happens - a specific syntax for secrets would be best

We should also support:

services:
  foo:
    build:
        # ...
        ssh: default

[Legogris]

Just supporting --ssh for docker-compose run would be great.

[chris-crone]

Thanks for the suggestion! This requires that we support BuildKit in the Python SDK, see the issue here.

[mattsrobot]

Has this ticket moved on, is there a way to use ssh secrets in docker compose?

[chris-crone]

@mattsrobot There is now a PR to add BuildKit support to Docker Compose (see: #6865).

The approach of the PR is to exec out to the Docker CLI for build so we could pass flags there.

[janbuchar]

@chris-crone I'm trying to do that, but I don't see how I could pass flags to the CLI from docker-compose.yml. Am I missing something?

[antiBaconMachine]

👋 I've taken a stab at this. I actually didn't find this issue until it came time to PR so I'm only reading suggestions now. Happily, I think what I've done is broadly in line with this discussion. I actually really like @ciaranmcnulty idea for a more general purpose flags array, particularly as this feature is still marked experimental. If people prefer that can implement. Anyway let me know what you all think. #7046

[itamarst]

Until the relevant PR is merged and released, it's possible as a workaround to have a Dockerfile which uses build args to get secrets during Compose-based local development, and BuildKit secrets to build the production image so you don't leak those secrets. Basically you expect the secret to be in an env variable, and in Compose you set it with a build arg, and for production builds you have a script that sets the env variable from the BuildKit-mounted secrets file.

I documented all this here: https://pythonspeed.com/articles/build-secrets-docker-compose/

[Maks3w]

Until the relevant PR is merged and released, it's possible as a workaround to have a Dockerfile which uses build args to get secrets during Compose-based local development, and BuildKit secrets to build the production image so you don't leak those secrets. Basically you expect the secret to be in an env variable, and in Compose you set it with a build arg, and for production builds you have a script that sets the env variable from the BuildKit-mounted secrets file.

I documented all this here: https://pythonspeed.com/articles/build-secrets-docker-compose/

Sorry but build time secrets are working with current version.

services:
  my-application:
    secrets:
      - my-secret

secrets:
  my-secret:
    file: <path to secret file inside of docker context: ./secrets/secret-filename>

DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker-compose build

[itamarst]

That's a runtime secret.

[greg-caffeine]

@Maks3w, thanks, but I am not able to get docker-compose build to be satisfied with the above example configuration. The above would give me "secret my-secret not found" if my Dockerfile contains a "RUN --mount=type=secret,required,id=my-secret,dst=secret.key do-something". From the docker build flag necessary to pass in the secret and make BuildKit happy ('--secret id=my-secret,src=path/to/my-secret') it seems the "id=" part isn't being satisfied by docker-compose.

[greg-caffeine]

FWIW, work on a related workaround is happening on this in #7296.

[tommie]

It was suggested in #7296 (comment) that this needs a compose-spec update. I filed compose-spec/compose-spec#81 to continue the compose-spec discussion there.

[agates4]

I'm not sure how this is overlooked - it is a major functionality piece missing from docker-compose.

Paths forward:

• A) Global env option to pass arbitrary flags through docker-compose
• B) Hardcoded option to pass explicit flags through docker-compose

These are our two options. It is clear to me that option A) is not the preferred option (of the docker-compose devs).

So, can we see a PR to introduce the hardcoded flags? Rather than stagnating in the discussion of how many more times in the future we'll need to hardcode flags, why not just do it? Let's go!

Here is the solution:
#7046

Here the discussion is (required??) staled:
compose-spec/compose-spec#81

...

So, will we wait one more year to hear news that we need to wait one more year?

[agates4]

@antiBaconMachine

[ceresdarkmatter]

I cannot believe that such a major functionality gaping hole is still present 3.5 years after this issue has been raised.
Can anyone from the Docker team give us any update on this specific issue?

What are the blockers? Are you understaffed? Underfunded? Consider this issue a "nice to have, maybe someday when the Sun is shining"?

I am at a loss for words... can someone please have the courtesy of giving us a straight no-bullfeces answer?

[ndeloof]

@ceresdarkmatter tl;dr: compose has been missing a clear product governance. This is why we created compose v2 as a docker CLI plugin, so it can evolve at same speed.
We just introduced support for ssh, secrets is on his way (compose-spec/compose-spec#238)