Enable ssh option as build parameter for buildkit

[bert2002]

Is your feature request related to a problem? Please describe.
When building a container we need access to other Resources like ssh and dont want to expose a private key in the container.

Describe the solution you'd like
Support of --ssh in the build process that already works in buildkit.

Additional context
I tried 1.25.0-rc4, but it does not support --ssh option. Tried the same as in bug 6440:

export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
docker-compose build --ssh default my_image

This would be a major security improvement and save us so much pain.

Thanks,
bert

[ndeloof]

tracked internally as https://docker.atlassian.net/browse/COMPOSE-107

[sajusat]

Any timeline on this ?

[jufemaiz]

🤔 any news team?

[bert2002]

I saw that buildkit made its way into docker-compose, but ssh is not there yet. This feature would really help to make docker-compose an ideal team for large scale deployments and local developments in one tool.

[sajusat]

I adopted this feature early and now I am having a maintenance nightmare.

[jungwookim]

@sajusat How did you adopt this feature? How's it going?

[sajusat]

@jungwookim I kept the experimental meta tag in all the Dockerfiles and used --ssh option to do the builds. When I use these images I cannot used the --build option when there is a change in the dockerfile.

[alechirsch]

Any update on this? We resorted to using shell scripts for building, and then using docker compose to start everything.

[AdemUstaReminiz]

Any update on this guys ?

[sajusat]

It's been like two years since I am following this option. I have seen talks about should you guys implement it or not... Someone please say something concrete.

[ChosunOne]

Any update on this?

[dominik-stronk]

I know it may not be the best place to ask nooby questions, but it kind of relates to the topic:
What is a difference between 'docker compose' and 'docker-compose' ?
Obviously 'compose' command is not documented anywhere. I ask, because for some reason I am able to build image which relies on ssh connection with 'docker compose', tho it expectedly fails with 'docker-compose'.

My use case is only for dev purposes and I am using Docker Desktop for Windows with WSL 2.

Edit: Ok, all it took is to read first outputline of the command:
The new 'docker compose' command is currently experimental. To provide feedback or request new features please open issues at https://github.com/docker/compose-cli

[zoonage]

Forgot to update with #7997 which I'm working on to support arbitrary arguments to help with this

[jaanus]

+1. I am a regular Docker user working on an app with Docker and compose. I was glad to find out about the easy and secure way of using SSH in docker build --ssh default to give the build access to my SSH key to fetch private packages. “Cool”, I thought, “since it is so easy with docker build, I can do the same in my docker compose process too.” Which I apparently can’t yet due to this bug. I hope this will be available to compose.

[davide-c]

+1

[ekampp]

I would also love this to be a thing, or for the compose to be generally more transparent when sending information from or to the host.

[john012343210]

+1

[AnticliMaxtic]

I would also greatly appreciate this feature!

[GabrielPsantos]

Has anyone got some kind of workaround ? 🥺

[jaanus]

Has anyone got some kind of workaround ? 🥺

Workaround is that you build the individual images yourself with docker build using the SSH options, and then after that do the compose.

[kshcherban]

Has anyone got some kind of workaround ? pleading_face

We did it like described here, wrote a shell wrapper around docker-compose.
An alternative solution would be to fork docker-compose and add needed features there. Imho docker is dying and they don't care much about docker buildkit and compose compatibility anymore.

[squalvj]

+1

[nikolalukovic]

Is there any update about this?

[webmailcontatos]

Is there any update about this?

[tsturzl]

2 years later, what's blocking here? Seems someone opened a PR, that PR depended on changes from another repo, that PR is now dead in the water. Lots of people need to get SSH working inside containers to get private dependencies built. I'm more than happy to open a PR with a little guidance on the issue, and some assurance that someone will actually help push this feature through. Seems like the last attempt died in the water and this issue is just forgotten about now. I'd love to contribute here, but I'd need to know it won't become an abandoned effort.

[glours]

Hello @tsturzl

We're currently writing a proposal to extend the build section of the Compose specification which compile all the proposals already open on the subject. We should propose it soon and start working on an implementation right after the approval.
Just to let you know that we're on it actively ATM

[erm156]

Workaround is that you build the individual images yourself with docker build using the SSH options, and then after that do the compose.

I'm assuming the images that are built separately using the --ssh flag are included as services in the compose YAML and that I only need to run docker-compose up after the separate builds (the services in my YAML are either pulled from dockerhub or built separately with --ssh flag). Is this accurate?

[tsturzl]

@glours Not to be mean here, and certainly you're not to blame for this. But the entire idea of having a Compose Specification just so you can have multiple implementations seems like a massive drawback if it takes 2 years to push an idea through. Or maybe this is a problem with how that specification approval process works? Simply supporting new docker features seems like a high priority for docker tools, at least high enough priority to not be a multi-year delivery. I mean this issue is over 2 years old, the compose spec ticket was opened a few months later. It's taken 2 years to get this added to the spec, and it's not even done yet. It seems like the conclusion was already to just do what the ticket suggested. I just don't get why changing the spec was this time consuming, meanwhile I believe there has been at least one PR to add this that got rejected because the spec needed to change.

I really don't want to be harsh here, but it baffles me that the blocker here was adding some words to 2500 line document. I get OSS contributors are most often spending their own time on these things, but the process just seems nauseatingly overburdened for maintainers who may not have the time to keep the tool relevant. I'd also assume that docker, as a commercial entity could be pushing along supporting their own features in their own tools in a reasonable time frame. This just seems like an unacceptable long time to deliver given that these projects are part of a commercially supported tool. 2 years for a spec change is what I'd expect of the IEEE to make changes to the HTTP spec, not for a spec that less than a dozen pieces of software rely on.

[tsturzl]

My workaround for this is just to use Tilt since Tilt has supported buildkit for a while. I let Tilt build my images and then I just use compose to run those images, which is also managed through Tilt. In addition I also get automatic rebuild and relaunch based on file watchers. That's seemed like the least hackish workaround for this problem so far.

[ndeloof]

@tsturzl lack of progress on this topic within the compose specification is mostly caused by low feedback on the initial proposal.
In the meantime the buildx team proposed some x- extensions (which has been designed by the compose specification to offer a way to experiment and come with a concrete solution, vs a theoretical proposal) and we now have concrete bits we can debate on and move forward.

[bpogodzinski]

@ndeloof could you please tell me how much time, more or less, we will have to wait for this?

[sajusat]

@bpogodzinski Lol!!! I have been waiting for this feature for the past 2 and a half years and counting.....sort of gave up actually. Don't keep your hopes too high!

[ndeloof]

@sajusat @bpogodzinski things changed with the launch of Compose v2, sort of re-birth for the compose project, with an open-process to evolve the file format.
There are way too much issues which have been never answered, or even rejected just by the opinion of a single maintainer :'( Hopefully we are now out of this dark ages, and can move forward

[ndeloof]

@bpogodzinski next steps:

1. ssh support in compose yaml file format is being discussed here : Add ssh to build compose-spec/compose-spec#234. Feel free to comment it there's anything unclear in this proposal, or you're missing some important use-case.
2. Once adopted we can implement support in compose-go (loading yaml, not a big deal) and docker/compose, which should be easy as feature has already been implemented by bake. As both projects are implemented in Go this is just about importing the right func (or, worst case, copy/paste some code)

This is the major benefits for Compose v2 redesign: docker tools are now technically aligned, and this makes it way easier to import features from one into another, while Compose v1 in python required a significant effort just to backport API changes.

[mdiasribeiro]

The spec has been merged, so this should no longer be blocked?

[synek317]

Is there any workaround? I hit the wall at the very beginning of my journey with docker. I struggle with how are people even able to use docker-compose without this feature...

[thomsentner]

Is there any workaround? I hit the wall at the very beginning of my journey with docker. I struggle with how are people even able to use docker-compose without this feature...

Honestly, for me the only option was to write a bash script around a docker run command.

[caseysparkz]

You can also pass your keys into an intermediate container via an environment variable.

On Sat, Mar 26, 2022 at 06:28 Thom Sentner ***@***.***> wrote: Is there any workaround? I hit the wall at the very beginning of my journey with docker. I struggle with how are people even able to use docker-compose without this feature... Honestly, for me the only option was to write a bash script around a docker run command. — Reply to this email directly, view it on GitHub <#7025 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK2N5PJUE2WVYOWQKNFDGRLVB4GHRANCNFSM4JOOX2AA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- With regards, PGP: 0x3260255327CBCDA1

[glours]

A PR is ready to review, you can test it and give us your feedback