Feature Request: Github Action for Copy between registries

[akhilerm]

Currently the copy between registries uses buildx imagetools to copy image from one registry to another. Ref.

It would be better to have a github action that wraps around this functionality. This also helps in reusing the output of metadata-action to create multiple tags, without going through the hassle of using bash to do it in github actions.

[crazy-max]

We discussed about this with @jedevc. I think we could wrap the imagetools command in an action (or subaction) but we also need to think about providing additional features with Dockerfiles, images and registries in the same manner? (analysis, summary, promoting, lint, autoupdate, etc..).

[akhilerm]

but we also need to think about providing additional features with Dockerfiles, images and registries in the same manner? (analysis, summary, promoting, lint, autoupdate, etc..).

@crazy-max Isnt keeping the additional functionalities like analysis better in a separate action rather than clubbing everything to one single unit.

I assume this combination (build-push, login, metadata, copying) of actions are mostly used in CI systems (both private as well as public) and many of them have their own image analysis tools custom to their need. So I am not sure how we can have a pluggable kind of support in the new proposed action. IMHO, I prefer to keep it simple and just does copying images.

[jedevc]

We definitely should have something in this space, especially now that imagetools supports copying images: docker/buildx#1137 🎉 🎉

I think I agree with @akhilerm - splitting up functionality here seems like a good idea. The GitHub action ecosystem is already quite well built for small individual components, and there's already quite a few actions under the docker/ namespace, so I don't think adding too many more would be too tricky.

I think functionality like analysis, linting, promotion, etc is distinct enough to warrant different actions for this kind of functionality - also, by keeping the scope small, I think we could potentially build an action to do copying quickly, without needing to design a larger architecture that can handle all of the other different pieces.

If we're adding more actions, maybe something to be aware of - are we worried about growing the maintenance of components across repositories? Do we need to have a common share library that all of them can pull from, for example, so that we don't need to maintain CI and dev tooling, helper functions, dependabot alerts etc, across multiple repos? @crazy-max we've also discussed this in the context of #746, not sure if you have any more thoughts on this 👀

[polarathene]

TL;DR:

• I think the referenced Docker docs example is a bit misleading with the intent / benefit discussed here.
• It seems the docker buildx imagetools create feature mentioned is better served for similar scenarios that don't depend upon docker/build-push-action as noted below. The above suggestion for the feature to be implemented via separate action is perhaps justified those alternative scenarios.
• This feature request should adjust the title to clearly distinguish it's not about a limitation to push the same image build to multiple registries, as that is clearly supported via the tags input of docker/build-push-action.
  • The issue being resolved is more to do with pushing to a registry when the repository of the image name differs?
  • Or taking an image input without building it again, and pushing it with a modified image name (eg: <registry>/<repo>:<tag>)?

---

Currently the copy between registries uses buildx imagetools to copy image from one registry to another. Ref.

New link location: https://docs.docker.com/build/ci/github-actions/copy-image-registries/

Reference snippet of relevant content presently rendered at that resource:

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: |
            user/app:latest
            user/app:1.0.0
      - name: Push image to GHCR
        run: |
          docker buildx imagetools create \
            --tag ghcr.io/user/app:latest \
            --tag ghcr.io/user/app:1.0.0 \
            user/app:latest

---

We definitely should have something in this space, especially now that imagetools supports copying images: docker/buildx#1137

@jedevc

I have a question as your referenced PR discusses repositories rather than registries?:

This PR introduces support for multiple repositories: this allows creating manifest lists with manifests pulled from multiple other locations.

Whereas this feature request (with referenced documented action) is not clear to me how it differs from the below approach using docker/metadata-action? (full action reference):

      - name: 'Prepare tags'
        id: prep
        uses: docker/metadata-action@v5.0.0
        with:
          images: |
            ${{ secrets.DOCKER_REPOSITORY }}
            ${{ secrets.GHCR_REPOSITORY }}
          tags: |
            type=edge,branch=master
            type=semver,pattern={{major}}
            type=semver,pattern={{major}}.{{minor}}
            type=semver,pattern={{major}}.{{minor}}.{{patch}}

      - name: 'Build and publish images'
        uses: docker/build-push-action@v5.0.0
        with:
          push: true
          platforms: linux/amd64,linux/arm64
          tags: ${{ steps.prep.outputs.tags }}

The images input is paired with the tags input when creating the tags output. Seems to be a working approach to provide the tags and different registries to push to.

@jedevc @crazy-max

Is there a benefit / difference compared to docker buildx imagetools create? At least it's not clear if there would be any when only the registry is being changed?

EDIT: I seem to have identified this in below collapsed sections, and summarized in the above TL;DR.

Additional references

docker/buildx#1137 does reference fixing docker/buildx#486 that notes in Dec 2020 docker buildx imagetools create failed with:

multiple repositories currently not supported, found map[docker.io/crazymax/diun:{} localhost:5000/name/app:{}]

That differs from the Docker docs example where the image name and tags were consistent, with only the registry changing. I assume that's the differentiator? While docker/metadata-action could describe the different repository (image name?) in the images input like with changing the registry, I haven't verified if that would work. Presumably it'd fail with the quoted error.

Thus perhaps the issue title here should be adjusted to reflect that better? Changing the registry shouldn't be an issue if this is the only difference between the two approaches.

moby/buildkit#3647 further highlights the intention of docker/buildx#1137 to serve other scenarios where docker/metadata-action + docker/build-push-action is not used, but adjusting the image registry / repo / tag to publish/push with is required. A related example is this Github Action for DockerHub mirroring which intends to use the same docker buildx imagetools create feature.

Additional insights

Seems to be confirmed via this maintainer comment (June 2022):

Registry scopes blobs by repo. They need to be uploaded again if the repo changes. If you use same repo and different tags then everything should work today.

While a PR referencing docker/buildx#1137 uses docker buildx imagetools create approach to push a multi-arch image from a local registry to DockerHub as they could not verify a multi-arch built image locally without pushing to a local registry.

I am familiar with that multi-arch issue, but for docker-mailserver image tests we just split out the build and publish to separate reusable workflows, where we can build the image and export/upload cache, then run the docker/build-push-action again just to import the cache and build + load the linux/amd64 platform single-arch that we wanted to test.

While that may not be viable for some CI workflows if they need to leverage the multi-arch image build / manifest in some manner, I think in 2023 that's also being addressed (might have been with the containerd image store backend with Docker) - but as I haven't looked into that, it may not be easy to adopt in some environments / CI? 🤷‍♂️

---

It would be better to have a github action that wraps around this functionality.
This also helps in reusing the output of metadata-action to create multiple tags, without going through the hassle of using bash to do it in github actions.

@akhilerm

As shown above, I'm not sure if bash is necessary? (the images + tags input feature has been available prior to this issue being opened, so it's nothing new)

Is this just an issue regarding documentation for awareness? Or have I missed something important where that functionality is not served by docker/metadata-action?

EDIT: If I understand correctly, the concern is not copying between registries with docker/metadata-action tags output, but when the repository in the tags input for docker/build-push-action differs?

---

Do we need to have a common share library that all of them can pull from, for example, so that we don't need to maintain CI and dev tooling, helper functions, dependabot alerts etc, across multiple repos?

For reference, to anyone unaware since this was brought up here, later in Jan 2023 docker/actions-toolkit was created and docker/build-push-action adopted that in Feb 2023. That seems to be addressing the quoted concern.

[akhilerm]

The issue being resolved is more to do with pushing to a registry when the repository of the image name differs?

The issue is to push one image to multiple registries at the same time. Or to create images with different tags. Suppose an image is available in docker.io, I want to push it to multiple registries like gcr.io or ghcr.io . You can basically say its to copy images that are already available from one registry to another.

EDIT: If I understand correctly, the concern is not copying between registries with docker/metadata-action tags output, but when the repository in the tags input for docker/build-push-action differs?

Currently build-push-action supports pushing to multiple registries immediately after doing the build. i.e in the same step itself. But suppose a scenario like this happens: We have an image that is built and stored in local registry. After some testing, we need to push it to multiple other registries from which the image is published.

As shown above, I'm not sure if bash is necessary? (the images + tags input feature has been available prior to this issue being opened, so it's nothing new)

This feature request is not for changes to build-push-action, but create a new action for the functionality mentioned here. Currently its achieved using bash. If we can create a new action for the same, it will help to integrate with other tooling also.

[crazy-max]

Let's continue the discussion in docker/actions-toolkit#239. After that being implemented, we could consider a new action to handle imagetools command.