Publish arm64 Docker images #1307
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1307 +/- ##
========================================
+ Coverage 62.2% 63.1% +0.9%
========================================
Files 164 160 -4
Lines 12191 11914 -277
========================================
- Hits 7583 7520 -63
+ Misses 4608 4394 -214 ☔ View full report in Codecov by Sentry. |
|
@krevsbech any thoughts on these Docker build changes? |
|
This only seems to work with docker, podman raises an error when trying to use the build script: ./dockerfiles/build.sh --platform linux/amd64,linux/arm64
Error: unable to parse value provided `[]` to --cache-from: repository must contain neither a tag nor digest: docker.io/labgrid/client:latest
Error: unable to parse value provided `[]` to --cache-from: repository must contain neither a tag nor digest: docker.io/labgrid/exporter:latest
Error: unable to parse value provided `[]` to --cache-from: repository must contain neither a tag nor digest: docker.io/labgrid/coordinator:latest |
Emantor
added
the
needs author info
To allow customizing the arguments passed to docker/podman build(x) beyond just the platform, modify build.sh to pass all its arguments through to the build command. For the platform case, this means it is now necessary to pass "--platform linux/arm64" instead of just "linux/arm64" so modify README.md to reflect this. This change also has the effect of automatically using buildx if supported, even if no arguments are passed, so move the information about buildx further up in the text. The link to podman's buildx is dropped as (unlike Docker) "podman buildx" is just an alias for "podman build". Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
To simplify usage of build.sh on CI, use the same image names for local development as on the Docker hub (i.e. labgrid/client instead of labgrid-client). Allow both the prefix and the tag to be overridden using environment variables. Use the same prefix in docker-compose.yml, customizable using the same environment variable. Adjust the documentation to reflect the new prefix. It is no longer necessary to build the images locally before running docker-compose so remove this part of the staging instructions. Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
docker buildx is able to create multi-platform docker images in a single invocation (using emulation where required), but multi-platform images cannot be stored locally, only pushed directly to a regsitry. So refacor the workflow a little. Set common configuration as environment variables. Add actions to initialize QEMU and buildx. The remaining steps are modified to: * First build just the amd64 image and use this for validation using docker-compose (--load only supports single-platform images) * Build, tag and push the "latest" images for all platforms in a single invocation * For tags, override IMAGE_TAG and run the same command again. Built layers are cached so no redundant work is performed, despite running the same command multiple times. Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
It is observed that there are no images on Docker hub for Labgrid release tags, despite listening to tag events. This may be because this event is not triggered for tags created via the Github release UI, so add this event too. Additionally allow the workflow to be triggered manually for testing purposes. Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
|
It seems caching doesn't work consistently between Podman and Docker. If it's OK, I've dropped the commit that adds the caching arguments. I tried to enable caching just in the CI workflow but
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Address #917 by adding support for building and publishing arm64 Docker images on CI.
The most direct way to do this seems to be docker buildx, using Github actions provider by Docker, but buildx isn't able to push multi-platform images to a local image store so some supporting changes are required to build.sh:
Note that 1. changes usage for build.sh when building for a specific platform. This is addressed in README.md and hopefully addresses more use-cases, but not sure about Labgrid policy about breaking changes?
Building non-x86 platforms with Github Actions uses emulation, so enable layer caching and tweak the Dockerfile to make better use of caches.
There is an additional patch to enable image builds on release events and manual triggers.
I have run a action on top of these changes (using the manual trigger) over at my fork: https://github.com/hoyes/labgrid/actions/runs/7209149336, resulting in https://hub.docker.com/u/hoyes
Checklist