Generate and verify labels of Docker image during release #10013
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Blocked by #10007. I still need to manually test the release process as well. |
npepinpe
force-pushed
the
9940-oci-labels
branch
from
c8168fb
to
c773725
Compare
npepinpe
force-pushed
the
9940-oci-labels-release
branch
from
5ace47d
to
7c7b233
Compare
10 tasks
npepinpe
changed the title
npepinpe
force-pushed
the
9940-oci-labels-release
branch
from
7c7b233
to
bd01a5a
Compare
npepinpe
force-pushed
the
9940-oci-labels
branch
from
f6af7bc
to
ccb8527
Compare
npepinpe
force-pushed
the
9940-oci-labels-release
branch
from
bd01a5a
to
062a15f
Compare
npepinpe
force-pushed
the
9940-oci-labels
branch
7 times, most recently
from
166e0e2
to
b447062
Compare
npepinpe
force-pushed
the
9940-oci-labels-release
branch
from
062a15f
to
e76677a
Compare
zeebe-bors-camunda bot
added a commit
that referenced
this pull request
Aug 5, 2022
10007: Add OCI & OpenShift labels to Docker image r=npepinpe a=npepinpe ## Description This PR adds labels to the Docker image following the OCI and OpenShift specs. This includes modifications to the build process to inject the few values which are dynamic, namely: - the created at ISO 8601 timestamp - the commit SHA (or revision) of the artifact - the semantic version of the artifact You can find the specs here: - https://github.com/opencontainers/image-spec/blob/main/annotations.md - https://docs.openshift.com/container-platform/4.10/openshift_images/create-images.html#images-create-metadata_create-images On top of adding labels, this modifies the `Dockerfile` a bit and pins the production image to specific sha of the base image, ensuring reproducible builds. In the future we should update this sha when need be, and update the golden file (`docker/test/docker-labels.golden.json`). This also adds `hadolint` to lint our Dockerfile and applies some of the recommendations to it. A new code quality job is added, `Docker checks`, which runs hadolint and verifies that the labels are as expected. The verification is done via a bash script which grabs the labels from a `docker inspect`, and compares it with an interpolated golden file (since we have a few dynamic values). The comparison is done using `diff`, so the output should be familiar to most. ## Related issues related to #9940 blocks #10013 10012: refactor: let `dist` build the actor scheduler for gateway r=oleschoenburg a=oleschoenburg ## Description This moves the responsibility of creating an actor scheduler for the gateway to the `dist` module. As an added benefit, this allows us to remove some redundant constructors, simplify BrokerClient to never start the actor scheduler and a deprecated method from `BrokerStartupContext`. ## Related issues <!-- Which issues are closed by this PR or are related --> relates to #9996 Co-authored-by: Nicolas Pepin-Perreault <nicolas.pepin-perreault@camunda.com> Co-authored-by: Ole Schönburg <ole.schoenburg@gmail.com>
oleschoenburg
approved these changes
Aug 8, 2022
npepinpe
force-pushed
the
9940-oci-labels-release
branch
4 times, most recently
from
ef20069
to
afad210
Compare
|
I tested the release process manually, but it seems if you don't deploy the Maven artifacts the Docker job will not work. So fun. Anyway, I tested the Docker job separately manually. I had to hardcode the build params since the Jenkins always picks up what's on main first, so... 🤷 Anyway, it seems to work generally separately, I think that's OK for now. |
npepinpe
force-pushed
the
9940-oci-labels-release
branch
from
afad210
to
930c2d2
Compare
Updates the release process to generate the correct labels for the Docker image, and verify them, failing the release process (and not pushing the image) if they are wrong. Ideally, this would be done before the Maven release to avoid wasting time if the image would produce the wrong labels. Doing this is quite a bit of refactoring of the CI pipeline unfortunately, and producing the wrong labels is most likely an issue with the CI pipeline itself, which should not happen very often. Errors in the Dockerfile itself would be checked by the GHA pipeline, which should cover the important cases and runs on each pull.
npepinpe
force-pushed
the
9940-oci-labels-release
branch
from
930c2d2
to
51716bb
Compare
|
bors merge |
|
Build succeeded: |
10 tasks
zeebe-bors-camunda bot
added a commit
that referenced
this pull request
Aug 9, 2022
10017: Reusable build docker action r=npepinpe a=npepinpe
## Description
This PR aggregates the steps required to build the Docker image into a reusable action called `build-docker`. The action can build and push the Docker image, as well as package the artifacts for it if need be.
Further iterations may want to split packaging out into its own action.
Its usage is as:
```yaml
- uses: ./.github/actions/build-docker
with:
repository: localhost:5000/camunda/zeebe
version: current-test
push: true
package: true
```
The action will set up Docker & Buildx, Java, and Maven. If `package` is true, it will also setup `Go` in order to build `zbctl`.
When `push` is true, the image is pushed to the given repository, with the tag being made of the `repository` and `version`. In the above, this would be equivalent to `docker push localhost:5000/camunda/zeebe:current-test`. If `push` is false, then the image is built and loaded into the local Docker registry only.
If `package` is true, then `zbctl` is compiled, and all the modules are built and installed to the local maven repository. Additionally, the distribution TAR ball is built. If it's false, then the distribution TAR ball is expected to already be present under `dist/target/camunda-zeebe-<VERSION>-tar.gz`.
If the `version` is omitted, it will be read from the Maven project.
## Related issues
related to #10013 (as a follow up)
Co-authored-by: Nicolas Pepin-Perreault <nicolas.pepin-perreault@camunda.com>
Zelldon
added
the
version:8.1.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR updates the release process to generate the correct labels for the Docker image, and verify them, failing the release process (and not pushing the image) if they are wrong.
Ideally, this would be done before the Maven release to avoid wasting time if the image would produce the wrong labels. Doing this is quite a bit of refactoring of the CI pipeline unfortunately, and producing the wrong labels is most likely an issue with the CI pipeline itself, which should not happen very often. Errors in the Dockerfile itself would be checked by the GHA pipeline, which should cover the important cases and runs on each pull.
I opted to do the verification during the release process as it's very easy to miss that the labels are wrong, and it would somewhat unprofessional if a customer were to report it to us down the road.
Related issues
closes #9940
Definition of Done
Not all items need to be done depending on the issue and the pull request.
Code changes:
backport stable/1.3) to the PR, in case that fails you need to create backports manually.Testing:
Documentation:
Please refer to our review guidelines.