Skip to content

Commit

Permalink
merge: #10048
Browse files Browse the repository at this point in the history
10048: Common reusable actions r=npepinpe a=npepinpe

## Description

This PR aggregates the steps to set up the Zeebe environment for packaging, as well as packaging itself, into two reusable actions. By doing this, it removes the duplicate code in the `build-docker` action, and instead relies on the caller having set up and packaged Zeebe beforehand (much like we rely on the checkout step being invoked before).

`setup-zeebe` will prepare the environment to build the complete Zeebe distribution. The benefits here is that we centralize once the tech stack, how it's installed, and the versions we require, instead of spreading this everywhere. I've made it configurable so some parts can be ignored, primarily because of the `setup-go` action. If you use that, but do not do anything with Go, it will fail in its post-run when trying to clean a non-existent cache 🤷 The action requires the code to have been checked out beforehand.

`build-zeebe` will build the Go client (optional) and the Java distribution (optional). By default, it builds everything. It also requires the environment to be set up beforehand, and the code checked out. It uses the `clients/go/cmd/zbctl/build.sh` script to build the Go client, and will `install` the mvn modules, skipping checks and using `-T1C` to parallelize by default. You can customize the installation via a free-form field letting you set up additional properties to pass to the build. The benefits here is again centralization of the common set up, while allowing enough customization for the various steps.

I decided to split both set up and packaging because there are some places where we package the application differently, notably when deploying/releasing. As this will most likely be used in the future for releases, it made sense to me. So the packaging is really more of a "package for non-release cases". It could be changed to allow us to modify the command for packaging (right now you can only pass options, but cannot change the actual goals to execute), and then I suppose it could be reused. I think it might be misleading though to have a build Zeebe action which actually deploys, so I'd rather avoid that for now.

## Related issues

related to #10017 



Co-authored-by: Nicolas Pepin-Perreault <nicolas.pepin-perreault@camunda.com>
  • Loading branch information
zeebe-bors-camunda[bot] and npepinpe committed Aug 16, 2022
2 parents d5bddf2 + e5d9f43 commit df31635
Show file tree
Hide file tree
Showing 7 changed files with 177 additions and 144 deletions.
39 changes: 8 additions & 31 deletions .github/actions/build-docker/action.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
# This action expects the code to have been checked out beforehand, e.g. via actions/checkout@v3
# It will set up Java and maven as dependencies. If requested to package Zeebe, it will also set up
# Go, build zbctl, then build the Zeebe distribution.
#
# If no version is given, the version is set to the Maven project version.

---
Expand All @@ -16,10 +13,9 @@ inputs:
version:
description: 'The image version, e.g. SNAPSHOT, 8.1.0'
required: false
package:
description: 'If true, will package Zeebe, otherwise uses the tar-ball under dist/target'
default: 'false'
required: false
distball:
description: 'The path to the Zeebe distribution TAR ball'
required: true
push:
description: 'If true, will push the image'
required: false
Expand Down Expand Up @@ -51,29 +47,6 @@ runs:
driver-opts: network=host
endpoint: zeebe-context
install: true
# We need maven and Java (indirectly) to get properties from the project
- uses: actions/setup-java@v3.4.1
with:
distribution: 'temurin'
java-version: '17'
- uses: stCarolas/setup-maven@v4.4
with:
maven-version: 3.8.5
- if: ${{ inputs.package == 'true' }}
uses: actions/setup-go@v3
with:
go-version-file: 'clients/go/go.mod'
cache: true
cache-dependency-path: 'clients/go/go.sum'
- if: ${{ inputs.package == 'true' }}
name: Build Go
shell: bash
run: ./build.sh
working-directory: clients/go/cmd/zbctl
- if: ${{ inputs.package == 'true' }}
name: Package Zeebe
shell: bash
run: mvn -B -DskipTests -DskipChecks package -T1C
- name: Set semantic version from Maven project
id: get-version
shell: bash
Expand All @@ -86,6 +59,10 @@ runs:
id: get-image
shell: bash
run: echo "::set-output name=result::${{ inputs.repository }}:${{ inputs.version || steps.get-version.outputs.result }}"
- name: Set DISTBALL path relative to the build context
id: get-distball
shell: bash
run: echo "::set-output name=result::$(realpath --relative-to="${PWD}" ${{ inputs.distball }})"
- name: Build Docker image
uses: docker/build-push-action@v3
with:
Expand All @@ -95,7 +72,7 @@ runs:
push: ${{ inputs.push }}
no-cache: true
build-args: |
DISTBALL=dist/target/camunda-zeebe-${{ steps.get-version.outputs.result }}.tar.gz
DISTBALL=${{ steps.get-distball.outputs.result }}
DATE=${{ steps.get-date.outputs.result }}
REVISION=${{ github.sha }}
VERSION=${{ steps.get-version.outputs.result }}
Expand Down
45 changes: 45 additions & 0 deletions .github/actions/build-zeebe/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# This action packages the complete Zeebe distribution artifacts. This includes the Go client,
# zbctl, and the Zeebe distribution TAR ball. This excludes the Docker image. See the build-docker
# for that.

---
name: Build Zeebe
description: Builds & installs the complete Zeebe distribution

inputs:
go:
description: If false, will not build zbctl; defaults to true
default: "true"
required: false
java:
description: If false, will not build the Java distribution; defaults to true
default: "true"
required: false
maven-extra-args:
description: Additional CLI arguments which will be passed to the maven install command as is, e.g. "-am -pl util/"
default: ""
required: false

outputs:
distball:
description: "The path to the Zeebe distribution TAR ball"
value: ${{ steps.build-java.outputs.result }}

runs:
using: composite
steps:
- if: ${{ inputs.go == 'true' }}
name: Build Go
shell: bash
id: build-go
working-directory: clients/go/cmd/zbctl
run: ./build.sh
- if: ${{ inputs.java == 'true' }}
name: Package Zeebe
shell: bash
id: build-java
run: |
mvn -B -DskipTests -DskipChecks -T1C install ${{ inputs.maven-extra-args }}
export BUILD_DIR=$(mvn -pl dist/ help:evaluate -Dexpression=project.build.directory -q -DforceStdout)
export ARTIFACT=$(mvn -pl dist/ help:evaluate -Dexpression=project.build.finalName -q -DforceStdout)
echo "::set-output name=result::${BUILD_DIR}/${ARTIFACT}.tar.gz"
45 changes: 45 additions & 0 deletions .github/actions/setup-zeebe/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# This action sets up the environment with the required tech stack in order to build, install, and
# run Zeebe.

---
name: Setup Zeebe
description: Sets up the required stack to build, install, and run Zeebe

inputs:
go:
description: If true, will set up Golang; defaults to true
required: false
default: "true"
java:
description: If true, will set up Java; defaults to true
required: false
default: "true"
java-version:
description: The JDK version to setup
default: "17"
required: false
maven-version:
description: The Maven version to setup
default: "3.8.5"
required: false

outputs: {}

runs:
using: composite
steps:
- if: ${{ inputs.java == 'true' }}
uses: actions/setup-java@v3.4.1
with:
distribution: 'temurin'
java-version: ${{ inputs.java-version }}
- if: ${{ inputs.java == 'true' }}
uses: stCarolas/setup-maven@v4.4
with:
maven-version: ${{ inputs.maven-version }}
- if: ${{ inputs.go == 'true' }}
uses: actions/setup-go@v3
with:
go-version-file: 'clients/go/go.mod'
cache: true
cache-dependency-path: 'clients/go/go.sum'
29 changes: 10 additions & 19 deletions .github/workflows/code-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,24 +17,17 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3.4.1
- uses: ./.github/actions/setup-zeebe
with:
distribution: 'temurin'
java-version: '17'
- name: Set up Maven
uses: stCarolas/setup-maven@v4.4
with:
maven-version: 3.8.5
go: false
- name: Use CI Nexus cache
uses: ./.github/actions/use-ci-nexus-cache
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: java
queries: +security-and-quality
- name: Build
run: mvn -B -T1C -DskipTests -DskipChecks install
- uses: ./.github/actions/build-zeebe
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
Expand All @@ -58,11 +51,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
- uses: ./.github/actions/setup-zeebe
with:
go-version-file: 'clients/go/go.mod'
cache: true
cache-dependency-path: 'clients/go/go.sum'
java: false
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
Expand All @@ -73,11 +64,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3.4.1
with:
distribution: 'temurin'
java-version: '17'
- run: mvn -B -D skipTests -D skipChecks install
- uses: ./.github/actions/setup-zeebe
- run: mvn -T1C -B -D skipTests -P !autoFormat,checkFormat,spotbugs verify
docker-checks:
name: Docker checks
Expand All @@ -95,12 +82,16 @@ jobs:
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: ./hadolint.sarif
- uses: ./.github/actions/setup-zeebe
- uses: ./.github/actions/build-zeebe
id: build-zeebe
- uses: ./.github/actions/build-docker
id: build-docker
with:
# give it a fake name to ensure we never try pushing it
repository: localhost:5000/camunda/zeebe
package: true
distball: ${{ steps.build-zeebe.outputs.distball }}
- name: Verify Docker image
env:
DATE: ${{ steps.build-docker.outputs.date }}
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,16 @@ jobs:
with:
username: ${{ steps.secrets.outputs.REGISTRY_HUB_DOCKER_COM_USR }}
password: ${{ steps.secrets.outputs.REGISTRY_HUB_DOCKER_COM_PSW }}
- uses: ./.github/actions/setup-zeebe
- uses: ./.github/actions/build-zeebe
id: build-zeebe
- uses: ./.github/actions/build-docker
id: build-docker
with:
repository: camunda/zeebe
version: SNAPSHOT
package: true
push: true
distball: ${{ steps.build-zeebe.outputs.distball }}

notify-if-failed:
name: Send slack notification on build failure
Expand Down
40 changes: 13 additions & 27 deletions .github/workflows/qa-testbench.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,42 +74,28 @@ jobs:
secret/data/products/zeebe/ci/zeebe ZEEBE_GCR_SERVICEACCOUNT_JSON;
secret/data/products/zeebe/ci/zeebe TESTBENCH_PROD_CLIENT_SECRET;
secret/data/products/zeebe/ci/zeebe TESTBENCH_PROD_CONTACT_POINT;
- uses: actions/setup-go@v3
with:
go-version-file: 'clients/go/go.mod'
cache: true
cache-dependency-path: 'clients/go/go.sum'
- name: Build Go
run: ./build.sh
working-directory: clients/go/cmd/zbctl
- name: Package Zeebe
run: mvn -B -DskipTests -DskipChecks package

- name: Login to GCR
uses: docker/login-action@v2
- name: Login to GCR
uses: docker/login-action@v2
with:
registry: gcr.io
username: _json_key
password: ${{ steps.secrets.outputs.ZEEBE_GCR_SERVICEACCOUNT_JSON }}
- uses: ./.github/actions/setup-zeebe
- uses: ./.github/actions/build-zeebe
id: build-zeebe
- uses: ./.github/actions/build-docker
with:
registry: gcr.io
username: _json_key
password: ${{ steps.secrets.outputs.ZEEBE_GCR_SERVICEACCOUNT_JSON }}

- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
tags: "${{ env.IMAGE }}:${{ env.TAG }}"
repository: ${{ env.IMAGE }}
version: ${{ env.TAG }}
push: true
no-cache: true
build-args: DISTBALL=dist/target/camunda-zeebe-*-SNAPSHOT.tar.gz
target: app
distball: ${{ steps.build-zeebe.outputs.distball }}
# Executes the Testbench QA run and awaits the result
- name: Run Testbench QA
run: .ci/scripts/distribution/qa-testbench.sh
env:
ZEEBE_CLIENT_SECRET: ${{ steps.secrets.outputs.TESTBENCH_PROD_CLIENT_SECRET }}
ZEEBE_ADDRESS: ${{ steps.secrets.outputs.TESTBENCH_PROD_CONTACT_POINT }}


notify:
name: Send slack notification
runs-on: ubuntu-latest
Expand Down

0 comments on commit df31635

Please sign in to comment.