ci: overhaul and release automation

[k3rnelpan1c-dev]

Description

In the same veins as DependencyTrack/dependency-track#1419, this PR offers a CI overhaul and migrates the current release workflow from a bash script based solution to a GHA Workflow.

I took the liberty to include a proposal configuration for renovate-bot, as I personally find that a better configurable dependabot alternative for frontend repositories. An example of what this config would produce

Changes

• update current CI workflows
• overhaul the current Containerfile
• add a Debian flavoured Containerfile
• add a release workflow (can handle pre- and regular rerelease)
• propose a renovate-bot config as it supports grouping and very granular configuration (opted to move to followup PR)

Issues

• addresses Migrate Docker image from Alpine to Debian Slim #78 (by adding a Debian flavour)
• closes Migrate Docker image from Alpine to Debian Slim #78 by discussing and opting to keep Alpine Linux for the frontend

[nscuro]

Overall this looks solid, just a few minor comments that may need clarification.

Also huge +1 for using Renovate, it's much better than Dependabot. It will need to be activated first however.

[nscuro]

@k3rnelpan1c-dev @stevespringett I think we should decide on either Debian or Alpine. There's really no reason to have both. I'd personally vouch for Alpine - smaller attack surface and I've never seen issues with it. See also @k3rnelpan1c-dev's response here: #78 (comment)

I'd rather not have the complexity of handling multiple flavors in CI if we can avoid it.

Thoughts?

[stevespringett]

The API Server moved to Debian due to use of glibc and to avoid some issues that Alpine has with K8s.

The frontend we have a bit more flexibility over. I also prefer Alpine due to the smaller attack surface, but I need to find out what that K8s issue with Alpine was. But I would prefer to have a single base image rather than multiple.

[k3rnelpan1c-dev]

I cannot imagine Alpine having any issue on K8s, after all the most commonly used external Ingress controller happens to be Nginx, which is using its Alpine based version IIRC.

But I am curious, if you happen to find the reference / remember what this issue might have been that would be great input. The only "issue" that I am aware with Alpine was that programs relying on glibc could experience very odd behaviour, however that was in every container environment not just K8s.

[nscuro]

Known issues appear to evolve around DNS resolution. Small selection:

• Dns server is null in some pods. kubernetes/kubernetes#30215
• Containerization image improperly resolves DNS dependency-track#1054
• ndots breaks DNS resolving kubernetes/kubernetes#64924
• DNS resolve issues in Alpine docker image grafana/grafana#20153

The K8s team themselves claim:

If you are using Alpine version 3.3 or earlier as your base image, DNS may not work properly due to a known issue with Alpine. Kubernetes issue 30215 details more information on this.

The frontend doesn't do any DNS resolution though. Even if the issue(s) above persist, it won't be a problem for this particular container.

[k3rnelpan1c-dev]

🤦‍♂️ I actually heard of that before. Thank you for the detailed response!

Side-note: fun that it is used for an ingress then when using the Nginx Ingress xD

Edit:
I second that, even should this bug still exists today in Alpine Linux 3.15+, that the frontend would continue to work as all it does is host the static HTML and JS via an ingress, reveres proxy or directly to to a end user. Thereby not needing any DNS resolution at all.

[k3rnelpan1c-dev]

took the liberty and removed the Debian flavour from the build and release process, as we all seem to be in agreement that we would prefer to only offer one flavour and that the minimal attack surface of Alpine Linux is favourable.

[nscuro]

@k3rnelpan1c-dev Apologies for my commits. Wanted to test in my fork but forgot to specify the correct remote when pushing my changes... May I ask you to hard-reset them, verify that everything's still as you intended and force push? 🥲

[nscuro]

Anyway, I was now able to test this and it works like a charm!

• Build CI: https://github.com/nscuro/frontend/actions/runs/2316174514
• Build Release: https://github.com/nscuro/frontend/actions/runs/2316236491
• Release: https://github.com/nscuro/frontend/releases/tag/4.5.0
• Containers: https://github.com/nscuro/frontend/pkgs/container/frontend

Once we have my mess 🥲 cleaned up it's good to go from my side.

[k3rnelpan1c-dev]

all done + added a tiny hotfix that we should consider adding to the backed CI eventually.
The login against docker.io should only really run when we are actually attempting to push the image to it (or any registry for that matter).

edit:
applied the hotfix with another oversight fix to the backbend CI 😃
DependencyTrack/dependency-track#1623

[k3rnelpan1c-dev]

Thank you for testing this nscuro :)
+ I could have locked the PR if I wanted to prevent upstream maintainers to change my changes, so no harm done in any way.

[nscuro]

All good from my side, just need @stevespringett to approve it before we can merge.

[stevespringett]

Huge thanks @k3rnelpan1c-dev

[k3rnelpan1c-dev]

I enjoy working on CI and automation, happy to have helped out :)
thank you for the great project!

[nscuro]

It looks like, when releasing, _meta-build.yml/build-node does not "see" the version bump from ci-release.yml/prepare-release. actions/checkout per default checks out the Git commit that triggered the workflow run, which will always be the one before npm version was called in ci-release.yml/prepare-release.

For example, https://github.com/DependencyTrack/frontend/runs/6483307556?check_suite_focus=true checked out 01b253c instead of 3fe6bf7.

This results in the version field of package.json not being correct, which also shows in the UI:

I'm wondering what would be the best strategy to resolve this. Some ideas:

• Decouple prepare-release from the rest of the workflow. Essentially, only run npm version (which creates a Git tag)
  • Introduce another workflow that's triggered by new tags. Run builds and create-release in that workflow
• Somehow make _meta-build.yml aware that the to-be-built commit changed. I feel like this may be wonky and error prone.

Would love to get your input @k3rnelpan1c-dev, your expertise is greatly appreciated.

[k3rnelpan1c-dev]

Oha, well this is interesting and certainly unexpected behaviour 🤔
I will try and toy around with this the coming weekend and see whats up with this behaviour.

Anyway, for feedback on your workaround proposals: Both can work.
We could split off the build and publish part of the release CI and have it only create a GH Release that would trigger a build + publish CI or, give I can figure out why GHA behaves the way it does, we fix whatever causes it. (given we can, as it dose not seem to have that problem in the backend CI)

EDIT:
I too am not a big fan of somehow passing the ref to check out to the "_meta" workflow.

[k3rnelpan1c-dev]

Okay I thought of this a bit more and it actually makes sense that by default every checkout step within the same workflow checks out the state that it got triggered with .... (and now I am annoyed that I didn't think of this earlier)

In other news I have a feeling that the idea of creating a pure "create release" workflow and letting a "publish" workflow trigger on "release creation" might be the right way to move forward.

I will analyse more, but otherwise would commit to that idea and iterate over the CI that way.
Given you are on board with that idea, ofc.

[k3rnelpan1c-dev]

Okay, tiny problem with the separated workflows: No matter how I change it using the default GITHUB_TOKEN cannot be used to realise it ☹️

Note on GitHub Actions: You can use the default token which is provided in the secret GITHUB_TOKEN. However releases done with this token will NOT trigger release events to start other workflows.
If you have actions that trigger on newly created releases, please use a generated token for that and store it in your repository's secrets (any other name than GITHUB_TOKEN is fine).

So to say that I am stuck with that idea is quite accurate, unless we opt to setup a "Dependency-Track-Bot" account that will serve as PAT donor to create the release and commits to the projects on release.