The Detekt Release Process

[cortinico]

Hi all,
It's time to talk about the elephant in the room: our release process :)

I'm opening this discussion to provide an insight on how it currently looks like and how it can be improved. Sadly, the current release process is really manual and requires a lot of steps, which we should look into automatizing.

Moreover, there are some processes that are fundamentally broken, which we should look into.

How does the release process looks like today.

To kickoff a release of Detekt, there is some preparatory work that needs to be done & a list of steps to kickoff a release.

Preparatory work

1. Make sure that the latest milestone doesn't have open issues/PR that are critical (that's mostly a call of the release manager to decide if anything can be postponed or not).
2. Make sure that all the PRs that landed have a milestone set up correctly.
3. That's critical as otherwise they will not appear in the release notes.
4. Maybe we could look into setting up a Github Actions or Danger that automatically sets the milestone once the PR is submitted or so.
5. Make sure all the landed PRs are labelled correctly (either housekeeping, notable changes, dependency or any other label) as they will be used for the Changelog Generation script

Actual release process

1. Bump the version here:
   detekt/Versions.kt at e052dbe94c24a8a3b323a9b023722455c7ff1ec4 · detekt/detekt
2. Publish the version to Maven Local with ./gradlew publishToMavenLocal
3. Run a full build with all the tests with ./gradlew build
4. Here is where the majority of the things break. As we don't rely on Composite builds, a lot of things pushed on main are not actually in use, specifically inside the Gradle plugin. If things break, you need to fix them.
5. Sometimes we even have to ask users to split their PRs into two as the CI won't be green with them (Better error classification in Gradle Enterprise. #4586 Follow up: Better error classification in Gradle Enterprise. #4588)
6. Publish everything to Maven Central with ./gradlew publishAllPublicationsToMavenCentralRepository --max-workers 1
7. To do so, you need a valid signing key configuration. You need to have access to the private GPG Key of Detekt (ask @cortinico for that)
8. You also need a valid set of username/password for sonatype configured (ask @cortinico for that)
9. Publish the Gradle Plugin to Gradle Portal with ./gradlew publishPlugins -DautomatePublishing=true
10. To do so, you need the valid API key for the Gradle portal configured (ask @cortinico for that).
11. You can release the Sonatype repository to Maven Central with ./gradlew closeAndReleaseRepository
12. You can bump the version inside the Docs and the version of the Gradle plugin we're using with: ./gradlew applyDocVersion applySelfAnalysisVersion
13. Generate the Release notes with ./scripts/github-milestone-report.main.kts
14. Create a PR with all the changes and send it to Detekt.
15. Wait till the PR is green (it needs time as it relies on the newly published artifacts to be green)
16. Once the PR is merged, you can publish a new Github Release with ./gradlew githubRelease

RC release process

Releasing a RC is essentially equivalent to releasing a stable version. The only difference is that, when generating the release notes, you'll have to ./scripts/github-milestone-report.main.kts -f as it will filter out PRs already listed in the changelog.

Point release process

When doing a point release, things are really similar, but you need to pay attention to a couple of extra points.

1. Point releases need a git branch. Usually the extra branch is called like release/1.18.1 You need to cut a release branch from the release tag.
2. You can then cherry-pick all the needed changes.
3. Bump the version number and perform all the publishing.
4. Send a PR out to merge the release branch against main.
5. Disable the Squash Merge inside the Detekt settings
6. Update the releasing.gradle.kts script here with the sha of the commit where you're bumping the version. This is needed as ./gradlew githubRelease will tag main otherwise.
7. Run ./gradlew githubRelease
8. Merge the PR

What are the problems

I see a number of problems with the current process. Here the one that comes on top of my mind.

0. RCs are pointless as we keep on merging things. Marked as 0 as I believe this is the root cause of our pain. We do RCs for historical reasons, but those are not Release Candidate in any form. Is just a way that we have to bundle our code and ask for some feedback from the community on the stability of Detekt. The problem is that between a RC1 and a RC2, we keep on merging changes that are not necessarily related to reported regression. So in a sense, we can't guarantee that RC2 is more stable than RC1. If we find regressions, we addresse them, but we can't guarantee that we haven't introduced more. I think that this can be addressed by either

   1. Switching to full release branch. Once we start doing 1.21, we cut a release branch, we do an RC, and then we set a feedback window of say X days. All the regression reported in that window, we will look into them. If there is a fix, we do a RC+1, until there are no more regressions reported. At that stage we call it a stable release.
   2. Switching to a always releasable main branch, deprecate the RCs and switch to full semantic versioning. This is similar to what we have now. Essentially we will be releasing more often, but without RCs or similar. We will just release a 1.21.0 at a certain point in the future. Should there be a regression, we can address them and do a point release. This also mean that if we release 1.21.0 at Day0, and we merge a PR for a new rule at Day1, then we find a regression for 1.21.0, fix & merge it on Day2, we are effectively releasing 1.22.0 two day after the release of the previous minor. We would have to clarify to our users that we're changing this.

1. Releases are built on local machine This needs to be changed. GPG Keys, Sonatype passwords and Gradle API keys should be stored as secrets on Github Actions. Specifically as we already have a Publish Snapshot workflow that is doing something similar so we already have most of the setup in-place.

2. Depending on Maven Local We can't discover at release time that our Gradle Plugin is broken. This adds a lot of burden to the release process and should instead be up to the PR author to make sure the Gradle Plugin works correctly. This should be easy to do with an Included build

3. Releasing using an older version of the Gradle plugin Similarly to the previous point, the whole release process is done with the previous version of the Detekt Gradle plugin. So by the time something breaks, that's too late as you already published a potentially broken plugin.

4. The Gradle plugin is not on Maven Central For some reason we're not publishing the Gradle Plugin to Maven Central (but only to Gradle Portal). I.e. we need to ./gradlew publishPlugins -DautomatePublishing=true. I believe that this should be changed and I'm unsure to why was this set up in this way. It makes really hard to work with Maven Local as the Gradle Plugin marker is not published there (for the similar setup). Similarly, users of our SNAPSHOT can't easily try the Gradle Plugin as the marker is missing on Sonatype, etc.

5. Release Notes are difficult to write I personally write the release notes, starting from the automated script. It's not an easy task as you need to go through all the PRs marked as notable changes and find a short sentence to describe it. Ideally we should have more precise labels like rules:added, rules:new-config, fixed:false-positive, fixed:regression and have a more categorized changelog. That's space for another topic.

6. We're not always tagging the correct commit As the ./gradlew githubRelease is done manually and always tags the commit on top of main, we sometimes tag a wrong commit. This happens if, by the time the Release PR is merged, another PR got merged first. What this means is that we're effectively tagging a SHA with a version that contains a change that we haven't shipped. Should we make a point release from that tag, we will include extra changed that were not intended.

7. PRs land without a milestone As mentioned above, we do have PRs landing without a milestone. This means that they won't show up in the release notes and people will have a hard time finding in which version such a change landed.

8. Release timing is unclear Not having a clear schedule makes hard for people to depend on us. As we don't know when say 1.20.x will be out, it's hard for tools that are building on top of Detekt to decide when to release and so on.

9. The Release PR can't be green till the release is available online The fact that we do have a "Prepare Detekt 1.21.0" PR that can't be merged till Detekt 1.21.0 is released and published to the public is a sympthom of our setup. In an ideal world we should:

   1. Send a PR to prepare the release (just bump the version)
   2. Make sure that all the tests are green (we should be able to test against a new version of Detekt before it hits the market)
   3. Tag the version (either manually or with a Github Action)
   4. Trigger the Release workflow once the tag is published
   5. Have the release workflow with a manual approval just to have a human check verifying that we're publishign somethign valid.

---

Before we jump into solution mode, I'm happy to hear some opinions and alternatives on tools & solutions that could help us here.

[TWiStErRob]

Re problem 4 (mavenCentral Vs gradlePluginPortal): the order has to be specific (not sure which way) or all together: during a recent 1.20.0 release dependabot eagerly updated one of my repos, and I got this failure:

A problem occurred configuring root project 'Sun'.
> Could not determine the dependencies of null.
   > Could not resolve all task dependencies for configuration ':classpath'.
      > Could not find io.gitlab.arturbosch.detekt:detekt-utils:1.20.0.
        Searched in the following locations:
          - https://repo.maven.apache.org/maven2/io/gitlab/arturbosch/detekt/detekt-utils/1.20.0/detekt-utils-1.20.0.pom
          - https://plugins.gradle.org/m2/io/gitlab/arturbosch/detekt/detekt-utils/1.20.0/detekt-utils-1.20.0.pom
        Required by:
            project : > io.gitlab.arturbosch.detekt:io.gitlab.arturbosch.detekt.gradle.plugin:1.20.0 > io.gitlab.arturbosch.detekt:detekt-gradle-plugin:1.20.0

used as

plugins {
	alias(libs.plugins.kotlin.detekt) apply false
}

[cortinico]

Re problem 4 (mavenCentral Vs gradlePluginPortal): the order has to be specific (not sure which way) or all together: during a recent 1.20.0 release dependabot eagerly updated one of my repos, and I got this failure:

The problem is that the Detekt Gradle Plugin depends on detekt-utils here

detekt/detekt-gradle-plugin/build.gradle.kts

Line 69 in 42e4d0a

implementation(projects.detektUtils)

So if you update the Gradle pluign, you also need to have detekt-utils and all the dependecies of that artifacts published somewhere (Maven Central). As Maven Central is slower to update than Gradle Portal, that's what you're experiencing.

IMHO We should break the dependency between the Gradle Plugin and detekt-utils.

[cortinico]

This is handled here #4748

[TWiStErRob]

On a related note, this is an interesting approach: https://github.com/jfrog/build-info/pull/536/files

[BraisGabin]

Nico, thank you so much for all these details!

My ideas:

• The changelog is a pain. It force us to be careful with milestones, tags and then you even need to write one line for each notable change. Some of them I assume that are "easy" but other should be tricky.
• The RC and patch versions are a pain and directly wrong because our release process is a mess. I think that right now we can just ignore that issue because, maybe, they will be fixed just by fixing the "normal release".
• We need to fix the issue of unsinc gradle plugin and the core. To be honest I don't completely understand that problem but for sure it's a pain point and it would be great if someone could fix it.
• We should create a kotlin/bash script to automate the release. Once we have that we can move it to a Github Action.

I have a clear idea to fix the first point. But I'm going to propouse it in a new "thread" to avoid overlaping topics.

[TWiStErRob]

We should create a kotlin/bash script to automate the release. Once we have that we can move it to a Github Action.

Careful, separate fixing the process and automating it, because you might end up automating a broken process.

[BraisGabin]

Sure! sure! We should automate only the parts that are "critical": compile, sign, upload to maven central, etc. That line was basically that before we can fix "Releases are built on local machine" we should automate them with a script and make them on local machine and THEN move to Github Actions.

[cortinico]

Agree on what @TWiStErRob mentioned. Automation is a further step.
The core of the problem is that "we depend on ourself" (here

detekt/gradle/libs.versions.toml

Line 44 in 7960453

detekt = { id = "io.gitlab.arturbosch.detekt", version = "1.20.0" }

) and we should move to a composite build.

[BraisGabin]

The changelog is an issue that I think that a lot of projects have (OSS and no OSS). There are a lot of proposals out there but I didn't find a good one. Random ideas:

• The changelog should not be a list of PRs/commits. If someone wants that it can make a git log.
• This project get a lot of changes in each release. It's very difficult to keep track to all of them.
• Forcing people to add a line in the unreleased changelog creates lots of conflicts. I did this in a project before and the conflicts were terrible. But the changelog was crazy good. All was written by a human with the complete knowledge of what was implemented.
• Tags and milestones are great but we can't relay on them.
• The users really like how we work with milestones, we should keep doing that.

In that project that I talked in the point 3 we found a solution that worked for us:

• In ALL the PRs you should crear a new text file inside the firectory changelog/.
• In that text file you should write the line that you want to add to the changelog once your code is released.
• One of the steps of the release process was to execute a code that was more less like this:

cat changelog/* >> CHANGELOG.md
rm -f changelog/*

This solves nearly all the problems:

• All the lines are written by a human. And they are not based on commit or PR message.
• There are no conflicts! Each person add a new file so git is completely fine with it.
• In the PR you can review what that person wants to add in the changelog! That helps to make the changelog more collavorative. Github even allow us to propouse a change on it!

But it adds one big problem: What happen if someone doesn't add it? It's strange that a project ask you to do this so nearly no one will do it.

My solution to this problem is to create a bot. That bot should have this features:

• If a PR doesn't have a file added to changelog/ it should add a request for changes and explain that a changelog should be added.
• The bot should allow to add that changelog file just by adding a comment in the PR. Something similar to what dependabot does. For example:
  • /changelogAdd Fix false positive on rule ABC. Will add a commit to the PR adding a file to changelog/ with that message.
• The bot should also allow to decide that "I don't want a changelog for this PR". In some cases that have sense. But the bot should ask for a reason so the reviewer can decide if that reason is valid or if it is not. Example:
  • /changelogSkip This PR is just a fix over other PR that already have a changelog and it was not published yet.

Things missing: We still need to add the correct milestone. The good part is that now we can create a bot that automatically add the milestone "next-release" to all the merged PRs. And then we just need to change the name of that milestone to the correct release and done. We can do that with another script. And we will not need any label.

[TWiStErRob]

Really nice idea!
Building on this: the file name could contain some specific naming for grouping?
"new_rule_1123", "new_feature_2523", "breaking_2434"

[BraisGabin]

Sure about naming. The is os kind of tricky because you need to add the file before the pr. But that's not that important.

[cortinico]

I'd like to suggest a different approach: I think we can solve this easily by using Danger in this repo (cc @f-meloni @gianluz :) ).

This will make sure that:

1. We have all the PRs merged with a valid milestone
2. We have all the PRs merged with a "Changelog" paragraph that needs to follow something like a conventional commit syntax. Something along the way of:

## Changelog Entry
[fix] [rulename] Resolved false positive on bla bla...

We can then parse those when generating the changelog.
3. Danger should also help us on some of the labelling work that labeller can't do at the moment (as it's really simple and does not much more than path regex matching).

[f-meloni]

Hey @cortinico :) Yes I agree Danger can definitely help with that:

We have all the PRs merged with a valid milestone

This is straightforward, if https://github.com/danger/kotlin/blob/master/danger-kotlin-library/src/main/kotlin/systems/danger/kotlin/models/github/GitHub.kt#L87 is null then Danger can fail the build and make the PR not mergiable (In case you make Danger status required for merge).
This approach requires Danger to run when you make a new commit, but also when a milestone is added/removed from a PR. GitHub Actions should already offer an event for that (https://docs.github.com/en/github-ae@latest/actions/using-workflows/events-that-trigger-workflows#milestone).

We have all the PRs merged with a "Changelog" paragraph that needs to follow something like a conventional commit syntax. Something along the way of:
....

If you have a "parser" for the changelog, should be simple to make Danger run that and see if fails or not, and make Danger fail if it fails.

Danger should also help us on some of the labelling work that labeller can't do at the moment (as it's really simple and does not much more than path regex matching).

Danger misses only one thing in order to make this possible, which is present for example in Danger Swift, and is an API Client for GitHub, however this is just a REST API call, so should be possible to make it ad hoc directly on the Dangerfile .

I'm happy to provide some help with this if needed :)

[cortinico]

I'm happy to provide some help with this if needed :)

Awesome, thanks for that.
Happy to hear others opinion about this flow

[TWiStErRob]

Separating out the milestone discussion

@BraisGabin said Things missing: We still need to add the correct milestone. The good part is that now we can create a bot that automatically add the milestone "next-release" to all the merged PRs. And then we just need to change the name of that milestone to the correct release and done. We can do that with another script. And we will not need any label.

[TWiStErRob]

I think milestones serve a double purpose:

1. you track what you want to do in the release
2. you track what has been done in the release

2 is unnecessary until the release is done, so how can it be improved? How about this process?

• While a release is being prepared (development ongoing) the milestone is used to track what the team wants to do. There's no need to milestone every single PR, only the ones that you definitely want done.
• Some PRs (e.g. contributors) might get merged without milestone, that's ok.
• During the release process the following happens:
  • Move all PRs from milestone "X" to "X punted"
  • Automatically pick up all merged PRs from commit diff between "X-1" and "X" releases.
    (Note: this works because detekt is squashing PRs)
  • Change the milestone of all those PRs to "X"

This leaves us with a state after release:

• All PRs are milestoned that are actually merged
• All not included PRs are grouped in some milestone ("X punted" = vNext?)

Problem: milestone-ing issues related to PRs. But GitHub has a way of linking these which could be queried: the commits -> PRs -> linked issues

[TWiStErRob]

This of course assumes that "13. Generate the Release notes with ./scripts/github-milestone-report.main.kts" is either executed after this process, or simply not using milestones as source of truth.

[BraisGabin]

I agree! We used milestones to mark things that we do want in the next release. So we should keep doing that. But we should be able to automatically add the milestone of we didn't add it after merge/before release.

[cortinico]

But we should be able to automatically add the milestone of we didn't add it after merge/before release.

I'm more for auto-adding to the milestone by default, and removing the milestone if unnecessary.
If a user is fixing a typo on the website or on the README, that doesn't need a milestone. Otherwise it should be added automatically