Use a release notes generator

[coilysiren]

surfaced via #963 (comment)

[j0n3lson]

Hey there,

Been stalking this repo for a while. This one seems interesting to me :)

Would using kubernetes release notes (as suggested) be suitable or did you have something else in mind as well?
https://github.com/kubernetes/release/tree/master/cmd/release-notes

[j0n3lson]

I did take a look at release-notes. I think the only draw back is that it requires the GitHub API token. I can't think of a general way (e.g. anyone can publish the release notes with out access to the GitHub token).

Would be nice if release-notes had an option to look at the local git history instead of querying GitHub :(

Thoughts?

[saschagrunert]

I did take a look at release-notes. I think the only draw back is that it requires the GitHub API token. I can't think of a general way (e.g. anyone can publish the release notes with out access to the GitHub token).

Would be nice if release-notes had an option to look at the local git history instead of querying GitHub :(

Thoughts?

It parses the PR body for a release-notes block so it will not work without GitHub API access. You just need a read-only token which can be generated by anybody with a valid user account. The tool also supports JSON output, so we could use our own template around it to show-up the notes.

[AudriusButkevicius]

Github actions (if we do move to that), provide tokens

[j0n3lson]

I'd be happy to take this up then 🤗. Is there any existing documentation on the current release process?

[coilysiren]

the current release process?

I do the following steps:

1. I create the release notes Releases 1.22.2 and 2.0.0 #935
2. git pull the repo
3. git checkout the commit mentioned in Releases 1.22.2 and 2.0.0 #935 that corresponds to a particular release version
4. git tag v...
5. git push --tags v...
6. open the release page, for example https://github.com/urfave/cli/releases/tag/v2.0.0
7. copy the release notes from Releases 1.22.2 and 2.0.0 #935 into the release page https://github.com/urfave/cli/releases/tag/v2.0.0

[coilysiren]

@j0n3lson assigned this to you! 🚀

[j0n3lson]

I experimented with a few options today:

Experimented with

Nightapes/go-semantic-release

Pros

• Easy to configure (stock config file gets you generated CHANGELOG and can create a release.
• Manages semantic versioning: Can automatically increment or set a specific value on release
• Uses Angular JS Commit format to figure out what the next version should be automatically.
• You can set a release version template easily.
• Standardized formatting for commit messages!
• Supports markdown output
• Auto-updates CHANGELOG
• Lightweight, written in golang.

Cons

• Only supports angular commit message format. I see heavy usage of markdown format so this might be a deal breaker
• No support for PRs -- doesn't grab original author or PR#

Kubernetes Release Notes

Pros

• Supports PRs
• No config needed
• Supports markdown output
• Auto-updates CHANGELOG (the output of the command is a markdown)

Cons

• Does not manage semantic versioning
• You need to have a github organization in order to use it (Is urfave already a github org?)
• You have to calculate start and end SHAs (e.g. START_SHA=$(git rev-list -n 1 v2.0.0), END_SHA=$(git rev-list -n 1 HEAD)
• Not really lightweight (as a lot of dependencies judging by go get ...)

Honorable Mentions

semantic-release/semantic-release

Pros

• Supports PRs (example release notes)
• Manages semantic version
• CI (including Github Actions) out of the box
• Support markdown output
• Manages Changelog (via plugin)

Cons

• Additional dependency on nodejs (the entire project is nodejs) -- Another environmental to manage
• Pretty heavy weight
• There might be learning curve

I didn't mess around with this because I didn't feel like messing with nodejs which can be wonky to install sometimes.

Summary

• If we want to get up and running with generated change logs and releases, Nightapes/go-semantic-release seems like the best fit abut attribution will not be great (no PR support).
• Kubernets release can may work but I'd need someone to make urfav a github organization.

[j0n3lson]

I think also with Nightapes/go-semantic-release we can easily integrate with the exiting Github Actions setup (I'm no expert here). We'd probably need another action to run on merge and it would:

1. Fetch Nightapes/go-semantic-release
2. go mode download dependencies
3. Generate the change log and commit
4. Create the release (TODO: test that it release notes == change log)
5. Push changelog to master

If we go this route we would want:
1.1 Update project documents [Make it work]
1.1.1 Document that we're moving to Angular JS commit format
1.1.2 Document how to manually do a release with go-semantic-release (install, commands, etc)
1.1.3 Distribute a basic githook that checks angular js commit message format (for contributers)
1.1.4 Authorize a bot to check the PR commit messages (something like https://probot.github.io/apps/semantic-pull-requests/)

2.1 Integrate with existing workflow [Make it better]
2.1.1 Create a GitHub action to run on merge and do what I described earlier.

What do you all think?

[AudriusButkevicius]

I think the thing we're mostly after is something that compiles release notes, version tagging et al and actually performing the release is somewhat secondary objectives (perhaps I am wrong, as I've been pretty hands off).

My vision of what I would want (but I am not the one making the releases so it's irrelevant to some extent).
Add a status check on PRs forcing them to be tagged skip-release-notes, release-note-bugfix kr release-note-feature, and then something that takes all PR's since last tag/release, filters based on release note tags, and computes small blob of text with PR titles (potentially status check could enforce some more strict PR naming).

Do any of the tools do something like that?

[saschagrunert]

The Kubernetes release notes generator tests for a release-notes Code Block which could be part of the pull request template. It supports a somewhat opinionated markdown (sorted by special interest group label, what we don’t have here) and JSON output. So we could write a simple tool around the generator and use the JSON to print the notes into our own template, like labeling PRs by “feature” or “bugfix”.

OTOH we could add another option for the release notes generator to use a template as input, which is an idea I had in the past but did not implemented it right now (because there were no external consumers of this tool.

[j0n3lson]

I think the thing we're mostly after is something that compiles release notes, version tagging et al and actually performing the release is somewhat secondary objectives (perhaps I am wrong, as I've been pretty hands off).

Add a status check on PRs forcing them to be tagged skip-release-notes, release-note-bugfix kr release-note-feature

I thought this project might like using labels vs parsing commit messages which seems to agree with what @saschagrunert pointed to. From your description it seems like we'd need something that enforces a rule like: All PRs must have at least one label {skip-release-notes, release-note-bugfix kr release-note-feature}. It would be used at pull request time (and possibly at merge time).

and then something that takes all PR's since last tag/release,

Kubernetes and semantic-release support those use-cases ootb.

filters based on release note tags, and computes small blob of text with PR titles (potentially status check could enforce some more strict PR naming).

I'll look into filtering based on labels in the direction of writing a tool around release-notes as @saschagrunert suggested.

[saschagrunert]

From a use case / generating perspective: In the k8s release notes generator you can specify an empty notes block or something like NONE, and the PR gets skipped automatically (no label needed). The good thing is that a maintainer (or the author) could edit the PR if someone says “the note is not appropriate”. Then a re-generation of the notes results in the improved output. So from my point of view it’s easier than relying on the git history.

[j0n3lson]

Hey everyone. So I'm a little confused on how to get started/make progress here :(
I think the release-notes tool can already do what @AudriusButkevicius pointed out.

There seems to be two use-cases here:

1. Generating docs/CHANGELOG.md
2. Generating release notes (e.g. https://github.com/urfave/cli/releases/tag/v2.0.0)

I guess my main question is would the default output of release-notes (example) be sufficient for both cases? That is, would it be okay if both the CHANGELOG.md and the release notes page both look like the default output of release-notes.

For either case we'd need to do the following to even use the tool:

1. Adopt the same labels used by kubernetes (e.g. "kind/release-notes" or "kind/bug")
2. Enforce that a PR has a minimum of (a) a title, (b) a release notes block, and (c) a valid (kubernetes) label. Do this enforcement as a precheck before the PR is sent for review or at merge time (something like https://probot.github.io/apps/semantic-pull-requests/)

The CHANGELOG.md currently follows a simple Added, Removed, Changed format. If you all are really attached to this format then what @saschagrunert suggested (e.g. get notes using release-notes in JSON format and print to template) makes more sense to me.

I'm I thinking about this the right way?

[coilysiren]

@j0n3lson

I guess my main question is would the default output of release-notes (example) be sufficient for both cases? That is, would it be okay if both the CHANGELOG.md and the release notes page both look like the default output of release-notes.

The output from release-notes looks just like what I already write, and I already copy from the changelog to the release notes page, so yes 👍👍👍

Adopt the same labels used by kubernetes (e.g. "kind/release-notes" or "kind/bug")

👍 give me the list and I can do it now?

Enforce that a PR has a minimum of (a) a title, (b) a release notes block, and (c) a valid (kubernetes) label. Do this enforcement as a precheck before the PR is sent for review or at merge time (something like https://probot.github.io/apps/semantic-pull-requests/)

This is a distinct task - so I cut a new issue => #1007

The CHANGELOG.md currently follows a simple Added, Removed, Changed format. If you all are really attached to this format then what @saschagrunert suggested (e.g. get notes using release-notes in JSON format and print to template) makes more sense to me.

I would described myself as "fond of" rather than "attached" 🙂 Fully automating the releases is more important to me than keeping the Added, Removed, Changed format.

I'm I thinking about this the right way?

Yep 🚀

[saschagrunert]

What if we agree on a set of kinds for every PR, where multiple kinds per PR are possible as well, for example:

• kind/bug
• kind/cleanup
• kind/documentation
• kind/feature

Then we could create the notes via the tool, parse the JSON and print it into our own preferred output, like:

Welcome to urfave/cli version x.y.z

# Features
- Adding a shiny new feature which solves all our problems (PR #89317, @lynncyrin)

# Bugfixes
…

[coilysiren]

What if we agree on a set of kinds for every PR, where multiple kinds per PR are possible as well, for example

Love this idea ✨

[j0n3lson]

I ended up going with @saschagrunert earlier suggestion (support templates in release-tool). I have a PR#1008. With that PR we can specify whatever format we need in the template (and skip bits we don't need like the SIG stuff).

I experimented with wrappers around release-notes as suggested but cried a bit inside because of the duplicate code.

PTAL and let me know what you think!

@lynncyrin Any objection with using the PR template they use in kubernetes (see my PR)? The minimum need is to have either a "release-notes" or "Does this PR introduce a user-facing change?:" block in order to get notes. However, release-notes makes some assumptions about the PR body (I saw some regexps) and it might just be simpler to do as the Romans do here.

[coilysiren]

Any objection with using the PR template they use in kubernetes (see my PR)?

No objections, totally ship it!

[saschagrunert]

I experimented with wrappers around release-notes as suggested but cried a bit inside because of the duplicate code.

I think I know what you mean 😊

I'm currently working on a tool which uses the release notes parser in a programmatic way, the logic can be mainly found here: https://github.com/kubernetes/release/blob/137f2f6ae4b79c0a02f37517cb147d3bd76e6e9d/cmd/krel/cmd/changelog.go#L131-L199 (might be a source for inspiration)

Edit: Saw the PR in k/release, I think that is the best approach 👍

[j0n3lson]

I'm currently working on a tool which uses the release notes parser in a programmatic way, the logic can be mainly found here: https://github.com/kubernetes/release/blob/137f2f6ae4b79c0a02f37517cb147d3bd76e6e9d/cmd/krel/cmd/changelog.go#L131-L199 (might be a source for inspiration)

Yep. That's the one I saw before. I filed issue#1019 on k/release to discuss migrating to the same template-based approach approach as PR#1008.

[j0n3lson]

In terms of this issue, the only think left to do is:

1. Wait for the PR to land in k/release/cmd/release-tool to land
2. Add a urfave/cli specific template to the repo

Missing anything else?

[saschagrunert]

In terms of this issue, the only think left to do is:

1. Wait for the PR to land in k/release/cmd/release-tool to land
2. Add a urfave/cli specific template to the repo

Missing anything else?

We can think about adding a GitHub action to generate the notes, but I'm not sure if we want to do that.

[coilysiren]

We can think about adding a GitHub action to generate the notes, but I'm not sure if we want to do that.

This definitely sounds like something we'd want to do 👍

[j0n3lson]

Finally pushed it. Got busy with taxes and perf and what not. Hopefully can merge it soon :)

[j0n3lson]

Just a quick update.

1. I validated the templating works for both kubernetes use-case and also an arbitrary urfave specific template. I had to valid using my pending pr1148 since release-notes seems to be broken at head due to the template change.

2. I tested it by pointing at kubernetes/kuberntes but using the urfave template and it produced a good template

3. I tested by pointing at urfave/cli but got not output becase we are actually missing the release notes block. I had thought it was a header but we actually need to update the template so it has this in the PR body:

```release-notes

```

I should be able to give a demo once my PR get's merged in k8s/release...

[zekth]

Github has now automated release note generator: https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes

[meatballhat]

I'd like to call this done since we've been using the GitHub automatic release notes for many releases.