Validation errors in action and no draft created but action reports success

[jdforsythe]

I'm seeing odd behavior from configurations that have been working for many months across over a hundred repositories.

We have seen duplicate draft releases created, one by the "release-drafter" user and another by a "github-actions" user, both for the same version number. I have been deleting one of them and publishing the other, and then when we merge more PRs, there are duplicates again.

Then we saw a release deleted after it was published. There was a discrepancy in the version number when we looked at the releases. The latest release was v1.1.1 but we knew we had published a v1.1.2 (the tag exists and it was published to NPM, which can only happen when we publish a release). The newest draft release from release-drafter was showing as v1.1.2, so I simply updated the tag number and name of the release to reflect v1.1.3 and published it. The workflow ran again when the draft was published, which I don't understand - it's only configured to run on push to master and the 3 pull request types, opened, reopened, and synchronize - and it overwrote the published release with a new draft release.

This explains the issue with the missing release, but I don't understand why this action would be running when the draft gets updated from draft to published - the config shows it should not:

name: Release Drafter

on:
  push:
    branches:
      - master

  ## required for auto labeler
  pull_request:
    types: [opened, reopened, synchronize]

jobs:
  update_release_draft:
    runs-on: ubuntu-latest
    steps:
      - uses: release-drafter/release-drafter@v5
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Digging into what's happening, I triggered the action to run with a new PR and noted several errors in the logs:

Run release-drafter/release-drafter@v5
"pull_request_target.opened" is not a known webhook name (https://developer.github.com/v3/activity/events/types/)
"pull_request_target.reopened" is not a known webhook name (https://developer.github.com/v3/activity/events/types/)
"pull_request_target.synchronize" is not a known webhook name (https://developer.github.com/v3/activity/events/types/)
"pull_request_target.edited" is not a known webhook name (https://developer.github.com/v3/activity/events/types/)

***ORG_NAME***/***REPO***: Found 8 releases
{ name: 'event', id: '4660822018' }
***ORG_NAME***/***REPO***: No draft release found
{ name: 'event', id: '4660822018' }
***ORG_NAME***/***REPO***: Last release: v1.1.3
{ name: 'event', id: '4660822018' }
***ORG_NAME***/***REPO***: Fetching parent commits of refs/pull/45/merge since 2023-04-10T19:50:38Z
{ name: 'event', id: '4660822018' }
***ORG_NAME***/***REPO***: Creating new release
{ name: 'event', id: '46608220[18](https://github.com/***ORG_NAME***/***REPO***/actions/runs/4660822018/jobs/8249426167#step:2:19)' }

Error: Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}

...

data: {
      message: 'Validation Failed',
      errors: [
        {
          resource: 'Release',
          code: 'invalid',
          field: 'target_commitish'
        }
      ],
      documentation_url: 'https://docs.github.com/rest/releases/releases#create-a-release'
    }

...

request: {
    method: 'POST',
    url: 'https://api.github.com/repos/***ORG_NAME***/***REPO***/releases',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'probot/12.2.5 octokit-core.js/3.5.1 Node.js/16.16.0 (linux; x64)',
      authorization: 'token [REDACTED]',
      'content-type': 'application/json; charset=utf-8'
    },
    body: '{"target_commitish":"refs/pull/45/merge","name":"v1.1.4 🌈","tag_name":"v1.1.4","body":"## Changes\\n\\n* No changes\\n","draft":true,"prerelease":false}',
    request: {}

Even with these errors, the action reports that it is successful, and no draft release is created.

Then I looked back at older release drafter actions, and there are several errors shown in some of them, again with no error reported on the action:

***ORG_NAME***/***REPO***: Updating existing release
{ name: 'event', id: '4660682245' }
Error: Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
{
  name: 'HttpError',


...



Error: HttpError: Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
        at /home/runner/work/_actions/release-drafter/release-drafter/v5/dist/index.js:8462:21
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async Job.doExecute (/home/runner/work/_actions/release-drafter/release-drafter/v5/dist/index.js:30793:18)
{
  name: 'AggregateError',

[jetersen]

Delete your current draft, this most likely happened because you switched default branch.

[jdforsythe]

@jetersen We've always used master as the default branch and that hasn't changed. I'm the only person in the org with permission to make those changes and I wouldn't because we have lots of internal tooling built around master being the default branch.

There is currently no draft release in this repo to delete.

[jdforsythe]

@jetersen We are about to make a new release for this repo. The PR was created a few minutes ago and then merged later. Looks like the PR creation triggered a new draft release created by "github-actions" user and the merge triggered a separate draft release created by "release-drafter" user

[jdforsythe]

Now I merged a PR to bump the version numbers in the repo in preparation for release. When the action ran, only the release by user "github-actions" was updated - the one created by "release-drafter" doesn't have the new merged PR in it.

[jetersen]

You ought to only use GitHub Actions. The GitHub App is not recommended to use.