Exit with error when dry run does not trigger new release

[tszawara]

New feature motivation

I'd like for my github action to fail, if PR is raised and new commits do not adhere to required format that could trigger a new release.

Aside of this, it would be an advantage to be able to define number of compliant commits expected, so it would fail if you require 1 compatible commit, and you encounter 0,2,3 and so on.

New feature description

Aside of this, it would be an advantage to be able to define number of compliant commits expected, so it would fail if you require 1 compatible commit, and you encounter 0,2,3 and so on.

New feature implementation

Currently I have to resort to this:

semantic_release_dry_run:
  needs: prepare
  runs-on: ubuntu-latest
  steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Create dry branch
      uses: peterjgrainger/action-create-branch@v2.2.0
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      with:
        branch: 'dry-run'
    - name: Checkout dry branch
      uses: actions/checkout@v2
      with:
        ref: dry-run
    - name: Get node modules cache
      uses: ./.github/actions/prep-node-env
    - name: Run semantic-release dry run
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: |
        # Trick semantic-release into thinking we're not in a CI environment
        # note: semantic-release requires that dry-run branch is defined as releasable in release config (setting --branches $GITHUB_HEAD_REF does not seem to take precedence of this requirement, hence the need to create a specific dry-run branch)
        unset GITHUB_ACTIONS
        unset GITHUB_EVENT_NAME
        echo "Pull request branch is: $GITHUB_HEAD_REF"
        npx semantic-release --branches dry-run --dry-run --no-ci

Exit code of above job is always a success (regardless if commit label was correct or not).

It would be sufficient if there was a commit analyzer somwhere, which without specifying any extra configs, could with 100% compatibility with semantic-release expectation evaluate a commit label. Please let me know if a tool such as this exists.

[sheerlox]

The tool you're looking for is commitlint.

You'll also find a guide to setup checking PR commits in GitHub Actions here: https://commitlint.js.org/#/guides-ci-setup?id=github-actions

Your second request about specifying a specific number of compliant commits seems odd, but you can probably implement such a thing using commitlint CLI's output or its JS API.

[tszawara]

f this, it would be an advantage to be able to define number of compliant commits expected, so it would fail if you require 1 compatible commit, and you encounter 0,2,3 and so on.

@sheerlox Thanks, do you think @commitlint/config-conventional is the exact match for what semantic-release commit '@semantic-release/commit-analyzer' does while checking commit label for correctness (and will those two always be in sync)?

The usefulness for a specific number of commits serves a common use case, where, if you're building an application, you might require, that there would be only one commit/ per PR/ per release, before a PR merge. Therefore it would be handy if while checking commit label for compliance, you'd also allow only one. I don't exactly remember how semantic-release behaves, when for example it encounters two 'feat' commits, but I guess it would still make 1 release, but with two features mentioned in the changelog.

Coming back to commitlint, it might be good enough, but the dry-run, checks a few things more then just commit labels, and it would be an advantage to be able to use it in a CI job.

[sheerlox]

do you think @commitlint/config-conventional is the exact match for what semantic-release commit '@semantic-release/commit-analyzer' does while checking commit label for correctness (and will those two always be in sync)?

Both parsers are based on the same standard convention (Conventional Commits), so as long as the same preset is used, they should always be in sync.

However semantic-release uses conventional-changelog-angular as its default parser preset, which for a commitlint config corresponds to @commitlint/config-angular.

if you're building an application, you might require, that there would be only one commit/ per PR/ per release, before a PR merge. Therefore it would be handy if while checking commit label for compliance, you'd also allow only one.

That's definitely out-of-scope for semantic-release, but I'm sure that's very straightforward to implement yourself with a simple GitHub Action running on PRs and being listed in the required checks, and someone probably implemented a generic solution to this problem already.

I don't exactly remember how semantic-release behaves, when for example it encounters two 'feat' commits, but I guess it would still make 1 release, but with two features mentioned in the changelog.

Correct.