Fail command does not run if plugin fails in publish

[hanseltime]

Current behavior

I had an issue where my npm registry was not permitting my token to publish. When this failed, my fail phases were not running because an error was thrown that was not a "semanticRelease" error.

Expected behavior

I expected that the fail phases would have run with the error so that I could send a slack message (and even potentially remove tags as clean up , etc.)

semantic-release version

21.1.2

CI environment

Github Actions

Plugins used

@semantic-release/exec
@semantic-release/commit-analyzer
@semantic-release/release-notes-generator
@semantic-relesae/changelog
@semantic-release/npm

semantic-release configuration

module.exports = {
  branches: [
    'main',
    { name: 'develop', prerelease: true },
  ],
  plugins: [
    [
      "@semantic-release/commit-analyzer",
      {
        preset: 'angular',
        releaseRules: [
          { type: 'refactor', release: 'patch' },
          { type: 'perf', release: 'patch' },
        ],
      },
    ],
    '@semantic-release/release-notes-generator',
    '@semantic-relesae/changelog',
    [
      '@semantic-release/npm',
      {
        npmPublish: false,
      },
    ],
    [
      '@semantic-release/exec',
      {
        // Send packages to a standard file for CI to look at
        successCmd: `echo "${fullName} $\{nextRelease.version}" >> ../../current_releases.txt`,
        failCmd: "echo "We failed"",
      },
    ],
  ],
  ci: true,
}

CI logs

Given that this happened in a company repo, I don't want to copy-paste logs.

The gist is that there was a github error for github packages and then no Failcmd log.

[hanseltime]

I would love to know if this is a bug and if there is a reason that we're not allowing errors to trigger "fail" phases. If we can define the behavior expected, I am more than willing to contribute the fix.

From my end, I would expect all errors to get sent to the fail plugins. If that's not the case, I would also be okay with setting a config option like allErrorsTriggerFail command that would effectively feature flag the behavior.

[travi]

would appreciate any further investigation into the issue that the community is willing to invest. it would be valuable to target defining an integration test for this situation as a result of the investigation. that would give us an executable definition to compare to other failures to help us determine how intentional this was. i expect that it is more likely an oversight, but a test would help us confirm.

[hanseltime]

Adding here that the PR above demonstrates the failure with a test. #3180

I can pull out the second commit to if we want it to fail as a matter of demonstration