I agree that the current process is fragile, I'd love to see it improved

OK so, I opened up a WIP PR: #1820

Is there a better way? Also curious if there's any objections to making getNextVersion becoming async.

I'll still need to tighten up the code (make sure that we strictly match only our.prerelease.1, but not out.prelease.something.else.5, fix up the logging to only log 'The next release version is %s' after adjustments, add missing coverage), but just want to hear some feedback if this is mostly OK.

I'm willing to work out a PR for this idea, but socializing the changes first and gather some feedback. How about an optional build suffix formatter to append build information on to the release. I've 2 automatic use cases in mind, one for a CI build number, and the other for the git head commit sha.

Usage: --build-suffix-format "\${gitHead}" or --build-suffix-format "\${build}"

Example: 1.2.3-alpha.1 can become 1.2.3-alpha.1+1a2b3c4 with the git sha or 1.2.3-alpha.1+1234 with the build number.

You could also customize the suffix to use multiple build segments (rebuild of current commit maybe), or whatever other meta info a user wishes. Any rebases, would likely restart the build counter, but you don't have conflicts with previous tags as the sha and build numbers should each more or less guarantee uniqueness. For example: --build-suffix-format "sha-\${githead}.build-\${build}" to yield 1.2.3-alpha.1+sha-1a2b3c4.build-1234

Doing this as a build suffix still follows semver standards, and would be easy enough to use, and shouldn't inhibit other auto release features.

I first preferred complete control on customizing the build number, but that introduces a fair amount of complexity in determining the correct auto increment number and determining the previous version diff becomes much more difficult. Plus that puts more difficulty on the user to know about proper versioning formatting and can easily break their own version generation.

Example: 1.2.3-alpha.1 can become 1.2.3-alpha.1+1a2b3c4 with the git sha or 1.2.3-alpha.1+1234 with the build number.

Although aside from the fact that the semver standard only prescribes, that build metadata does not figure into precedence, and does not explicitly state that two builds of the same version must contain the same features, I still think this approach might be a violation of that?

I think in most practices, divergent prereleases couldn't happen since semantic-release auto appends the branch name to the prerelease tag. So while 1.2.3-alpha... and 1.2.3-beta..., and 1.2.3-feature... can all exist at the same time, they can't really be compared to one another for what set of commits/features should be contained in them. Logically, alpha should come before beta, but those are just human labels attached to prereleases, and not actually automatic upgrade paths (yarn up from an alpha branch wouldn't suggest beta as an upgrade).

Only situation I can think of where unexpected feature sets would exist would be if contributors force push entirely different commits to the same remote branch, which I think is indicative of a larger workflow issue and not a responsibility of this library. Otherwise I think this change should enable workflows that use linear history over merging the upstream branch into the source branch when updating a branch.

Is there another situation that I'm just not seeing where this complicates things? It is an opt-in feature, so it could always be deemed a "use at your own risk" or "advanced usage (use only if you know what you are doing)" kind of feature.

Thing is the numbering restarts after each rebase. So you could release an alpha.1+beef, alpha.2+f00d, alpha.3+dead and then rebase, resulting in an alpha.1+1337.

I'm not against your proposal, and yes - this is advanced usage. However making releases like this does mean that all other tooling that relies on semver (semver.compare() itself) would produce incorrect results.

Instead of that, do you think #1894 would address your concerns?

seems to match our usecase where we have 3 branches:

• main (1.21.4)
• rc (1.21.4-rc.1)
• beta (1.21.4-beta.1)

we create a beta release (1.21.5-beta.1), then rebase on RC, release a RC version (1.21.5-rc.1).

In RC we flagged a bug then fixed it in beta branch. RC release failed to find the correct version:

Found git tag 1.21.4 associated with version 1.21.4 on branch beta
The next release version is 1.21.5-beta.1
fatal: tag '1.21.5-beta.1' already exists

{
  "branches": [
        "main",
        {
            "name": "rc",
            "prerelease": "true"
        },
        {
            "name": "beta",
            "prerelease": "true"
        }
    ]
}

Is there any preview/beta version which we can test?

Rebasing is basically destroying the semantic release and rebase is one of the core features of git used by many devs.

Hi, as can be seen above I've been digging into this issue as well. Adding build info to the version seemed like a good fix for prerelease version collisions, but after testing I found two drawbacks:

1. As mentioned before, the version number will reset after a force push. Potentially causing issues when comparing versions (or when selecting the highest version, which will remain the latest version before the force push).
2. And (arguably more important) some package services (like NuGet) don't really support build info in versions. Defeating the whole point of adding the build info (for this issue) in the first place.

To fix this, I opened a PR to override the prerelease version number itself. This way, the pipeline id can be used instead of the generated one. This ensures an increasing id and it works with NuGet.

i think the conversation is starting to diverge here from the original request. the way i understand the request is to highlight that it is currently difficult to recover from accidental situations where a pre-release branch has it's history rewritten. we agree that the recovery process is currently painful, and we would like to find a way to improve that.

however, we have not said that we are wanting to enable a workflow that encourages the rewriting history of any release branch as a part of a supported workflow. releases are intended to keep an association of the published version and the source commit that resulted in that published version. rewriting history of a release branch breaks that association, so it is not a practice we encourage outside of accidents. the linked thread about using pre-releases for PRs also highlights that semantic-release is not designed to handle such situations with pre-releases. managing disassociated versions that do not share history is not one of our goals and attempting to do so goes down complicated paths involving republishing versions.

we would like to improve the recovery process, but in a way that makes it clear that it is not encouraged to rewrite history of a release branch and not in a way that associates releases across branches that do not share history. we understand the possibility of appending a suffix could solve some of these workflows. however, it is important to clarify that such a feature would be fairly independent of the current pre-release workflow that we support, if we were to accept supporting that functionality, and we would not consider it a way to encourage workflows that intentionally include re-writing a release branch's history.

i think the conversation is starting to diverge here from the original request. the way i understand the request is to highlight that it is currently difficult to recover from accidental situations where a pre-release branch has it's history rewritten. we agree that the recovery process is currently painful, and we would like to find a way to improve that.

Say I have a beta branch used for prereleases. When adding new fixes/features to main i would like to incorporate these commits into the beta. One way of doing so is using a rebase, which would require (not accidental) recovery.

however, we have not said that we are wanting to enable a workflow that encourages the rewriting history of any release branch as a part of a supported workflow. releases are intended to keep an association of the published version and the source commit that resulted in that published version. rewriting history of a release branch breaks that association, so it is not a practice we encourage outside of accidents. the linked thread about using pre-releases for PRs also highlights that semantic-release is not designed to handle such situations with pre-releases. managing disassociated versions that do not share history is not one of our goals and attempting to do so goes down complicated paths involving republishing versions.

You mention that rewriting history breaks association. What do you mean here? As far as i'm concerned, existing version tags stay exactly where they where. Only the new versions will be added on the rebased branch.

we would like to improve the recovery process, but in a way that makes it clear that it is not encouraged to rewrite history of a release branch and not in a way that associates releases across branches that do not share history. we understand the possibility of appending a suffix could solve some of these workflows. however, it is important to clarify that such a feature would be fairly independent of the current pre-release workflow that we support, if we were to accept supporting that functionality, and we would not consider it a way to encourage workflows that intentionally include re-writing a release branch's history.

Well, fair enough. I currently use the version from my PR to support rebasing beta and pr branches (from which prereleases are created) which solved my problems perfectly. I view rebasing as an integral part of prerelease branches, as they (or anything using a force push) are the only way to keep the branch up to date without introducing additional commits. But if that's not something this library wants to support in this way, so be it.

Say I have a beta branch used for prereleases. When adding new fixes/features to main i would like to incorporate these commits into the beta. One way of doing so is using a rebase, which would require (not accidental) recovery.

the recommended way to accomplish this without rewriting history is to simply use a normal merge

You mention that rewriting history breaks association. What do you mean here? As far as i'm concerned, existing version tags stay exactly where they where. Only the new versions will be added on the rebased branch.

existing version tags do not stay exactly where they were. that is the whole point. when you rewrite the history of a branch, the sha for each commit changes. it changes because the commits before it changed, so it no longer has the same history. sure, you can put a replacement tag on the newly resulting commit, but that is no longer the commit that resulted in the version that was published. the reason to associate source commits with published versions is to be able to clearly determine the changes that are included in that version. when you rebase another branch into your release branch, the history of the branch now includes commits that were not present when the version was published. merging without rebasing brings those changes into the release branch so they can be included in the next released version without changing the history of the commits that resulted in an already released version.

merging without rebasing accomplishes the that you describe. simply dont rebase when merging other branches into your release branches

existing version tags do not stay exactly where they were.

They do, they stay exactly where they were. On the old commit, on the old version of the branch. Unless you manually move them. (Unless i'm unaware of some automatic conversion.) This might result in a bit more clutter in the git history, since commits might show up more than once. But versions are still link to their original commit.

They do, they stay exactly where they were. On the old commit, on the old version of the branch. Unless you manually move them.

if you rewrite the history of a commit (in any branch), the sha of that commit changes. if the sha changes, the tag goes away because it was tied to the previous sha. this has nothing to do with semantic-release, but is how git works.

it sounds like you must be describing a different workflow where the commits that have releases tagged must not be getting rewritten, so i'm not following what you mean by rebasing the beta branch if you are saying the tag is sticking around. you must be describing something beyond the purpose of this particular thread, so please open a separate issue if you want to continue discussing that path. we've already distracted this thread enough

if you rewrite the history of a commit (in any branch), the sha of that commit changes. if the sha changes, the tag goes away because it was tied to the previous sha. this has nothing to do with semantic-release, but is how git works.

if you rewrite the history of a commit (in any branch), a new commit is created with a new sha. For example:

If we create the problem scenario as described before, I have a beta branch that's behind on master:

gitGraph
   commit id: "Initial Commit" tag: "v1.0.0"
   branch beta
   checkout beta
   commit id: "New Feature" tag: "v1.1.0-beta"
   checkout main
   commit id: "Bug Fix" tag: "v1.0.1"

If I rebase beta on master, this is the result:

gitGraph
   commit id: "Initial Commit" tag: "v1.0.0"
   branch "beta (old)"
   checkout "beta (old)"
   commit id: "New Feature (old)" tag: "v1.1.0-beta"
   checkout main
   commit id: "Bug Fix" tag: "v1.0.1"
   branch beta
   checkout beta
   commit id: "New Feature"

Where the "New Feature (old)" is just a floating tag without a branch (and with the original name). Which, notably, refers to the same exact commit as it did before rebasing. This is how git works.

it sounds like you must be describing a different workflow where the commits that have releases tagged must not be getting rewritten, so i'm not following what you mean by rebasing the beta branch if you are saying the tag is sticking around. you must be describing something beyond the purpose of this particular thread, so please open a separate issue if you want to continue discussing that path. we've already distracted this thread enough

I'm not

semantic-release should check the list of existing tags for the specific pre-release across all commits (not just the ones in the current branch) to determine the next increment.

My PR aims to solve exactly this, having multiple prereleases outside (floating) the current (force pushed) branch, with working versioning.

if you rewrite the history of a commit (in any branch), the sha of that commit changes. if the sha changes, the tag goes away because it was tied to the previous sha. this has nothing to do with semantic-release, but is how git works.

if you rewrite the history of a commit (in any branch), a new commit is created with a new sha.

this is a valid distinction. my statement was based on the context of being relative to the beta pre-release branch, so my statement was imprecise. admittedly, that was partly on purpose because many folks refer to the changeset that results in a commit as "commit" in that relative perspective. after rewriting the history of a branch, the same changeset exists in the rewritten branch, but as a different commit because the changeset is applied to a different history than before. lets avoid debating correctness as long as we can agree that the original tagged commit that resulted in the published version still exists and the tag still points to it, but that commit is no longer in the history of the pre-release branch.

This is how git works.

so, if we've established that we agree on how git works, lets highlight how this applies to semantic-release and the purpose of me calling out the details that i have, even if i initially included some imprecision due to focusing on the perspective of the pre-release branch. the purpose of semantic-release is to simplify adherence to semantic versioning. the semantic part of that is to communicate to consumers what has changed since the previous release and what level of impact those changes are expected to have. by removing the commit that resulted in the previous release from the history of the pre-release branch, semantic-release can no longer make any semantic assertions about the changes that are included in the pending release because it has no way of knowing how it relates to the previous version. as we established, the commit that resulted in the previous published version was a different commit and that commit does not exist in the history of the commit triggering a new pending release, even though the same changesets exist (but in a different order).

the semantic-release project is not interested in working around this problem because it breaks that core promise of the project.

merging without rebasing accomplishes the that you describe. simply dont rebase when merging other branches into your release branches

this is still the correct answer for this situation. treat a pre-release branch as a release branch that lives the entire lifespan of the pre-release until it is promoted to stable (merged into your default branch). any changes that should be appended into that release line need to be merged/pushed to that release branch without rewriting its history. the only potential goal that this does not accomplish is the trend of desiring linear history over all else. a release branches is a situation where maintaining accurate history over a linear history is valuable.

it is currently difficult to recover from accidental situations where a pre-release branch has it's history rewritten. we agree that the recovery process is currently painful, and we would like to find a way to improve that.

this conversation has made me rethink/think more clearly about a potential approach to improving this. the current recommendation is to manually adjust the tag and git notes in the branch that results from the rebase. i admit that my initial thought for making that easier was to provide a way to "fix" that more automatically. that current recommendation was because recovering the original history of the branch is difficult and involves interacting with reflog, which is more advanced that many of our users are prepared for.

however, avoiding recovering the original history from before the accidental history rewrite still leaves folks in the undesirable situation of disconnected histories. while it may be more of a challenge for us to provide a more automated way of fixing this situation, that is the more correct approach and should probably be our goal for resolving this.

@travi fair points. One last remark

the only potential goal that this does not accomplish is the trend of desiring linear history over all else

For me it's not about the history being linear, it's about the history being cleaned up. During the development of the prerelease we might make mistakes, change design decisions, or simply make changes only intended for the prerelease itself (like adding dependencies on other prerelease packages). Before we release this change, and generate a changelog referencing these commits, we want to clean them up. So that non-prerelease package consumers only have to think about the actual relevant changes, linking to commits without buggy code.

i really appreciate you sharing that thought. i think it is totally valid, especially in the context of pre-releases, where using them as intended is to work though uncertain changes and changing direction is likely, at least before promoting to stable. semantic-release does not handle that all too well. this is a situation that we are aware of, but have mostly written off as beyond the scope of the automation that is possible because it essentially requires a human to navigate that complexity.

our recommendation, when questions come up related to this, is to manually adjust the release notes to account for the context that semantic-release cannot understand. i can understand how it could also lead to teams wanting to make their own adjustments by modifying the history of the pre-release so that it is clear when promoted to stable. i can admit that even our documentation likely does not give a very clear recommendation for how to handle this situation.

i'm completely willing to discuss this further in another thread. i'm not sure it can result in much more than a better recommendation in our docs, but i think there is value to that, at least. would you be willing to open a thread on this topic? please feel free to share thoughts if you think there are options beyond docs that align to the goals from the rest of this thread.