[question] What is the recommanded way for branch versions ?

[sagi-ottopia]

What is your question?

My question is regarding Conan 2.0 and branch versioning.
With conan 1.X, we used the channel to distinguish between branches for our versions in our remote.
I read conan documentation and I understand that it is not recommended to use channel anymore.

We implemented set_version function in our conanfile which uses git describe for setting the version. We also use tags in our repositories for labeling the released versions.
for example: we might have a tag 0.7.1 for that version. (major.minor.patch) and we use the number of commits from git describe command and the build.
so on a development branch we might have the version 0.7.1.45 (45 commits since the tag).
When developers implement something, they branch out to a side branch, do their work, do several commits, for example 10 commit. On their branch the package version is now 0.7.1.55
but after the squash and merge to the develop branch the version will be 0.7.1.46
Which makes it hard for other developer to understand what is the latest version that they want to use.

I though to add the branch name to the build number, for example: 0.7.1.46-the-most-scary-bug-ever

What do you do at your companies ? how do you get around versioning branches and resolve the version conflicts between the branches ?

Thanks,
Sagi.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @sagi-ottopia

Thanks for your question, it is good one.
Indeed using channel is no longer recommended. The main challenge is that you need to modify the requires = downstream to include that specific channel. So it "viralizes" down the graph, and it can be difficult to manage. The idea is to avoid this complexity.

When developers implement something, they branch out to a side branch, do their work, do several commits, for example 10 commit. On their branch the package version is now 0.7.1.55
but after the squash and merge to the develop branch the version will be 0.7.1.46

I think the idea is that with revisions it is not needed to do so. Every commit doesn't need to generate a new version, it can generate a new revision of the 0.7.1.46. Then, when that is merged to the main branch, it maintains an incremental and sequential order without issues.

I though to add the branch name to the build number, for example: 0.7.1.46-the-most-scary-bug-ever

For this purpose you might be able to use build version metadata, that is 0.7.1.46+build.bug_fix_ugly.1 Which works with version ranges and also implements version ranges inside the +build metadata, so +build.bug_fix_ugly.2 will be resolved as "latest" if requires = "mypkg/[>0.6 <1] for example.

[sagi-ottopia]

Hi @memsharded
Thank you for your answer.
I will try to use build version metadata, due to the advantage, that you can see very clearly from the version name if a package was built from a side branch.
Can you point me to a documentation explaining how to use this feature ?
Is it the same like the pre-release builds ?

[memsharded]

You can find some info in https://docs.conan.io/2/tutorial/versioning/version_ranges.html#semantic-versioning

Pre-releases and build-metadata are not the same, very different. To use pre-releases they should be explicitly opted-in, in the version-range itself, or in general better via a conf core.version_ranges:resolve_prereleases. But build-metadata can be always used and will be valid if in the valid range.

Sounds good, give it a try and let us know. Maybe we can close this issue, and you can create a new ticket if you have new questions regarding this? Thanks!

[sagi-ottopia]

thanks.
Just to be sure, all I have to do is to modify my set_version function to add +build string to my version.

[memsharded]

I'd recommend carefully reading the discussions in conan-io/conan#15638, I think they are very related.

Just to be sure, all I have to do is to modify my set_version function to add +build string to my version.

Yes, and depending how you want to pass the build metadata, you can pass just the build metadata in command line --version and the recipe version defines its versions then append the +<buildmetadata, or it can append the build metadata automatically based on some env-var, etc, depending on how you want to pass the information.

[opajonk]

Hi! We have the (to my understanding) same issue with Conan, and I did not fully understand the proposed solution.

To clarify, let me give a scenario:

• we are currently at 1.0.0-beta on a main branch of component A.
• we develop multiple dependent components (say B depends on A)
• two (or more) branches are branched off of main on A and B, on which different developers / teams work in parallel
• during development we now need a surefire way to reference A from B, on the correct branch

It would be fatal if during building of component B, Conan would pick a snapshot of A that belongs to "another branch". For this use-case, what is the recommended approach? As far as I understand, revisions would not work ("latest" may come from any branch) and build metadata also not (it is not considered for versioning).

[memsharded]

Sure, @opajonk, that is a related question, but slightly different.

When you create different versions or revisions of a given package (A), then you want to test those changes in isolation "downstream", to see if something breaks. The approach is more or less:

• Define your main "products" that you want to ensure they keep working and integrate your changes. They can be different versions of the same app, and different apps, like app1/1.0, app2/2.3
• When you do the branch of package A, you are creating a new version or revision, depending on the approach
• Capture the lockfile for that version or revision, upload that version or revision to a "build" remote repo (not the main "develop" one)
• Depending on the scale and distribution, you want to:
  • For parallelized large scale, use the lockfile to capture the conan graph build-order for each of the products app1, app2 and for every configuration, capturing the lockfiles too. Then do graph build-order-merge and iterate the result, distributing the builds in your build agents, making sure you are passing always the lockfile.
  • For building in the same machine, just conan install --build=missing every app product, passing the lockfile to the specific A component. You might want to do some quick capture of lockfiles too for the products with conan lock create to avoid changes while building
• For every thing built in the process, upload it to the "build" server repo.
• Only when everything is green and working, you run a promotion from the "build" server repo to the "develop" one. This is typically done after merging the source code changes of package A

This is what we will be writing in the docs, hopefully soon, for different cases and flows.

[opajonk]

Thanks for the quick reply! I cannot say I fully understood the answer, but that is very likely due to my lack of knowledge about Conan (2). However, we will try this out!

[memsharded]

The core idea is that using lockfiles is the way to make sure that specific dependency is being used and not other. And indeed Conan 2 allows using the lockfiles for this goal orders of magnitude more easily than 1.X

[opajonk]

The core idea is that using lockfiles is the way to make sure that specific dependency is being used and not other. And indeed Conan 2 allows using the lockfiles for this goal orders of magnitude more easily than 1.X

Ah ok, got it... maybe a last question then: what if I do not need a very specific dependency (forcing the handling and passing around of lockfiles) but rather a "latest from a certain branch" dependency? I mean, at least during our development cycles, this is the by far most common use-case: modify a component/module/whatever A, and then check: does it still integrate downstream? Now maybe component B breaks for whatever reason. Consequence: fix A or modify B (as part of the same unit of work, i.e. the same branch on a different repository).

That means: use "latest A from the branch on the same branch on B". This semantic actually does not change, so I would like to avoid creating and passing around of lockfiles... it seems like overhead.

[memsharded]

That means: use "latest A from the branch on the same branch on B". This semantic actually does not change, so I would like to avoid creating and passing around of lockfiles... it seems like overhead.

In practice it does change. Any other possible change that is concurrent, and then your CI builds are using one revision in the first part of the builds and a different revision at other part of the build (or like the Windows build uses one revision, and the Linux build uses another revision). This also happens for dependencies. Imagine you are doing a change in "A". But it happens that A has a dependency on "D", and "D" has also got an update by other team. It is very possible that part of the builds are done with one revision or version of D and another part of the build are done with another version or revision of D.

And this is both very complicated to understand and debug, as the failures can get unnoticed until later at runtime.
The only way to protect against it is using lockfiles. It does not only fixes the A specific version and revision, but also the dependencies of A that are being used for that build.

Furthermore, when building integrating the products "apps", it is also possible that other packages are concurrently changed, packages that depend on A and are used by the "apps" packages. Having a lockfile for the "apps" before doing the builds, specially if they are distributed, different platforms, etc, becomes very important.

One of the major lockfiles improvements in 2.0 is that only 1 lockfile is necessary, and it can be used to lock all different binary configurations, highly simplifying working with them.

[opajonk]

Thanks a lot for the detailed explanation!

[schwaerz]

@memsharded Just found this discussion - and it already helps. maybe my question fits here, too:

According to the Conan documentation - if I understand it correctly - pre-release versions take higher precedence compared to the release versions if core.version_ranges:resolve_prereleases=True is set, so, as seen in the table:

1.0.0-alpha > 1.0.0

In semver 2.0.0 however, they define it like:

1.0.0-alpha < 1.0.0

Is there some misunderstanding on my end, or does Conan not fully follow semver here?

[schwaerz]

Finally I had some time to try this myself. Conan seems to behave in very useful way here :)

However I feel the documentation could be little bit more precise.

Details:

If I have the following packages

• foo/1.2.3-alpha
• foo/1.2.3-beta
• foo/1.2.3
• foo/1.2.4-alpha

a requirement like foo/[<1.2.4, include_prerelease] will resolve to foo/1.2.3
a requirement like foo/[<2, include_prerelease] will resolve to foo/1.2.4-alpha, until 1.2.4 was released.

[memsharded]

Thanks for the feedback @schwaerz, happy to hear that the current behavior is useful :)

I'd probably even recommend to drop the include_prerelease syntax in the requires and use the conf core.version_ranges:resolve_prereleases that activates and deactivates resolution to pre-releases.

I think we can close this question as responded?

[schwaerz]

Yes, thanks.
One last remark: To actually be able to select a pre-release version of a specific version (i.e. 1.2.3), one has to do (assuming that core.version_ranges:resolve_prereleases):

foo/[>=1.2.3 <=1.2.3]

Neither of the following options would choose that pre-release:

foo/[>1.2.3 <1.2.4]

foo/[<1.2.3 >1.2.2]

I think the documetation is little bit misleading in this case.

One more thing: Would be good to be able to specify the configuration option to whether include pre-releases on the command line when creating a package.

Usually I'd not like to specify this globally, as this would always affect all repositories. On the other hand I'd also not like to include it in the conanfile.py. Is there anything like that already planned?

[memsharded]

foo/[>1.2.3 <1.2.4]

Do you mean that this includes 1.2.3-pre.1? This shouldn't be the case. I have just tested it and seems to correctly excluded it.

One more thing: Would be good to be able to specify the configuration option to whether include pre-releases on the command line when creating a package.

We recently added the --core-conf/-cc command line argument that allows to pass configurations from global.conf in command line, without modifying the file and making it global. Is this what you are looking for?

[schwaerz]

@memsharded

Do you mean that this includes 1.2.3-pre.1? This shouldn't be the case. I have just tested it and seems to correctly excluded it.
Sorry for that. I accidentally added a not in my previous comment. Yes, the pre-release will be excluded for both of the following:

foo/[>1.2.3 <1.2.4]

foo/[<1.2.3 >1.2.2]

We recently added the --core-conf/-cc command line argument that allows to pass configurations from global.conf in command line, without modifying the file and making it global. Is this what you are looking for?

yes, this is what I was looking for.

[memsharded]

foo/[>1.2.3 <1.2.4]
foo/[<1.2.3 >1.2.2]

yes, it is totally expected that these do not include the pre-release:

• The first one is newer than 1.2.3, and 1.2.3-pre is even older than 1.2.3, so it is not included
• The second one, being strictly lower than 1.2.3 also is expected to exclude 1.2.3-pre. Because the pre release it is older than 1.2.3, but also it belongs to the 1.2.3 "train", so it is not strictly "less than" 1.2.3, but closer to an identity relation. <=1.2.3 would accept the pre-release 1.2.3-pre.1.

I think this might be clarified in the docs, transferring the ticket to the docs repo, if you have nay other question, please open a new ticket, thanks!