Conan version ranges not working with loosely names packages.

[Mark-Hatch-Bose]

To help us debug your issue please explain:

• I've read the CONTRIBUTING guide.
• I've specified the Conan version, operating system version and any tool that can be relevant.
• I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

Conan Version 1.12.2
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial

This is a workaround for: #4600, see latest comments

Conan version ranges don't work as expected with include_prerelease=True
I'm expecting it to be able to find my latest package, but it does not.

I have the following packages on artifactory:
CastleConanCommon/0.0.3@master/latest
CastleConanCommon/0.0.4-3-gff22b81@master/latest
The 0.0.4 is now using git describe to uniquely version every commit.

a test conanfile with

class testConanfile(ConanFile):
    name = "testConanfile"
    version = "1.0.0"
    generators = "virtualenv"
    build_requires = "CastleConanCommon/[>=0.0.4, include_prerelease=True]@master/latest"

This results in the conan install . -u to fail

# conan install . -u 
Configuration:
[settings]
arch=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=5
os=Linux
[options]
[build_requires]
[env]

ERROR: Version range '>=0.0.4, include_prerelease=True' from requirement 'CastleConanCommon/[>=0.0.4, include_prerelease=True]@master/latest' required by 'virtual' could not be resolved

However, if I change the build_requires to
build_requires = "CastleConanCommon/[>=0.0.3, include_prerelease=True]@master/latest"
it works...

I'd expect that semver would recognized the first part 0.0.4-3-gff22b81 as 0.0.4 and be able to find the package.

[Mark-Hatch-Bose]

Update:

I resolved this by reworking the git tag to fit SEMVER loose versioning requirements.
CastleConanCommon/0.0.4-3-gff22b81@master/latest
becomes
CastleConanCommon/0.0.4.3+gff22b81@master/latest

This works for me, and I no longer need include_prerelease=True which is a bonus!

[Mark-Hatch-Bose]

Another update:
Version ranges don't seem to be working with the renaming either, so I'm reopening this.

ERROR: Version range '>2.4.9' from requirement 'PackageName/[>2.4.9]@master/latest' required by 'UpstreamPackage/2.4.4.4+gdcb5124@name/testing' could not be resolved
ERROR: Version range '>=2.4.10' from requirement 'PackageName/[>=2.4.10]@master/latest' required by 'UpstreamPackage/2.4.4.4+gdcb5124@name/testing' could not be resolved

$ conan search PackageName -r <remote> | grep master
PackageName/2.4.10.6+g39f4971@master/latest
PackageName/2.4.10.7+gb043988@master/latest

[jgsogo]

Hi, @Mark-Hatch-Bose.

We are using SemVer versioning (package node-semver==0.6.1) in the implementation to compute this version ranges and everything related to it. Have a look at the following behaviour:

>>> from semver import max_satisfying, SemVer
>>> a = SemVer("2.4.10.6", loose=True)
>>> a.version
'2.4.1-0.6'
>>> vars(a)
{'loose': True, 'raw': '2.4.10.6', 'micro_versions': [], 'build': [], 'major': 2, 'minor': 4, 'patch': 1, 'prerelease': [0, 6], 'version': '2.4.1-0.6'}

As you can see, it has parsed 2.4.10.6 as 2.4.1-0.6 🙀 , so any of your expressions (>2.4.9 or >=2.4.10) will match it. I'm reading the semver documentation (https://semver.org/) and AFAIK the pre-release version should be preceded by a dash, so I don't know why it is parsing it this way (ok, it is loose=True, but...).

I'm sure it is the root of your problem, four numbers is not a valid semver, but I cannot tell you if this behaviour is a bug or a feature in the node-semver package, maybe it is worth asking this question there... or use valid semver string for your packages 2.4.10-6+fdsa.

[Mark-Hatch-Bose]

Thanks for showing me this! Looks like I'll need to create an issue with SemVer to fully understand.

Even formatting it as you mentioned, max_satisfying is not recognizing that it can grab either...

>>> a = SemVer("2.10.3-3+gdd213db", loose=True)
>>> b = SemVer("2.10.3-4+gdfz23bd", loose=True)
>>> 
>>> 
>>> vars(a)
{'minor': 10, 'build': ['gdd213db'], 'raw': '2.10.3-3+gdd213db', 'micro_versions': [], 'prerelease': [3], 'patch': 3, 'major': 2, 'loose': True, 'version': '2.10.3-3'}
>>> vars(b)
{'minor': 10, 'build': ['gdfz23bd'], 'raw': '2.10.3-4+gdfz23bd', 'micro_versions': [], 'prerelease': [4], 'patch': 3, 'major': 2, 'loose': True, 'version': '2.10.3-4'}
>>> 
>>> 
>>> 
>>> print(max_satisfying(versions, range_, loose=True))
None
>>> range_ = '>=2.10.3'
>>> versions = ['2.10.3-3+gdd213db', '2.10.3-4+gdfz23bd']
>>> 
>>> 
>>> max_satisfying(versions, range_, loose=True)
>>> 
>>> 
>>> print(max_satisfying(versions, range_, loose=True))
None

In testing though, I found that this works

>>> range_ = '>2.10.3-0'
>>> 
>>> print(max_satisfying(versions, range_, loose=True))
2.10.3-4+gdfz23bd

This does not work

>>> range_ = '>2.10.3-0'
>>> versions = ['2.10.3-3-gdd213db', '2.10.3-4-gdfz23bd', '2.10.3']
>>> print(max_satisfying(versions, range_, loose=True))
2.10.3

[Mark-Hatch-Bose]

I'm closing as I think I fully understand the behavior now, thanks @jgsogo for the help!

[jgsogo]

You are welcome! During the development of the version ranges, the loose and the pre-release arguments I found that SemVer matching is almost as hard as regex, you never know what is going to happen until you actually test it 😆

[Mark-Hatch-Bose]

Just in case anyone looks at this in the future:

To use git tag versioning:
package version should look like this:
2.10.3-4+gdfz23bd

Version ranges should look like [>=2.0.0-0, include_prerelease=True]

Everything seems to be working now.

[jgsogo]

Thanks for the recap, maybe it makes sense to add this example to the version ranges docs or to the package version from SCM ones. I will open an issue there.

[Mark-Hatch-Bose]

@jgsogo here is the code I'm using to restructure the git tag to fit the node-semver if it's useful for the documentation.
We are using tags with vX.Y.Z pattern on our master branches.
As mentioned, this was a workaround until the upcoming package revision feature is out (to enforce dependency tree rebuilds on every commit with full_package_mode). See #4600 for details.

def get_version_git_tag():
    try:
        git = tools.Git()
        gitTag = git.run("describe --tags --match 'v*.*.*' --long")
        gitTag = gitTag[1:]
        #we are renaming the package to fit prerelease versioning requirements
        gitTagRe = '([0-9]+\.[0-9]+\.[0-9]+)\-([0-9]+)\-([a-zA-Z0-9]+)'
        r = re.search(gitTagRe, gitTag)
        version = "%s-%s+%s" % (r.group(1), r.group(2), r.group(3))
        return version
    except:
        return None

[Mark-Hatch-Bose]

npm/node-semver#164

The way we are using prereleases is not working as intended with certain version ranges like compatible versioning. ^2 or ~=2.10 for example.
(This was due to my misunderstanding about prerelease versions.)

Loose versioning + micro versions seems to work, but there seems to be a issue in node-semver when the patch number is greater than a single digit.

[jgsogo]

I've read npm/node-semver#164 and npm/node-semver#167, but it looks like we cannot do much on our side.

If what you say about ^2 or ~=2.10 is an issue in the Conan side, can you please provide an example?

Thanks!

[Mark-Hatch-Bose]

Doesn't seem to be a Conan issue. This is all a workaround till package revisions is a stable feature.
How is package revisions coming? Maybe I should consider switching to using that rather than trying to monotonically increasing my version on every commit.

[jgsogo]

Hi! You should give a try to revisions, sure. It is integrated into Artifactory so, even if it is an experimental feature and it can be modified in the future, I'm pretty sure we0ll do our best to keep compatibility for every user using it.

[Mark-Hatch-Bose]

Updated with some testing on package revisions in #4600