[bug] silent recipe change breaks our pipelines

[h-2]

Describe the bug

We have a local conan setup with some official and some custom packages. We have our own artifactory, and we have set up conan to always pull binary packages from our artifactory. This has worked very well up until now.

Apparently, the fontconfig package has been updated to require conan-2.2. This is fine of course, except that it is not relfected in the version and for some reason this new recipe is forced into our setup. Because our setup is currently based on Conan-2.0.14, everything breaks.

1. How can I make sure that the recipes are also pulled from our artifactory and not from conan-official? I really don't want any upstream changes breaking our setup.

2. I think existing recipes/packages in conan-center should not be updated without this being reflected in the version number. Otherwise, you break semantic versioning. In this case, fontconfig-2.15 (the new package) obviously is not compatible to fontconfig-2.15 (the old package). I strongly suggest you add a version suffix for such cases, e.g. fontconfig-2.15~conanX where X is 1...n.

Thank you for your help!

How to reproduce it

I am currently running:

% conan install "./vsrc" --output-folder . --build=missing --profile:all=RelWithDebInfo --conf:all=tools.build:jobs=6

This returns:

*********************************************************
Recipe 'fontconfig/2.15.0' seems broken.
It is possible that this recipe is not Conan 2.0 ready
If the recipe comes from ConanCenter, report it at https://github.com/conan-io/conan-center-index/issues
If it is your recipe, check if it is updated to 2.0
*********************************************************

ERROR: Package 'fontconfig/2.15.0' not resolved: fontconfig/2.15.0: Cannot load recipe.
Error loading conanfile at '/home/hannes/.conan2/p/fontcf45107370b2f2/e/conanfile.py': Current Conan version (2.0.14) does not satisfy the defined one (>=1.64.0 <2 || >=2.2.0)..

Although I have a fontconfig/2.15.0 package in my local cache and in my artifactory. But it's the upstream recipe that is breaking by builds.

[h-2]

This problem even appears if I explicitly conan remote disable conancenter

[memsharded]

Hi @h-2

Thanks for your report.

Apparently, the fontconfig package has been updated to require conan-2.2.

I don't think this is possible. ConanCenter still do builds for Conan 1, and it won't accept changes that do not work with Conan 1. It might have got some updates, but the recipe should keep working with conan 1 too.

I think existing recipes/packages in conan-center should not be updated without this being reflected in the version number. Otherwise, you break semantic versioning. In this case, fontconfig-2.15 (the new package) obviously is not compatible to fontconfig-2.15 (the old package). I strongly suggest you add a version suffix for such cases, e.g. fontconfig-2.15~conanX where X is 1...n.

This is not going to happen. This was a deliberate decision and there was very strong feedback from the community that the current behavior was the desired one by users. Versions of packages in ConanCenter should match exactly the upstream versions of the original source/git-repos. If there are changes in dependencies or configuration, that will be modeled and managed by the package_id.

How can I make sure that the recipes are also pulled from our artifactory and not from conan-official? I really don't want any upstream changes breaking our setup.

I think that if you are looking for that kind of reproducibility you need to use one of the two approaches, maybe even both of them as recommended in https://docs.conan.io/2/devops/using_conancenter.html, basically:

• At the very least use a lockfile. That will guarantee that your dependencies will keep using the same versions and specially the same revision from ConanCenter.
• If you need even more guarantees that nothing break your builds, then host your own packages in your own server. This can also be done with 2 different approaches, copying the packages from ConanCenter, or building from source your own binaries and uploading those binaries to your own binary server. If you use a server, and you control the updates in that server, then you might not need a lockfile, if you don't need to go back in time.

How can I make sure that the recipes are also pulled from our artifactory and not from conan-official? I really don't want any upstream changes breaking our setup.

Conan 2.2 introduced the conan remote add --allowed-packages to define which packages are accepted from a given remote.

In any case, if I understood this correctly, this is not a bug in Conan client, this is expected behavior from ConanCenter that evolves and changes, and users needing stronger reproducibility guarantees should use lockfiles or host their own copy of their binaries in their own server.

Thanks very much for your feedback!

[h-2]

Thank you for your quick reply!

Apparently, the fontconfig package has been updated to require conan-2.2.
I don't think this is possible. ConanCenter still do builds for Conan 1, and it won't accept changes that do not work with Conan 1. It might have got some updates, but the recipe should keep working with conan 1 too.

Just to be clear, we were using conan-2.0.14. And the above error message quite clearly states that the recipe no longer works on this version:

Current Conan version (2.0.14) does not satisfy the defined one (>=1.64.0 <2 || >=2.2.0)

At the very least use a lockfile. That will guarantee that your dependencies will keep using the same versions and specially the same revision from ConanCenter.

Ah, OK. So the version of fontconfig-2.15 stayed the same, but the revision changed? And using a lockfile would have kept us at the old revision ? Then a lockfile would indeed have prevented this problem for us, I think. I will investigate this further!

If you need even more guarantees that nothing break your builds, then host your own packages in your own server. This can also be done with 2 different approaches, copying the packages from ConanCenter, or building from source your own binaries and uploading those binaries to your own binary server. If you use a server, and you control the updates in that server, then you might not need a lockfile, if you don't need to go back in time.

What exactly do you mean by "own server"? We already have our own artifactory, and we always pull our binaries from there. It just appears that the recipes are still automatically pulled from upstream.

Conan 2.2 introduced the conan remote add --allowed-packages to define which packages are accepted from a given remote.

But this still only affects the binaries and not the recipes, right?

In any case, I will investigate the lock command. I think, it will do what we need!

[memsharded]

Just to be clear, we were using conan-2.0.14. And the above error message quite clearly states that the recipe no longer works on this version:

Oh, I see now. This is very unusual, but it is true that some recipes using Meson did an update to latest Conan 2.2 to solve some issues, while retaining compatibility with Conan 1 (with some if conan_version < 2 in the recipe). Now I understand why it passed in ConanCenter CI. Thanks for clarifying!

Ah, OK. So the version of fontconfig-2.15 stayed the same, but the revision changed? And using a lockfile would have kept us at the old revision ? Then a lockfile would indeed have prevented this problem for us, I think. I will investigate this further!

Yes, exactly, this is totally the purpose of lockfiles, to guarantee that the same dependency graph, down to the recipe revision can be reproduced, no matter what new versions or revisions are uploaded to the servers.

What exactly do you mean by "own server"? We already have our own artifactory, and we always pull our binaries from there. It just appears that the recipes are still automatically pulled from upstream.

Oh, now I understand better, it seems that you have an Artifactory server that is using a "remote" to proxy ConanCenter. While this approach can be useful for some cases, for achieving better control, it is better to fully decouple it. Just create your own repo in Artifactory, but not remote to ConanCenter. Then you build your binaries from ConanCenter or just download them from ConanCenter and upload them to your own private "local" repository. That way no matter what changes in ConanCenter, until you repeat this process yourself absolutely nothing will change. The Artifactory "remote" repositories do continuously update, tracking changes in ConanCenter.

For building your own binaries from ConanCenter, and uploading them to your own "local" repository the new feature announced yesterday might be useful, please check https://blog.conan.io/2024/04/23/Introducing-local-recipes-index-remote.html.

But this still only affects the binaries and not the recipes, right?

It also affects the recipes, that filter will not allow recipes and packages by name patterns.

In any case, I will investigate the lock command. I think, it will do what we need!

Sounds good. If there is no immediate further action in this ticket, I think we can close it, and you can re-open or create new ones for any further question or issue. Many thanks for the feedback!