Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] Is merging lockfiles pre 2.0 possible? #16133

Open
1 task done
JackRenner opened this issue Apr 23, 2024 · 3 comments
Open
1 task done

[question] Is merging lockfiles pre 2.0 possible? #16133

JackRenner opened this issue Apr 23, 2024 · 3 comments
Assignees

Comments

@JackRenner
Copy link

JackRenner commented Apr 23, 2024

What is your question?

I am working with conan v1.59. We have not completed our migration to conan 2.0, and likely will not for some time. I am working with a basic diamond dependency problem to proof-of-concept our team's ability to "patch" upstream packages included in a given release, rebuild the dependency tree with all affected packages, and re-integrate into a single lockfile used at the leaf node (D).

     A
   /    \
 /        \
B         C
 \        /
   \    /
     D

Is this even possible in conan versions prior to 2.0? Without conan lock merge, I can't quite figure out a way to make this happen.

Have you read the CONTRIBUTING guide?

  • I've read the CONTRIBUTING guide
@memsharded memsharded self-assigned this Apr 23, 2024
@memsharded
Copy link
Member

Hi @JackRenner

Thanks for your question.

I am afraid not, the Conan 2 major breaking release was necessary for a multitude of reasons, one of the most relevant ones was the complexity of Conan 1 lockfiles (derived from the legacy graph computation). Being a full graph representation (lockfiles in Conan 1), it makes very challenging any attempt to try to merge anything. It is not just the lack of a built-in feature that didn't happen to be implemented, I think it is not possible to do it reasonably in Conan 1.

Lockfiles in 2.0 were designed exactly with your use case in mind, and we have tests that basically implement that flow (still pending to document it properly in our public docs).

Sorry not being able to bring better news for your usage in 1.X, the best I could do is to encourage to try to accelerate the upgrade to Conan 2, and if there is anything we can do to help you in the process, please let me know.

@aander80
Copy link

I am looking into a similar thing, and my idea (I have not tested it out fully yet, nor do I believe it will work in all cases) is to start with an empty cache. From there, I would run conan lock install on the dependent lockfiles, i.e.

$ conan lock install b.lock
$ conan lock install c.lock

This will populate the cache with the packages in the respective lockfiles. Then, I imagine running conan install or conan create (without --update) would use the packages from b.lock or c.lock in the local cache unless not fulfilled by version requirements in d/conanfile.py - those requirements would be downloaded from remotes. This is mimicking --lockfile-partial to some extent, I suppose?

This is a workaround which I have yet to test and will probably not work in all cases, but could perhaps be good enough depending on the use-cases?

@memsharded
Copy link
Member

That is actually a smart, pragmatic and effective approach @aander80, thanks very much for sharing.

Making sure you start from a blank cache, then installing very specific (locked) versions, and reconstructing the graph with exclusively the cache versions, will be effectively "merging" the contents of those locked dependency graphs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants