How to repackage existing packages?

[liberforce]

This is a follow up of several issues:

My goal is to create slightly modified versions of existing packages. For example, to add some metadata file for consumption by a custom build system. I don't want to fork the recipes because it would make it harder to move to newer versions of the dependencies.

So I tried several approaches:

1. Repackaging like explained in runtime packages: that didn't work as I need to repackage libraries too, and want to distribute a variant of each binary package. This would need to keep settings and options in sync.
2. Deriving the package using python_requires. I then felt in exports not copied when inheriting from another package with python_requires #3538 as this only derives the python code in the recipe but not the package.
3. Deriving the package using both python_requires to get the exact settings + options set, and repackaging the artifacts. This is very close to what I want, but still has some drawbacks. The use of python_requires is a dependency at export-time, meaning that when repackaging recipes I have which are already a fork of official recipes, the order in which I export them becomes important. I fortunaltely only have boost in this case, but still. Other problems I got are conan create unnecessary forces rebuild of build-time dependency #3639 (rebuilding when not required, not that bad) and Repackaging a library pulls the wrong binary package #3759 (and that one is a problem, as it produces incorrect output). Those are a side effect of using a package that build_requires from a package with the same name, but different version.
4. Use plugins, as I've been told. But then I have no control on the events triggered when someone installs a package, as it may be built, or the binary package downloaded, and if it's not built I can't have a chance to repackage it on the fly as the new files may be added but then the package will get out of sync from the manifest.

So, does anyone have an hint on how to solve this that wouldn't mean forking all the recipes?
Currently what seemed closest to work was (3), as it could just download existing binary packages and repackage them without rebuilding, but I'd need to know if the trick of depending using build_requires on a package with the same name but different version is seen as a valid use case or not.

Thanks.

Using Conan 1.8.2 on Ubuntu 14.04.

• I've read the CONTRIBUTING guide.
• I've specified the Conan version, operating system version and any tool that can be relevant.
• I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

[jgsogo]

We've being thinking about the repacking issue, and we agree that there is no out-of-the-box solution. The best solution is the one you propose: python_requires + build_requires as it is the only one that will be able to grab all the transitive information from dependencies (thanks to the python_requires) and eventually would be able to reuse the binaries.

Nevertheless, at this moment, the build_requires doesn't give you any advantage (#3639) or even has some problems (#3759), and we are not sure if it is the right approach neither. So at this moment, the advice is to use only python_requires given that #3538 is already working (we should document it, we forgot about this conan-io/docs#916 😨 ).

[jgsogo]

This issue is still alive and it is an open question. Up to now:

• repackaging using the same ref.name leads to an error (fix recursive build requires #4783)
• python_requires won't automatically export sources (Refactor local workflow: export and export_sources order #4340)
• more changes are to be discussed regarding python_requires (Python Requires restrictions #4635)

Let's keep this open, it is an interesting use case, but we don't have an answer yet. Thanks!

[memsharded]

Closing this in favor of the 2.X roadmap "re-package concept" ticket: #13171, please follow up there.