[question] Conversion of AutoToolsBuildEnvironment based recipe to conan v2.0 unsuccessful

[sorny92]

What is your question?

I have a recipe which uses AutoToolsBuildEnvironment to manage the build process.

The key part more or less looks like this:

        autotools = AutoToolsBuildEnvironment(self.conanfile)
        env_vars = autotools.vars

        tools.mkdir("./build")
        with tools.chdir("./build"), tools.environment_append(env_vars):
            self.conanfile.run(f"../src/configure {config_args}")
            self.conanfile.run("make -j8")
            self.conanfile.run("make install -j8")

This recipe has some build_requires so tools.environment_append(env_vars) it's used to expose CFLAGS and LDFLAGS to make.

I'm trying to convert this recipe to conan 2 but I'm suffering a problem when
self.conanfile.run("make install -j8") is run as it won't be able to find one of the libraries that is a tool_requires.

I understand the key part is not being able to do properly with tools.chdir("./build"), tools.environment_append(env_vars) with conan 2.

I've replicated https://github.com/conan-io/conan-center-index/tree/master/docs/package_templates/autotools_package for my package but I haven't been able to succeed.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @sorny92

Thanks for your question.

Your issue might not be that related to the helper, but to the fact that tool_requires cannot contain libraries, but mostly executables (cmake, meson, etc). The reason is that tool-requires live in the "build" context that can have a totally different configuration, for example when cross-compiling, or when building as Debug. If it is a library, the first step is to convert it to a regular requires.

[memsharded]

Did you manage to make it work @sorny92 with that hint?
Don't hesitate to re-open or create new tickets if you have any further question

[sorny92]

It did not fix it, but the problem was what I mentioned in the Issue.

If you are curious about this is the link to the library:
https://github.com/lumicks/embedded_python

We build a custom embedded Python so there's some dependencies that are just needed to build the dependency tree, as such tool_requires are ideal because they don't leak requires to whatever consumer uses embedded_python.

We are not really using it as it should I guess but that's better than leaking all those build time dependencies to the consumers

[memsharded]

We build a custom embedded Python so there's some dependencies that are just needed to build the dependency tree, as such tool_requires are ideal because they don't leak requires to whatever consumer uses embedded_python.

With Conan 2.0, the dependencies are not leaked to consumers (in the headers + linkage meaning), because the new dependency graph takes care of that. The recipe of the dependency might still be needed, to correctly compute the package_id and make sure that it is not necessary to rebuild the embedded python, but the binary of the dependency will be completely skipped, not even downloaded from the server.

I suggest having a look to the first video in https://docs.conan.io/2/knowledge/videos.html, it will explain the new "traits" in 2.0.

For a even-more decoupling approach, it would be the "re-package" concept, this is planned in the 2.X roadmap, this is the ticket that you could track to see progress: #13171

[sorny92]

With Conan 2.0, the dependencies are not leaked to consumers (in the headers + linkage meaning), because the new dependency graph takes care of that.

In our case the dependency that failed was zlib this was just needed to unpack some packages for Python.
When a consumer uses embedded_python and zlib is a tool_requires this will no appear in the dependency tree of the consumer, but if it's a requires it will appear. If a consumer made use of zlib but required a different version this will conflict even that internally they really don't, right? Or am I understanding it wrong?

[memsharded]

In our case the dependency that failed was zlib this was just needed to unpack some packages for Python.

If you are using zlib strictly as a tool (you only want the executable to unzip things), it is perfectly valid to use it as tool_requires. Not only valid but recommended.

If any library links with zlib libraries, then by default Conan 2.0 will propagate things to make conflict if that consumer library is static (because you can only link statically 1 version of zlib), otherwise, Conan 2.0 makes possible to hide static libraries by shared libraries to avoid these conflicts. There is also a explicit visible=False trait for some extreme cases, but it shouldn't be necessary for many common cases as the typical relations shared-static-header are already modeled by 2.0

[sorny92]

If you are using zlib strictly as a tool (you only want the executable to unzip things), it is perfectly valid to use it as tool_requires. Not only valid but recommended.

A tool needs to link to zlib library (there's no executable) but only during the build step. That's the weird part. As you say, if it's an executable it's easy, but it's just dynamically linking during the build process.

So setting zlib as a require but with visible=False should be the way forward? Would that still collide if something else required zlib but with a different version?

[memsharded]

So setting zlib as a require but with visible=False should be the way forward? Would that still collide if something else required zlib but with a different version?

Yes, that might be one of the cases that would benefit from visible=True

Also it depends on the "embedded_python" package_type. Is that a library or only the executable interpreter. Because defining package_type="application" it will also make the transitive dependencies not conflict downstream, visible, but not conflicting.