Building with Miniconda

[benjaminrwilson]

Hi all,

I had a few questions regarding best practices + handling environment variables during builds.

1. Is there any downside to using conda for dependency management during the build process (e.g., https://github.com/marketplace/actions/setup-miniconda)? I've previously tried using system-level package management such as apt, brew, and choco for various platforms, but I found that the install locations weren't consistent.
2. With conda, an environment variable, $CONDA, is set as a reference to the path to the current conda installation. I've tried using this in cibuildwheel by setting: CIBW_ENVIRONMENT: "CONDA=$CONDA, but it seems to not initialize this variable during the build. What is the correct way of doing this?

Thanks, for the awesome project!

[Czaki]

Is there any downside to using conda for dependency management during the build process (e.g., github.com/marketplace/actions/setup-miniconda)? I've previously tried using system-level package management such as apt, brew, and choco for various platforms, but I found that the install locations weren't consistent.

For macos both conda and brew are bad sources. Both could provide libraries compiled against the current macOS version (this one from host runner), not targeted one.

apt, brew, and choco are great for install the build tools but I suggest build libraries from the source.

[henryiii]

Agree here, Conda supplies their own compiler stack and does not use the system one. It's very unlikely this will work for a redistributable wheel for macOS or Linux. Maybe it would work on Windows, but by that point, you are back to a single-OS system anyway. Yum (manylinux1-2014) or apt (manylinux_2_24) should be fine, if you can build with such old versions of things. Choco's probably okay on Windows.

[benjaminrwilson]

Thank you for the responses @Czaki and @henryiii. This is very helpful!

[joerick]

Hi @benjaminrwilson !

Irrespective of @Czaki's response above, which is probably good advice,

2. With conda, an environment variable, $CONDA, is set as a reference to the path to the current conda installation. I've tried using this in cibuildwheel by setting: CIBW_ENVIRONMENT: "CONDA=$CONDA, but it seems to not initialize this variable during the build. What is the correct way of doing this?

I'm assuming you've hit this issue on the Linux build? On Linux, the ability to passthrough environment variables isn't yet implemented (see #730 and #117). But in this case, passing through the path wouldn't help, because that path isn't valid inside the linux container. Linux builds happen inside Docker 'manylinux' containers, so software installed on the host machine isn't available. You'd have to install conda inside the build container, using CIBW_BEFORE_ALL.

[benjaminrwilson]

Thank you, @joerick. That helps a lot!

[isuruf]

Although it's not advisable to use conda to provide dependencies, it's possible if you know what you are doing. See https://github.com/symengine/symengine-wheels#repository-to-build-wheels for some catches.