[feature] Environment variables for self-consumption

[ytimenkov]

Hi @memsharded, I'd like to follow up on #8534

I assumed that when installing package with develop = True (i.e. conan install . in a directory with the recipe) means that I want to run it as well.

In my case it would be a simplification if <build_folder>/bin was added to PATH so I can run tests with same command as if I installed my package by reference (or other variables I set in package_info).

However I couldn't find way to add more variables to produced neither build_env nor run_env from inside recipe. package_info is not called in this case and VirtualEnv class doesn't provide means to add custom variables before calling its generate() method.

Something similar to layouts duality: local vs in the cache.

I wonder if this use-case makes sense or your (a Conan team) thoughts about it.

[memsharded]

Hi @ytimenkov

Do you mean the current <build_folder>/bin of the current package, or the one of the dependencies? If it is from dependencies, this should already happen with the new auto VirtualEnv (needs to activate the tools.env.virtualenv:auto_use).

Heads-up: It seems we are refactoring VirtualEnv -> VirtualBuildEnv and VirtualRunEnv. The logic is different, the use case is different, seems it make sense. This is merged in develop for 1.39.

Indeed, package_info() shouldn't be involved here, and everything should be managed in generate().
In the generate() method you could instantiate your own Environment() and then, compose it for example with the VirtualEnv output, and save the file. We are also adding an "aggregator" of environment files, to simplify this.

If necessary we could provide more examples about this, but probably the question is if Conan should do something automatic, like take the current layout and add it to the conanrunenv automatically? So far it has not been necessary, apparently the build systems know enough about where the things are to run tests, but seems it could make sense.

[ytimenkov]

Yes, I mean <build_folder>/bin of current package. So when developing package I can activate the environment, run make myprog and then run myprog (or script does this, therefore I'd like PATH to be populated).

Indeed, package_info() shouldn't be involved here, and everything should be managed in generate().

Being bit more on explicit end of spectrum in that case is fine. Repetition could be avoided by extracting into functions anyways.

In the generate() method you could instantiate your own Environment() and then, compose it for example with the VirtualEnv output, and save the file.

Ah, I see now... I think I got the impression that it provides only generate() method which wrote files. But what you're suggesting is to reimplement generate() method it in my own code: calli same VirtualEnv.build_environment and VirtualEnv.run_environment, add other variables I need and write to the same conanrunenv/conanbuildenv files.

This could work. In fact that's what I do now, but with virtualenv generator, but if new generator returns Environment object with well-defined API is much more reliable.

I also was under impression that each generator "owns" files. For example if one runs -g VirtualEnv -g MyVirtualEnv they won't overwrite conanrunenv.sh. But on the other hand recipe's generate() method has better control.

So far it has not been necessary, apparently the build systems know enough about where the things are to run tests, but seems it could make sense.

Build systems - yes, but this is for consuming / testing, especially in multi-stage CI. There are more open questions in this area...

the question is if Conan should do something automatic, like take the current layout and add it to the conanrunenv automatically?

The tribe could be asked 😉 Although I agree that this could be an abuse of everyones time... Maybe as a part of some bigger epic.

[ytimenkov]

Heads-up: It seems we are refactoring VirtualEnv -> VirtualBuildEnv and VirtualRunEnv. The logic is different, the use case is different, seems it make sense. This is merged in develop for 1.39.

I've looked at the code. Now reimplementing generate() method is trickier because of auto_activate parameter which calls to non-public save_script or register_environment_script (at least they are not re-exported from __init__.py).

I found that toolchains also register to auto-activate 👍. I just wonder how to make it easier for developer and avoid need to "activate" too many files. Generate an "activate_everything.sh" 🤭?..

Maybe register_environment_script is a step in the direction of long-waited command conan run which runs something in the recipe's environment 😜.