Support different backends for pipx?

[huxuan]

How would this feature be useful?

There are several open issues about supporting different "backends", such as #1370 for uv and #853 for virtualenv. Moreover, there is already a project pipxu that claims to have reimplemented pipx while using uv. So I would like to have your comments about whether we should support different backends.

Describe the solution you'd like

1. Abstract all the operations related to managed virtual environments.
2. Implement venv, virtualenv and uv for these abstracted operations.
3. Add virtualenv and uv as optional dependencies and use them with higher priority than venv when pipx is initialized.

Describe alternatives you've considered

There might be two other approaches to implement the same feature:

1. Create a backend plugin system and implement the backend in a separate repo.
2. Create a standalone and universal backend library that can be used directly in pipx.

[chrysle]

1. Abstract all the operations related to managed virtual environments.
2. Implement venv, virtualenv and uv for these abstracted operations.

Sounds good, ans would be similar to what nox is already doing. We must prevent that uv steals our jobs ;-)!!

Actually, I find it really strange that someone just re-implements an actively developed program for a functionality that could be added to the same, and on his own, too!?

[cs01]

the uv readme says:

Think: a single binary that bootstraps your Python installation and gives you everything you need to be productive with Python, bundling not only pip, pip-tools, and virtualenv, but also pipx, tox, poetry, pyenv, ruff, and more.

Our goal is to evolve uv into such a tool.

so if they are successful in implementing their roadmap, it will subsume all of these tools including pipx (and pipxu)

Ultimately it will be up to users if they want to use uv, but I do suspect users want a single well-built answer to Python packaging and common workflows. I think it’s a real possibility pipx becomes essentially deprecated in favor of uv. Not sure if or when that will happen exactly but I think it’s important to be realistic that this is a possible future.

It doesn’t mean pipx is bad - it means pipx was successful in making its feature set an expectation for Python packaging ecosystem. This is a win for Python users and the language itself.

Even if uv implements pipx, having competing tools is usually a good thing for users, since they push eachother to be better and faster. So +1 to having pipx use uv as a backend.

[huxuan]

Hi @cs01, nice to meet you!

I agree with you completely, and personally I am eager to see that happen, like how 'ruff' replaced all lint tools. So, just as mentioned in the issue, do you have any comments whether we should think about making a single, universal backend library? Just like @chrysle said, 'nox' already implemented something similar. And I've seen 'pdm' made it too. We can avoid the reinventing of the wheels and advocate a consistent approach. This could be useful during migration.

[chrysle]

So, just as mentioned in the issue, do you have any comments whether we should think about making a single, universal backend library?

Hmm, doesn't sound like a bad idea. However, for more projects than pipx to use it, it would have to be highly customizable. And there are different backends for different needs, e.g. installation, virtualenv creation, lockfile management etc., thus it would require differentiated modularization. And 'single, universal' sounds like a standard could be needed.

[huxuan]

And 'single, universal' sounds like a standard could be needed.

Yep, that is the point, and I suppose PyPA might be the most suitable or even the only organization to do that.

[AdrianVollmer]

I'd love to see uv as a backend for pipx.

I think it’s a real possibility pipx becomes essentially deprecated in favor of uv. Not sure if or when that will happen exactly but I think it’s important to be realistic that this is a possible future.

Keep in mind that astral is VC backed, so there are investors who presumably want to see returns at some point. The interests of users and investors often don't exactly align. While I can see avenues that might be win-win, I can also see avenues resembling enshittifcation, so I think there will always be value in having and maintaining pipx.

[cs01]

Nice to meet you too @huxuan :)

So, just as mentioned in the issue, do you have any comments whether we should think about making a single, universal backend library?

Would the backend library be in Rust, call into uv, and have python bindings for callers like pipx?

tbh I don't feel I am actively involved in the Python ecosystem or on pipx anymore to have a strong opinion, but it does sound pretty interesting. I'm just glad to see so much interest in continuing to push pipx and forward. Thanks so much.

[huxuan]

Would the backend library be in Rust, call into uv, and have python bindings for callers like pipx?

What I mean is a Python wrapper for different modules with capability of managing virtual environments, such as venv, virtualenv, conda and uv. It should be almost the same as the abstraction and implementation pattern, but the advantange is that we can maintain a more robust library and use it in different projects.

[henryiii]

I think there should be separation between --installer (uv or pip) and --venv-backend (venv, virtualenv, uv). I also like having a uv|pip option, which means use uv if available, fall back to pip.

The main questions, IMO, are:

• Should uv|pip be the default, or should pip remain the default, for backward-compat? (build and nox went with the former, hatch is doing something inbetween, with all new features like PEP 723 using uv and older ones being opt-in).
• Is there a way to make this configurable without having to pass a command line option? An envvar would make sense, but is there a place we could put a config option?

[huxuan]

Hi @henryiii,

Thanks for the suggestions! It is quite informative. I did not realize that there should also have an --installer option.

For uv|pip option, a fallback strategy is definitely useful, but we may not need to speicify it explicitly. In my assumption, when we specify uv for the --installer and uv is not available, we can fallback to pip with a warning message. WDYT?

[pfmoore]

I would love to see uv as a backend for pipx. In particular, the speed of uv would make it ideal as an installer backend for pipx run, particularly when running a script. The run command is typically used for occasional tasks that don't necessarily benefit as much from caching the environment.