Confusing Error if Internal Python Builds do not Run

[WhyNotHugo]

$ rye add atomicwrites
Bootstrapping rye internals
error: No such file or directory (os error 2)

With #29 I get:

Bootstrapping rye internals
Error: error bootstrapping venv

Caused by:
    0: unable to create self venv
    1: No such file or directory (os error 2)

[mitsuhiko]

@WhyNotHugo What CPU architecture and operating system are you on?

[WhyNotHugo]

Alpine Edge, amd64.

Not sure if related, but I've no idea where where the 3.10 one came from:

> rye toolchain list
cpython@3.10.9
custom-cpython@3.11.3

[mitsuhiko]

Alright, so here is what's happening. At the moment rye always uses cpython@3.10.9 from the indygreg builds. I am assuming that for whatever reason that build is not working for you. Might be worth trying to run this and see what it outputs:

$ ~/.rye/py/cpython@3.10.9/install/bin/python3 --version

[WhyNotHugo]

I'd rather opt out of downloading python binaries entirely. My distro has working Python packages and I'd rather rely on those than pulling binaries from elsewhere.

[WhyNotHugo]

> ~/.rye/py/cpython@3.10.9/install/bin/python3 --version

zsh: no such file or directory: /home/hugo/.rye/py/cpython@3.10.9/install/bin/python3

[mitsuhiko]

@WhyNotHugo I absolutely understand that desire, but in some sense my goal is to get an experience going first that involves ensuring a working cpython installation from the internet, before going into solving distribution provided python installations. These have been a historic challenge to providing a reliable packaging experience.

[mitsuhiko]

@WhyNotHugo this is puzzling

1. does the error still happen if you rm -rf ~/.rye/py/cpython@3.10.9?
2. what is the contents of ~/.rye/py/cpython@3.10.9 and ~/rye/py/cpython@3.10.9/install/bin?

[WhyNotHugo]

my goal is to get an experience going first that involves ensuring a working cpython installation from the internet

But this is exactly the problem that a software distribution solves. E.g.: Alpine has python3, py3-pip, etc. Fedora, Arch, OpenBSD, and many others all have equivalent packages.

Your distribution no existing mechanism to have a working Python at all?

[WhyNotHugo]

I think this comes down to #30

[mitsuhiko]

I would not say so. Nothing is ever quite as black and white, it's a matter of priorities. The experience I care most about is the one where a developer downloads a tool (let's call it rye) which they can use to get a functioning Python installation going on their system that gives them the same experience as on any other system (eg: the rustup + cargo) experience. This might very well involve having to fall back to system installations of Python, I don't quite know the answer to it.

However what I do know is that priority wise right now is making sure that on systems where the self installed downloaded versions should work, they do work, before looking at system Python installations. System python installations are a huge source of frustration right now because of how many customizations they have.

Here some motivational reasons on this:

• Ubuntu 22.04, Python 3.10, and broken virtual environment layouts python-poetry/poetry#6371
• https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1967920
• fix building python3.10 virtualenvs on debian derivatives pypa/virtualenv#2415

[hemangjoshi37a]

To resolve this issue, please follow these steps:

1. Remove the existing cpython@3.10.9 directory by executing the following command: rm -rf ~/.rye/py/cpython@3.10.9.
2. Check the contents of the ~/.rye/py/cpython@3.10.9 and ~/rye/py/cpython@3.10.9/install/bin directories.
3. Verify the Python version on your system by running python3 --version.
4. If you have a working Python installation on your system, use it instead of the one provided by Rye.
5. Share the output of the above commands with the repository maintainers for further assistance.

Keep in mind that the Rye project is prioritizing the user experience of having a working Python installation downloaded from the internet. This may change in the future to accommodate system installations of Python, but for now, the focus is on ensuring the downloaded versions work properly.

[alissonlauffer]

It doesn't work because Alpine Linux uses musl, and the installed python build is built against glibc. The "no such file or directory" error does not mean that the executable was not found, but rather that the required interpreter to execute it (/lib64/ld-linux-x86-64.so.2) was not found because the host uses musl.

[WhyNotHugo]

Oh, I though that the error had been because rye had tried to execute a python which I had not downloaded/fetched first (e.g. Python interpreter not found).

To be honest, I'm pretty discomforted by the fact that rye silently downloaded some Python binaries from some random third party source and executed them without any confirmation or indication. rye should definitely prompt before doing such things.

In any case, this general approach is a recipe for disaster, you can't just pull binaries from one distribution channel and run them elsewhere and expect them to work. You should never pull binaries from different sources unless they're explicitly built for the target host. Maybe that would work if they're statically linked, but it still sounds like a bad idea with tiny gotchas all over.

This approach of "my distribution is broken and doesn't ship a working Python, I'll pretend that distributions don't exist and reinvent things from scratch" is rather childish IMHO, and you're basically making a solution designed for broken distributions and ignoring the non-broken ones. But hey, you build for the future you want to see, so if you want to see a future where these distributions (which don't even ship a working Python) flourish, then this is definitely the way to go.

In any case, this is covered by #30, there's nothing actionable on this ticket.

Keep in mind that the Rye project is prioritizing the user experience of having a working Python installation downloaded from the internet.

Yup, it'd been pointed out to me. Regrettably, I'm not looking to switch my Python distribution nor package manager, both the ones I have work for me. I landed here expecting to be able to use rye as a dependency/virtualenv manager, but it's seems this is an all or nothing, so I'll just stick to pip-tools and mkvirtualenv until somebody tries to build something better on top of what already exists.

[mitsuhiko]

Closing this as resolved for now. The error message is better now. For actually supporting MUSL I do not have a plan yet.