How do you determine the file name that wheel will build?

[geofft]

We have a set of Makefiles that orchestrate some builds, including wheel builds. Here's a super simplified conceptual version (that is heavily abstracted from what we're actually doing, but I think it's illustrative):

all: ... ${XYZ_WHEEL} ...

XYZ_SDIST := xyz-1.0.tar.gz
XYZ_WHL := xyz-???.whl

${XYZ_WHL}: ${XYZ_SDIST}
    pip wheel ${XYZ_SDIST}

or perhaps

XYZ_SOURCES := this.py that.py
XYZ_WHEEL := dist/xyz-???.whl

${XYZ_WHEEL}: ${XYZ_SOURCES}
    python -m build

How do we figure out what the wheel filename is going to be?

In the past, we did this by setting a variable to the output of

from wheel import pep425tags
print("-".join(pep425tags.get_supported()[0]))

for the current interpreter, and setting XYZ_WHEEL := xyz-1.0-${WHEEL_SUFFIX}. Obviously this broke after the removal of pep425tags in 2020 (yes, I'm a little behind on filing this).

The recommendation in e.g. #370 is to use packaging.tags, but as far as I can tell that's not quite helpful here. That gets you a list of all possible tags that the current interpreter could install. It's perhaps of some use in wildcarding a directory after it's built, but for writing a Makefile (or several other types of similar build orchestration logic, judging from the pings on #346), you'd want to know the filename in advance. What we need is a way to ask wheel what particular tag it's going to use.

I'm currently going with the following, which seems to work:

from setuptools import Distribution
from wheel import bdist_wheel
cmd = bdist_wheel.bdist_wheel(Distribution())
print("-".join(cmd.get_tag()))

but that seems pretty fragile too - apart from wheel not having a public API at all, this constructing a setuptools command object by hand seems particularly fragile. (The proposed public API in #472 does not seem to refactor the get_tag() logic into a public function, as far as I can tell.)

Furthermore, there is a legitimate objection to what we're even trying to do: the concept of "what filename will get used" is not a constant, it's a function of what's going on in the particular source package. For instance, for universal / pure wheels, they get a more generic tag. (You can get this out of the second approach with cmd.universal = cmd.root_is_pure = True, but that just makes it even more fragile....) Arguably we should be looking at the specific sdist / pyproject.toml / whatever to get the exact answer, but I think you can approximate this in most practical use cases with a couple of flags about whether it's universal / pure / limited API / etc.

Is there a more obvious way to do this that I'm missing? Judging from the fact that the actual logic is in bdist_wheel.get_tag() and not in a separate library like packaging, I think there is not.

In that case, would you be willing to expose a command like python -m wheel get-tag or a public API function to address this use case? I think something like the following would work for a command line:

usage:
    wheel get-tag [--universal] [--pure] [--limited-api] [--platform PLATFORM]
    wheel get-tag <sdist.tar.gz>
    wheel get-tag <directory>

or maybe python -m wheel get-filename which requires passing in the distribution's name and version in the first case, or something. (In the sdist case, it would be nice if it could avoid unpacking the entire sdist and can just look around in the tarball for pyproject.toml/setup.cfg.)

(I could probably send in a PR to implement this, if the approach is acceptable to you, but I mostly just want to get discussion on what you think of the problem in general.)

[henryiii]

You cannot compute what a wheel tag will be in advance, because that depends on the backend. A setuptools / wheel wheel will depend on what's in the setup.py - adding an extension causes it to be a platform specific wheel instead of a general one. You can also get universal wheels (py2.py3), ABI3 wheels, and potentially others. Which one you get can depends on environment variables, like SETUPTOOLS_EXT_SUFFIX, _PYTHON_HOST_PLATFORM, entries in DIST_EXTRA_CONFIG, the file pointed at by _PYTHON_SYSCONFIGDATA_NAME, and more.

And for all other backends, that depends on the backend. Backends like poetry, hatchling, scikit-build-core, cmeel, py-build-cmake, meson-python, maturin, enscons, and pdm-backend can produce a variety of outputs. I think flit-core, flit_scm, whey, trampolim, jupyter_packaging, and sphinx_theme_builder mostly produce pure Python wheels. I'm not familiar enough with the others, like pbr, mina, partis, sipbuild, etc., to say much on those.

[geofft]

Isn't the implementation in wheel.bdist_wheel.bdist_wheel.get_tag() relatively limited? Like I think it's parameterized over universal, pure, limited-API (abi3), and platform tag override, and influenced by the current Python version (I am happy to assume that the interpreter running setup.py is the same as the interpreter in which I'm doing import wheel, including environment variables), but not really other than that. So with only that much information about how the project is configuring bdist_wheel, you should be able to determine the answer, right?

If it helps scope the problem - most of what I'm trying to avoid is hard-coding a list of platform tags into our own Makefile because of the special cases of things like the m at the end of cpython37m. We have a good sense of what the actual build backends are in the wheels we're building: we know what packages we're building, we're already installing their build dependencies, etc. (So every time a new backend comes along, we're paying at least a tiny bit of manual attention to it - yesterday I had to tell our build system that poetry_dynamic_versioning counts as poetry.) In particular we're currently hard-coding whether a specific wheel is universal or not, and I'm fine continuing to do that. If something changes in a way we didn't expect, and the wheel comes out with a different name, we'll get a build failure and we can tweak our Makefiles as needed. What I'm trying to avoid is needing to tweak the Makefiles in the common case.

I understand that by PEP 517 there is no formal answer other than actually calling the backend's build_wheel and seeing what it returns, and a backend doesn't even have to use wheel at all to build a wheel, but IMO it's still useful to have something that can answer the question in the typical cases ("practicality beats purity" :) ). But I can also see the argument that a public API that is intentionally imperfect is a little weird.