On PyPy, binary wheels broken on pip==20.0.1

[antocuni]

Environment

• pip version: 20.0.1
• Python version: PyPy (tested with 7.2.0 and 7.3.0)
• OS: tested on linux, likely affects other OS as well

Description
If you try to install a manylinux wheel (such as the ones produced here), pip 20.0.1 complains:

xxx.whl is not a supported wheel on this platform.

pip==19.3.1 installs it correctly.

Expected behavior
I would expect the wheel to be installed correctly :)

How to Reproduce

First, download and install PyPy 7.2.0, and install pip==19.3.1:

$ wget -q  https://bitbucket.org/pypy/pypy/downloads/pypy2.7-v7.2.0-linux64.tar.bz2
$ tar xf pypy2.7-v7.2.0-linux64.tar.bz2 
homer pip-bug $ ./pypy2.7-v7.2.0-linux64/bin/pypy -m ensurepip
[...]
homer pip-bug $ ./pypy2.7-v7.2.0-linux64/bin/pypy -m pip install -U pip==19.3.1
[...]
Successfully installed pip-19.3.1

Then, download and install a manylinux wheel: it works correctly:

$ wget -q https://antocuni.github.io/pypy-wheels/manylinux2010/psutil/psutil-5.6.5-pp272-pypy_41-manylinux2010_x86_64.whl

$ ./pypy2.7-v7.2.0-linux64/bin/pypy -m pip install psutil-5.6.5-pp272-pypy_41-manylinux2010_x86_64.whl 
[...]
Successfully installed psutil-5.6.5

Finally, upgrade to pip 20.0.1 and try again:

$ ./pypy2.7-v7.2.0-linux64/bin/pypy -m pip install -U pip==20.0.1
[...]
Successfully installed pip-20.0.1

$ ./pypy2.7-v7.2.0-linux64/bin/pypy -m pip install psutil-5.6.5-pp272-pypy_41-manylinux2010_x86_64.whl 
[...]
ERROR: psutil-5.6.5-pp272-pypy_41-manylinux2010_x86_64.whl is not a supported wheel on this platform.

I tried it with PyPy 7.3.0 as well, same result.

More info

I tried to debug it a bit. I managed to reduce the difference to this snippet:

from pip._internal import pep425tags
from pip._internal.models.wheel import Wheel

print 'Supported tags'
tags = pep425tags.get_supported()
for tag in tags:
    print '   ', tag

print
print 'File tags'
wheel = Wheel('psutil-5.6.5-pp272-pypy_41-manylinux2010_x86_64.whl')
for tag in wheel.file_tags:
    print '   ', tag

With pip==20.0.1, I get this:

Supported tags
    pp27-pypy_41-manylinux2014_x86_64
    pp27-pypy_41-manylinux2010_x86_64
    pp27-pypy_41-manylinux1_x86_64
    pp27-pypy_41-linux_x86_64
    [...]

File tags
    pp272-pypy_41-manylinux2010_x86_64

Note that "supported tags" start with pp27-, while "file tags" start with pp272-, so Wheel.supported returns False.

[andybergon]

Same problem here, had to downgrade to pip 19.3.1

[antocuni]

FWIW, I just tried and PyPy3 7.3.0 has the same problem, so this is not specific to Python2

[pradyunsg]

/cc @brettcannon @chrahunt

[pradyunsg]

Thanks for the git bisect @neishm! I was pretty sure it's because we switched to packaging.tags. I think this was discussed w/ upstream PyPy (pypa/packaging#233) but I'm not a 100% sure this is the right issue.

[pradyunsg]

I don't think so? If anything, it's related to #7626.

[webknjaz]

Ah, right. Overlooked it.

[mattip]

I think the python tag change from pp272 to pp27 is desirable, and in line with the discussion in pypa/packaging#233 and pypa/packaging#184. The pip19 python tag "pp272" is "pp" for pypy, "2" for python2, and "72" for pypy7.2.x. The python tag "pp27" is "pp" for pypy and "27" for python 2.7. Going forward, the new tag is more correct, especially as we start to support python3.6 and 3.7. Under the old naming scheme, both python3.6 and python3.7 would have the same python tag.

How to deal with the transition period? There are very few PyPy wheels on PyPI, most of them are on https://antocuni.github.io/pypy-wheels. Perhaps that site could host pip19 and pip20 wheels for a while, doubling the number of wheels there. Once the pip20 release stabilizes, PyPy could bundle pip20 with PyPy.

[gerph]

Whether there are wheels on PyPI or not, some users will be using binary wheels as part of their distribution system because it's significantly better than requiring that the end user compile the packages (or those end users may not have the ability to compile them). This is what I do with my CI, and have discovered today that the system that was working yesterday is no longer working, and (according to the message) the wheels that I'm trying to install are no longer supported. I can understand the need to differentiate between versions of Python for which the binary wheels were built for, but invalidating all existing versions seems to be... unhelpful.

If it were me, and you were intent on providing a differentiated tag by the versions and needed to keep old wheels working, I'd be thinking about making '2' be an alias for '27' in the pip 20 series on reading, and assume that if wheels are created with the '27' they're only for 2.7. Of course, this also means that the creation of wheels should explicitly allow for the creation of the tags with '2' present if you want to keep python 2 working.

I realise that it was stated that 'A future version of pip will drop support for Python 2.7', but did not quite expect that almost immediately after the 'end of life' date it would stop working.

[chrahunt]

Also note that manylinux will not be checked for PyPy.

[mattip]

@gerph it seems pinning pip and wheel (and for that matter all dependencies) versions would be prudent. Maybe when pypy binary wheels go more mainstream and the infrastructure to produce them becomes better tested, that restriction could be lifted, but it seems to me to be good practice even for CPython.

[neishm]

@ncoghlan Yes, you are correct. My problem was due to #7626 and not this issue.

[ncoghlan]

OK, having attempted to implement this by changing the set of platform tags generated when running on PyPy, and having looked into more of the history of the custom PyPy wheel tags, I no longer think modifying pep425tags.py is the right way to handle this.

Instead, now that wheel 0.34.0 has been released to build future PyPy wheels using the standard tagging scheme, I think it will be more fruitful to instead insert an extra tag into the compatibility set when pip is reading the wheel filename. Based on the Big Query results for [1], and the index of historical PyPy release notes at [2], the translation I'm going to propose/implement is:

1. If an interpreter tag present in the filename matches the regex pattern pp[0-9]{3}, then it's a legacy PyPy tag (future Python 3.10 compatible tags will look like pp3_10, so they won't match)
2. Any matching pattern starting with pp2 will result in pp27 being added to the file tags
3. For matching patterns starting with pp3, the following translations would be applied:
   • pp32x -> pp32
   • pp35x when x < 7 -> pp33 (i.e. PyPy 5.5.0 and earlier, as there was no PyPy3 5.6.0)
   • pp35x when x >= 7 -> pp35 (i.e. PyPy 5.7.0 and later)
   • pp36x -> pp35
   • pp37x when x < 2 -> pp35 (i.e. PyPy 7.0.0 and 7.1.0)
   • pp37x when x >= 2 -> pp36 (i.e. PyPy 7.2.0 and later)
4. No further patterns will be added - wheels for PyPy 8.0.0 or later will need to be built with wheel 0.34.0 or later, so they get the standard tags rather than the PyPy specific ones

@antocuni I know sys.version still reported 3.5 in PyPy 7.0.0 (as that's the version in the Fedora 30 repos), but am I right that it switched to reporting 3.6 in PyPy 7.2.0, when the Python 3.6 support left beta?

[1] There are enough full mirrors of PyPI running that the following should give all the relevant PyPy wheel interpreter tags:

#standardSQL
SELECT
  DISTINCT REGEXP_EXTRACT(file.filename, r"-(pp[0-9]{3})-") AS pypy_interpreter_tag
FROM
  `the-psf.pypi.downloads*`
WHERE
  file.filename LIKE '%-pp___-%'
  -- Only query the last 30 days of history
  AND _TABLE_SUFFIX BETWEEN FORMAT_DATE( '%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY))
  AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
ORDER BY pypy_interpreter_tag

[2] https://doc.pypy.org/en/latest/index-of-release-notes.html

[antocuni]

@antocuni I know sys.version still reported 3.5 in PyPy 7.0.0 (as that's the version in the Fedora 30 repos), but am I right that it switched to reporting 3.6 in PyPy 7.2.0, when the Python 3.6 support left beta?

PyPy 7.0.0 was a "triple release" [1]: we released a PyPy2.7, a PyPy3.5 and a PyPy3.6: I suppose that fedora decided to include only 3.5 because 3.6 was marked as "alpha". After that release, we dropped support for 3.5 so PyPy 7.1, 7.2 and 7.3 only support 2.7 and 3.6

Note that we might decide to do the same in the future when 3.7 will be in an alpha state. But I think that now that PyPy reports the correct tag, it should no longer be an issue

[1] https://morepypy.blogspot.com/2019/02/pypy-v700-triple-release-of-27-35-and.html

[ncoghlan]

Thanks @antocuni! I've updated the revised implementation in #7655 accordingly.

[ncoghlan]

Rather than relying solely on string manipulation as I suggested in my design sketch above, the updated PR ended up instead using a table of version thresholds for PyPy3.

Extracting that data table from the current PR:

# Note: the listed thresholds are the first non-alpha PyPy version that
#       *doesn't* report the given Python version in sys.version_info. This
#       means that PyPy 7.0.0 is handled as a Python 3.5 compatible release.
_PYPY3_COMPATIBILITY_TAG_THRESHOLDS = {
    'pp32': (5, 2),
    'pp33': (5, 7),
    'pp35': (7, 1),
    'pp36': (8, 0)
    # The legacy custom PyPy wheel tags are not supported on PyPy 8.0.0+
}

[pradyunsg]

Bumping this issue again - folks, if there's no updates to the PR in this week, I'm gonna defer it to pip 20.1 (April) since the 20.0 cycle is basically open only for this PR.

[ncoghlan]

Note that the PR for this issue should probably be split, with the warning deferred to a follow up enhancement.

Actually implementing what @chrahunt suggested for the warning in #7655 (review) would be quite a bit of work. Taking the warning out entirely, and letting someone keen to deprecate the workaround add the warning later would be easy.

[pradyunsg]

Sounds fair @ncoghlan! I think given that there's more work to be done for this bugfix, I'm gonna call it that it's fine to defer this to the next release.

[antocuni]

I just realized that the latest pip 0.20.2 still has the bug, which is a great inconvenience for any PyPy users using binary wheel. Event worse, the current workaround is to pip==19.3.1 in requirements in order to build: then I imagine that most people will simply forget to remove the version pinning even when this problem will be fixed, with the result of running an outdated pip for months (if not years).
Would it the possible to have this fixed in the next release?

[pfmoore]

@ncoghlan Are you still planning to look at this? The 20.1 release is due in April, and you mentioned above that there was potentially "a lot of work" here.

@antocuni If @ncoghlan doesn't have time to work on this, the likely blocker is anyone having time to pick up the outstanding work here.

[pradyunsg]

Does pip 23.0 still have this issue?

[mattip]

No. This can be closed.