Drop support for EOL Python 3.7 #11944
Conversation
|
Shouldn't we wait until supporting 3.7 is actually a maintenance burden before doing this? |
|
I see the attraction of using more modern constructs, but I agree we should be conservative about actively making things incompatible. Let’s just remove it from CI but not change |
I don't recommend that: it would allow installing on 3.7, but we don't know if it still works on 3.7... For a minimal change, we could remove from CI and change |
|
My point is that I'm -1 on actively blocking the use of pip under Python 3.7 without good reason. And "3.7 is EOL" doesn't feel like a good enough reason by itself. Actually, note our official Python support policy. It's not based on whether the core Python version is EOL, but on usage on PyPI being below 5%. Do we have evidence that this is the case for Python 3.7? |
|
Nope, it's 28%:
Date range: 2023-03-01 - 2023-03-31 Source: Python 3.6 was dropped in November 2021, for pip 22.0, a month and a half before its EOL: #10461 (comment) At a time when 3.6 was at 24%:
Date range: 2021-10-01 - 2021-10-31 Source: https://github.com/hugovk/pypi-tools/blob/main/data/pip-2021-10.json, archived from https://pypistats.org/ But that was to ease adopting Rich. Shall we close this and revisit in another year or two? |
|
Maybe pip could start to warn on eol python 3.7 to add extra drive to drop it Is there any way to distinguish ci usage from downstream usage? |
This was done for 2.7 (#6147) and 3.5 (#8778), although the 2.7 removal was already planned for EOL+1 year (the 5% policy was only for 3.x), and 3.5 was on track to reach <5% by the target date.
Not really, although the numbers do exclude mirrors such as bandersnatch and Artifactory. |
|
At a minimum, please change the support policy if you want to drop 3.7 support before usage drops to 5%. But I don't see a good reason to change the policy; there may be other Python distributions whose EOL is further out. |
|
Let's leave this open for now but defer further discussion until we see a need to do this beyond EOL.
We can look at https://pypistats.org/packages/__all__ in 2-3 months to see if the ratios change, and if this blocks something for us that we'd find problematic. |
|
My 3 cents. If people really would want to use Python 3.7, they will still be able to use old They will not have new features and some breaking changes will not be enforced for them. Are we seriously think people will want to upgrade to new pip, when they are not upgrading EOL Pyhon ? Suggestion: I think it is important (unless it is already done like that) that |
Sadly, that's not realistic. Far too many people just blindly pull in the latest pip in their CI and similar. Saying they shouldn't do that has very little effect, so we have to be conservative - which is why we have the policy that we do. |
But if you publish |
|
Another category of complaints is when a package requires some new feature of Pip but otherwise would work fine with Python 3.7 and the user is stuck on Python 3.7 (for whatever choices of the organization they are in). I think this was more of an issue in years past when packages in general didn't have a strong Python version support policy, but it would surprise me if it didn't come up again, Users still report issues they have on 3.6 (#11959), though this was because of the package support policy not because of lacking a new Pip. |
|
3.7 seems to be gradually dropping below 3.8, but not enough that we should be desupporting 3.7 in pip 23.2: 
|
Similar to what the other person said, unless I'm missing something, that will continue to work as expected. Can you please explain what would cause the Python version requirement to not be respected? |
Sorry, you're right. I withdraw that comment. What I meant was that I'm -1 on actively preventing people using the latest version of pip under Python 3.7 without good reason. So yes, the version requirement would give 3.7 users a working pip, but not the latest pip. And in any case, our existing policy is based on usage, not Python support lifecycle. Ultimately, I don't think we gain enough to justify dropping 3.7 support, and our formal criteria for desupport haven't been met yet. It makes no personal difference to me (I haven't used 3.7 in years) but I've been in situations where I've been on old platforms that I can't change and things get desupported, and it sucks. So I'd rather not rush to inflict that pain on our users. |
|
I didn't realise we hadn't done this yet. I think we should drop Python 3.7 now, even though that's not what the usage numbers suggest.
I agree it sucks, but I don't want to be subsidising the costs for individuals/organisations staying on EOL Python versions indefinitely in my free time. :) Quoting past me:
I understand and appreciate that the network effects of pip dropping a Python version exist, but I absolutely do not think that we should be maintaining support for up to 10+ Python versions just because someone with a large server farm keeps downloading stuff from PyPI with EOL Python versions -- a hypothethical situation that we could end up with in a decade if we just followed our own "usage" rule with just download counts blindly, without looking at the actual usage pattern. As of today, Python 3.7's usage trends are not trending down, if anything, they seem to be actually trending up -- whatever large institutional users are relying on Python 3.7 and PyPI, they're continuing to do so. 
We came up with the current "rule" written in our documentation and we can change it if it does not fit our needs. It evidently didn't with Python 3.6, and it certainly doesn't with Python 3.7 either. I'll file a separate PR with a proposal for adding language to limit how long we end up supporting things post-EOL. All that said, @hugovk could you update this PR? I think it'd be useful to do that eagerly to have smaller conflicts. :) |
|
Somewhat relatedly, I can no longer fetch the last 30 days of download counts for PyPI, with Python version + pip version + OS information with the monthly free 1 TiB on BigQuery. I can fetch Python version and pip version data but that isn't particularly useful to identify the problem platforms like I've done in the past. Here's a nice stacked chart and a Google Sheet with the raw data. Shoutout to @pavithraes for helping me figure out how to actually create this neat chart! The BigQuery SQL query#standardSQL
SELECT
details.installer.version,
REGEXP_EXTRACT(details.python, r"[0-9]+\.[0-9]+") AS python_version,
COUNT(*) AS num_downloads
FROM
`bigquery-public-data.pypi.file_downloads`
WHERE
DATE(timestamp)
BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 31 DAY)
AND DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)
AND details.installer.name = 'pip'
GROUP BY
details.installer.version,
python_version
ORDER BY
num_downloads DESC,
python_version,
details.installer.version |
This comment was marked as resolved.
This comment was marked as resolved.
@hugovk Is this of pip 23.3.2, or with pip 23.3.2? i.e. is this project.name / project.version or details.installer.name / details.installer.version? |
|
Of pip 23.3.2, with anything. And I think |
|
The CI is passing after merging from |
|
@hugovk thanks for this. I think the time has now come to drop python 3.7 support. This PR looks good to me. Would you mind rebasing and handling https://github.com/pypa/pip/pull/11944/files#r1161726499 ? @pypa/pip-committers shall we update our Python support policy to say we support officialy supported python versions (link?), and may support older versions at the discretion of pip maintainers? |
I'm happy to do that. Our discretion can still be based on download numbers, but we won't have to deal with debates over precise figures if we don't want to 🙂 |
|
Done! I also ran pyupgrade, but let me know if you want to drop that commit. It didn't change much. |
|
The proposed updated Python Support Policy is at #12529 |
| sys.stdout.buffer.write(b"\nThis package prints out UTF-8 stuff like:\n") | ||
| sys.stdout.buffer.write("* return type of ‘main’ is not ‘int’\n".encode()) | ||
| sys.stdout.buffer.write( | ||
| "\nThis package prints out UTF-8 stuff like:\n".encode("utf-8") | ||
| ) | ||
| sys.stdout.buffer.write( | ||
| "* return type of ‘main’ is not ‘int’\n".encode("utf-8") | ||
| ) | ||
| sys.stdout.buffer.write( | ||
| "* Björk Guðmundsdóttir [ˈpjœr̥k ˈkvʏðmʏntsˌtoʊhtɪr]".encode("utf-8") | ||
| "* Björk Guðmundsdóttir [ˈpjœr̥k ˈkvʏðmʏntsˌtoʊhtɪr]".encode() |
There was a problem hiding this comment.
btw, why did one of these lines drop the encode and become a binary string, but the others didn't?
There was a problem hiding this comment.
I assumed because it's pure ASCII so a bytestring is unambiguous? I'm not a huge fan of tools making changes like this (I tend to feel it's a case where I write it a certain way because it expresses what I want it to better that way) but I'm not going to get up tight about it, frankly.
There was a problem hiding this comment.
The argument is the default which makes it superfluous https://docs.python.org/3/library/stdtypes.html#str.encode
There was a problem hiding this comment.
The question wasn't about the argument it was about why the first line uses a bytes literal and the next two lines use regular strings + encode.
@pfmoore had the right idea, in fact using non-ASCII characters are illegal for bytes literals: https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
I do find this ugly though, having 3 lines where they made to be bytes in two different ways. But this isn't my codebase so I'll consider this question answered and stop here.
FYI, a lot of the rules from pyupgrade could be automatically enforced via ruff: https://docs.astral.sh/ruff/rules/#pyupgrade-up. I would be happy to raise this as a seperate PR if Pip maintainers are now comfortable to introduce breaking changes to pre-Python 3.8 within the Pip Otherwise if Pip maintainers are still not comfortable with intentionally breaking Python 3.7 I will point out this commit and all the I have no strong opinion, just unsure what the conclusion was from this discussion. |
|
@notatallshaw my view is that if we drop Python 3.7 support, we can use Python 3.8 constructs without restraint. I leave this open for a little while, in case this would be controversial. |
|
I'm fine with using 3.8 constructs. I'm not sure I'd want to apply pyupgrade-type rules automatically, the way we apply formatting rules - I think they need a manual review, like we did with this. |
Co-authored-by: Paul Moore <p.f.moore@gmail.com>
Fixes #11934.
Or it will be end-of-life on 2023-06-27: https://devguide.python.org/versions/
Which will happen before pip 23.2 is released, sometime before 2023-07-31: https://github.com/pypa/pip/milestone/58