bpo-40747: Make py_version_nodot 3_10 not 310 #20333

mattip · 2020-05-23T20:56:51Z

Fixes pypa/packaging#308, pypa/wheel#354, and maybe pypa/pip#8312.

The current value of 310 is ambiguous so use 3_10 instead. While this may fix pip and wheel.bdist_wheel, it has the potential to break other projects that depend on _PY_VERSION_SHORT_NO_DOT being 2 characters or that calculate the value via '%d%d' % sys.version_info[:2] themselves.

https://bugs.python.org/issue40747

pablogsal · 2020-05-23T23:50:50Z

Hmmm, this change may have some other unforeseen consequences. For instance, IIRC this affects the path (indeed, there is a test failing for Windows). Someone more familiar with packaging should take a look as well to make sure this does not have some other unwanted consequences. CC: @brettcannon

ned-deily · 2020-05-24T01:08:52Z

Should probably have @zooba take a look as well. I don't see where the short no dot variable from either copy of sysconfig is used in Unix-y builds of cpython. I've done a quick macOS framework style build (--enable-framework) and I haven't noticed any obvious change in paths or any test suite failures. I'd be inclined to not make the change though unless it is likely to cause problems for downstream packages that do use the variable.

But is it really ambiguous? cpython versions are always of the form 3.x, never 3.x.y, as far as constructing path names in builds, AFAIK. That is, for unix-y builds using autoconf. @zooba could speak for the Windows build system.

And I'm assuming by the time we get to Python 9 there will be bigger issues :)

brettcannon · 2020-05-25T01:57:03Z

It's ambiguous from the perspective of what is "310": 3.10 or 31.0? While we don't exactly plan on changing our numbering scheme right now, this is an ambiguity. Plus it just looks odd to me, which does somewhat matter as this will show up in wheel file names.

mattip · 2020-05-25T09:02:45Z

The FAQ for PEP 425 has this:

Why isn't there a . in the Python version number?
CPython has lasted 20+ years without a 3-digit major release. This should continue for some time. Other implementations may use _ as a delimeter, since both - and . delimit the surrounding filename.

I think "major release" in this context means 3.4, 3.5, 3.6, ... so we are about to hit a 3-digit major release.

mattip · 2020-05-26T06:28:42Z

Tests pass, including one new test for consistency.

ned-deily · 2020-05-27T09:47:40Z

configure.ac

 AC_MSG_RESULT($SOABI)

 # Release and debug (Py_DEBUG) ABI are compatible, but not Py_TRACE_REFS ABI
 if test "$Py_DEBUG" = 'true' -a "$with_trace_refs" != "yes"; then
  # Similar to SOABI but remove "d" flag from ABIFLAGS
  AC_SUBST(ALT_SOABI)
-  ALT_SOABI='cpython-'`echo $VERSION | tr -d .``echo $ABIFLAGS | tr -d d`${PLATFORM_TRIPLET:+-$PLATFORM_TRIPLET}
+  ALT_SOABI='cpython-'`echo $VERSION | tr -d "." "_"``echo $ABIFLAGS | tr -d d`${PLATFORM_TRIPLET:+-$PLATFORM_TRIPLET}


On macOS, at least,

$ ./configure --with-pydebug [...] checking SOABI... cpython-3_10d-darwin usage: tr [-Ccsu] string1 string2 tr [-Ccu] -d string1 tr [-Ccu] -s string1 tr [-Ccu] -ds string1 string2 checking LDVERSION... $(VERSION)$(ABIFLAGS) [...]

Huh. Strange it passes CI. Is the change on line 4740 also problematic? I cannot reproduce, could you try removing the " so that the "." "_" becomes . _

The tr error doesn't cause the configure run to fail. It happens on the Linux CI runs, too: from the Github Ubuntu test:

676 checking SOABI... cpython-3_10d-x86_64-linux-gnu 677 tr: extra operand '_' 678 Only one string may be given when deleting without squeezing repeats. 679 Try 'tr --help' for more information. 680 checking LDVERSION... $(VERSION)$(ABIFLAGS)

(You don't see it in the log for the most recent Travis CI run because it appears that the build environment for the PR is cached and it didn't rerun configure.)

fixed by removing the -d from the tr command. Checked it by rerunning autoconf to recreate configure from configure.am.

(FWIW the error makes sense: tr is either original replacement or -d original to delete)

bedevere-bot · 2020-05-27T09:48:07Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

mattip · 2020-05-28T05:43:00Z

I have made the requested changes; please review again

bedevere-bot · 2020-05-28T05:43:06Z

Thanks for making the requested changes!

@ned-deily: please review the changes made to this pull request.

mattip · 2020-10-13T05:48:29Z

Alternatively, we don't really need to do this, since we're not even planning to get to Python 4.x, let alone 31.x ;) Seems like plenty of time to come up with alternative file layouts.

I am fine with closing this, but then pypa/packaging should align itself with that approach.

mattip · 2020-10-13T06:35:26Z

I searched Doc, Tools/msi, and PC/layout for nodot and (case insensitive) major and added fixes. The only change in Doc was the outdated directions for linking in windows, which did not mention the linking pragma in pyconfig.h.

mattip · 2020-10-13T07:32:02Z

somewhat surprisingly, CI passed with the MSI changes

A python 3.10 wheel would produce a wheel with the platform tag as `cp310`, which would be invalid, as only 3_10 is accepted by the wheel package. See: pypa/packaging#308 pypa/wheel#354 python/cpython#20333

pablogsal · 2020-10-13T19:41:07Z

Unfortunately, this issue is also blocking running the pyperformance suite with master. @zooba, as you requested changes, how would you like to proceed?

brettcannon · 2020-10-13T19:45:34Z

We should aim to resolve this by the end of the core dev sprint so we can unblock the rest of the ecosystem.

hugovk · 2020-10-13T19:54:44Z

We should aim to resolve this by the end of the core dev sprint so we can unblock the rest of the ecosystem.

For those of us who didn't know, it's next week:

The sprint is scheduled for Oct 19 - 23, 2020. This will be an online event instead of the in-person sprint.

Sounds good! Should it be added to https://python-core-sprint-2020.readthedocs.io/projects.html?

brettcannon · 2020-10-13T23:30:57Z

@hugovk nah, it will just be something that some of us go and discuss on the side.

brettcannon · 2020-10-14T00:09:17Z

@hugovk I decided not to risk forgetting, so I went ahead and added it to the TODO list. Thanks for the idea! (Currently the build has not happened yet, so don't be shocked if you don't see it at the bottom.)

brandtbucher · 2020-10-14T03:56:35Z

@pablogsal downgrading to pyperformance==1.0.0 is a temporary workaround.

pablogsal · 2020-10-14T09:58:45Z

@pablogsal downgrading to pyperformance==1.0.0 is a temporary workaround.

Thanks @brandtbucher ! Unfortunately, we were working on the benchmarks for the speed.python.org server and downgrading uses a different versions of the packages being tested and therefore the generated data cannot be compared with the previous data 🙁

brettcannon · 2020-10-15T17:56:22Z

For fellow core devs: feel free to add/remove yourself from https://python-core-sprint-2020.readthedocs.io/projects.html#wheel-interpreter-naming-for-3-10 and I will try to schedule a meeting when we can chat about this. I will wait until Monday to schedule based on who does (not) want to attend to make it timezone friendly (e.g. Pablo I know is London and I'm Vancouver).

pablogsal · 2020-10-15T18:03:25Z

I will wait until Monday to schedule based on who does (not) want to attend to make it timezone friendly (e.g. Pablo I know is London and I'm Vancouver).

My sleep schedule is already destroyed and I am a night person, so feel free to push the meeting closer to your timezone if needed 😉

zooba · 2020-10-20T11:59:17Z

Tools/msi/doc/doc.wixproj

@@ -10,7 +10,7 @@
    </PropertyGroup>
    <Import Project="..\msi.props" />
    <PropertyGroup>
-        <DocFilename>python$(MajorVersionNumber)$(MinorVersionNumber)$(MicroVersionNumber)$(ReleaseLevelName).chm</DocFilename>
+        <DocFilename>python$(MajorVersionNumber)_$(MinorVersionNumber)$(MicroVersionNumber)$(ReleaseLevelName).chm</DocFilename>


We should probably separate the minor and micro version of the doc filename too - python3_10_0.chm rather than python3_100.chm. The actual change ought to be somewhere under Doc, IIRC.

ned-deily · 2020-10-20T17:36:50Z

FYI, I've built and run with the PR merged up to the current HEAD of master with both a vanilla installed Linux config and my standard macOS installer build. In both cases, no test failures due to the PR were noted. In both cases, I also ran python3.10 -m test.pythoninfo and searched the output for the substring 310. I found two instances of that:

the zipfile path (which @zooba noted above). On Linux with an install prefix of /tmp/b:
sys.path: ['/tmp/a', '/tmp/b/lib/python310.zip', '/tmp/b/lib/python3.10', '/tmp/b/lib/python3.10/lib-dynload', '/home/nad/.local/lib/python3.10/site-packages', '/tmp/b/lib/python3.10/site-packages']
the cache_tag attribute of sys.implementation:
sys.implementation: namespace(name='cpython', cache_tag='cpython-310', version=sys.version_info(major=3, minor=10, micro=0, releaselevel='alpha', serial=1), hexversion=50987169, _multiarch='i386-linux-gnu')

I have no opinion ATM of the importance of either one, just reporting them.

brettcannon · 2020-10-20T18:50:13Z

@mattip we discussed this at the sprint and decided that once Steve's concerns are addressed and I write a quick PEP this week on the chnage we can merge the PR.

zooba · 2020-10-20T19:04:35Z

The docs filename change seems to be at line 108 of https://github.com/mattip/cpython/blob/py_version_nodot/Doc/conf.py

Turns out the MSI build in PR is using the "friendly" option that doesn't fail when the doc file is missing, but this line shows that it wasn't found.

I think we should be fine to just allow the extra dots in the .chm name though, rather than going to underscores. And actually, I think we should try going to dots in the DLL name as well. Windows should be able to handle it, and it's more consistent between libpython3.10.so and python3.10.dll than having the underscore in there.

And if we're going to cause (a few) people to have to update their code anyway, we may as well end up somewhere more consistent :)

mattip · 2020-10-20T19:29:03Z

I'm not sure I understand what still needs to be changed. Could a reviewer who does understand push changes to this PR?

zooba · 2020-10-20T19:32:20Z

I can push changes, just didn't want to intrude since you've been active.

mattip · 2020-10-20T20:47:26Z

@zooba or anyone else, please feel free to make any necessary changes to move this forward

zooba · 2020-10-21T16:26:05Z

I've had to reopen as #22858 with a branch on my own fork, since I can only edit mattip's through the web UI (which, as you'll see, was not going to be worthwhile).

Make py_version_nodot 3_10 not 310

fd5a12d

the-knights-who-say-ni added the CLA signed label May 23, 2020

bedevere-bot added the awaiting review label May 23, 2020

mattip mentioned this pull request May 23, 2020

Python 3.10: _version_nodot, interpreter_version disagree on how to format it pypa/packaging#308

Closed

zware requested a review from pablogsal May 23, 2020 22:35

fix another place

76a596c

pablogsal requested a review from brettcannon May 23, 2020 23:50

pablogsal requested a review from zooba May 24, 2020 01:52

mattip and others added 4 commits May 25, 2020 12:38

fix more places that do not use py_version_nodot

5eb6a84

📜🤖 Added by blurb_it.

9759ea9

test SOABI and py_version_nodot agree on the python version

b9fe6c2

change test

7149e08

mattip force-pushed the py_version_nodot branch from 8f246cd to 2dd2c34 Compare May 25, 2020 12:10

change test, SOABI has a 'd' after the python version

4f9b869

mattip force-pushed the py_version_nodot branch from 2dd2c34 to 4f9b869 Compare May 25, 2020 22:03

ned-deily requested changes May 27, 2020

View reviewed changes

bedevere-bot removed the awaiting review label May 27, 2020

bedevere-bot added the awaiting changes label May 27, 2020

remove '-d' from tr

48ec1e9

bedevere-bot added awaiting change review and removed awaiting changes labels May 28, 2020

bedevere-bot requested a review from ned-deily May 28, 2020 05:43

pquentin mentioned this pull request Oct 13, 2020

Fails to build wheel for Python 3.10 pypa/wheel#354

Closed

changes from review

b549f09

jonringer mentioned this pull request Oct 13, 2020

[Draft] Make python310Packages usable NixOS/nixpkgs#100389

Closed

10 tasks

mattip mentioned this pull request Oct 15, 2020

Prepare for python 3.10 changing its nodot version boostorg/python#323

Open

zooba reviewed Oct 20, 2020

View reviewed changes

beniwohli mentioned this pull request Oct 27, 2020

add Python 3.10 beta/RC builds to the matrix elastic/apm-agent-python#952

Merged

HacKanCuBa mentioned this pull request Nov 7, 2020

Py3.10 python-poetry/poetry#3187

Closed

3 tasks

cschramm mentioned this pull request Nov 8, 2020

Update python versions blueman-project/blueman#1426

Merged

RhinosF1 mentioned this pull request Dec 4, 2020

CI: Add 3.10-dev to matrix FOSSBots/MirahezeBots#366

Merged

mattip closed this Dec 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-40747: Make py_version_nodot 3_10 not 310 #20333

bpo-40747: Make py_version_nodot 3_10 not 310 #20333

mattip commented May 23, 2020 •

edited by bedevere-bot

pablogsal commented May 23, 2020 •

edited

ned-deily commented May 24, 2020 •

edited

brettcannon commented May 25, 2020

mattip commented May 25, 2020

mattip commented May 26, 2020

ned-deily May 27, 2020

mattip May 27, 2020

ned-deily May 28, 2020 •

edited

mattip May 28, 2020

merwok Oct 20, 2020

bedevere-bot commented May 27, 2020

mattip commented May 28, 2020

bedevere-bot commented May 28, 2020

mattip commented Oct 13, 2020

mattip commented Oct 13, 2020

mattip commented Oct 13, 2020

pablogsal commented Oct 13, 2020

brettcannon commented Oct 13, 2020

hugovk commented Oct 13, 2020

brettcannon commented Oct 13, 2020

brettcannon commented Oct 14, 2020 •

edited

brandtbucher commented Oct 14, 2020

pablogsal commented Oct 14, 2020 •

edited

brettcannon commented Oct 15, 2020

pablogsal commented Oct 15, 2020

zooba Oct 20, 2020

ned-deily commented Oct 20, 2020

brettcannon commented Oct 20, 2020

zooba commented Oct 20, 2020

mattip commented Oct 20, 2020

zooba commented Oct 20, 2020

mattip commented Oct 20, 2020

zooba commented Oct 21, 2020

bpo-40747: Make py_version_nodot 3_10 not 310 #20333

bpo-40747: Make py_version_nodot 3_10 not 310 #20333

Conversation

mattip commented May 23, 2020 • edited by bedevere-bot

pablogsal commented May 23, 2020 • edited

ned-deily commented May 24, 2020 • edited

brettcannon commented May 25, 2020

mattip commented May 25, 2020

mattip commented May 26, 2020

ned-deily May 27, 2020

Choose a reason for hiding this comment

mattip May 27, 2020

Choose a reason for hiding this comment

ned-deily May 28, 2020 • edited

Choose a reason for hiding this comment

mattip May 28, 2020

Choose a reason for hiding this comment

merwok Oct 20, 2020

Choose a reason for hiding this comment

bedevere-bot commented May 27, 2020

mattip commented May 28, 2020

bedevere-bot commented May 28, 2020

mattip commented Oct 13, 2020

mattip commented Oct 13, 2020

mattip commented Oct 13, 2020

pablogsal commented Oct 13, 2020

brettcannon commented Oct 13, 2020

hugovk commented Oct 13, 2020

brettcannon commented Oct 13, 2020

brettcannon commented Oct 14, 2020 • edited

brandtbucher commented Oct 14, 2020

pablogsal commented Oct 14, 2020 • edited

brettcannon commented Oct 15, 2020

pablogsal commented Oct 15, 2020

zooba Oct 20, 2020

Choose a reason for hiding this comment

ned-deily commented Oct 20, 2020

brettcannon commented Oct 20, 2020

zooba commented Oct 20, 2020

mattip commented Oct 20, 2020

zooba commented Oct 20, 2020

mattip commented Oct 20, 2020

zooba commented Oct 21, 2020

mattip commented May 23, 2020 •

edited by bedevere-bot

pablogsal commented May 23, 2020 •

edited

ned-deily commented May 24, 2020 •

edited

ned-deily May 28, 2020 •

edited

brettcannon commented Oct 14, 2020 •

edited

pablogsal commented Oct 14, 2020 •

edited