Fail to run tests for peg generator under Python 3.11.4

[ghost]

Bug report

Bug description:

I am running Python 3.11.4 on Ubuntu 22.04.3 and experienced the following test failure when trying to compile Python 3.11.5. Every sub-test for the peg generator failed with the same error status:

ModuleNotFoundError: No module named 'distutils.tests'

I followed the procedure described below:

 ./configure CFLAGS='-DPYMALLOC_DEBUG' --with-pydebug
 make
 make test

I have been unable to attach the log file as it is too big (2000+) lines but it is stored on my system and when I can find a way it can be supplied.

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Some notes:

• distutils.test does not appear to be in the GitHub repository
• Optimized Python builds with configure --enable-optimizations work

[ghost]

I checked and the file distutils.test was not found anywhere on my system

[iritkatriel]

distutils was removed in python 3.12. Could it be that you are running the 3.11 test suite with a 3.12 python?

[smontanaro]

This seems stale, but I ran into it today. (I built and tested 3.11 alongside other versions on a regular basis.) I am just executing ./configure with no command line args. I get the peg failures, like this:

Traceback (most recent call last):
  File "/Users/skip/src/python/py3.11/Lib/test/test_peg_generator/test_c_parser.py", line 225, in test_left_recursion
    self.run_test(grammar_source, test_source)
  File "/Users/skip/src/python/py3.11/Lib/test/test_peg_generator/test_c_parser.py", line 115, in run_test
    self.build_extension(grammar_source)
  File "/Users/skip/src/python/py3.11/Lib/test/test_peg_generator/test_c_parser.py", line 112, in build_extension
    generate_parser_c_extension(grammar, Path('.'), library_dir=self.library_dir)
  File "/Users/skip/src/python/py3.11/Tools/peg_generator/pegen/testutil.py", line 105, in generate_parser_c_extension
    compile_c_extension(
  File "/Users/skip/src/python/py3.11/Tools/peg_generator/pegen/build.py", line 54, in compile_c_extension
    from distutils.tests.support import fixup_build_ext  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils.tests'

This is in my 3.11 repo with no changes. I'm not sure why distutils.tests isn't found. It's available at the interpreter prompt:

% ./python.exe 
Python 3.11.8+ (heads/3.11:d83b4c570c, Mar 14 2024, 18:34:44) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils.tests
<stdin>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

The full failing import also succeeds at the interpreter prompt:

% ./python.exe
Python 3.11.8+ (heads/3.11:d83b4c570c, Mar 14 2024, 18:34:44) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.tests.support import fixup_build_ext
<stdin>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

It's almost as if sys.modules["distutils"] doesn't refer to the (deprecated) module we're expecting.

[zware]

It's almost as if sys.modules["distutils"] doesn't refer to the (deprecated) module we're expecting.

That sounds a lot like setuptools shenanigans.

[smontanaro]

The question would then be what changed with setuptools? Looking at the 3.11 commit log, the last change I see which mentions that package is from July 2022. I backed up to the commit just before that, then ran:

git clean -fdx
./configure
make -j
./python.exe -E  ./Tools/scripts/run_tests.py test_peg_generator

and it still failed. It wouldn't seem to be anything obvious with a recent version of setuptools.

My day-to-day box is a MacBook Pro M1. I tried the same experiment on my old Dell laptop (running up-to-date XUbuntu 22.04). The test succeeded. It's not at all obvious to me why this test might be platform- or architecture-dependent. That I can get it to fail so far back in time (when it has routinely been succeeding for me until the past day or two) suggests maybe a change in the external tool chain is (partly?) to blame.

[lysnikolaou]

Indeed, setuptools is the culprit here. See #91169 and pypa/setuptools#3007 for more info.

Running the tests with a freshly built 3.11 without setuptools succeeds.It's only after running make altinstall that it starts to fail, cause setuptools has been installed.

There's always the work-around of setting the SETUPTOOLS_USE_DISTUTILS=stdlib environment variable:

❯ SETUPTOOLS_USE_DISTUTILS=stdlib ./python.exe -m test test_peg_generator   
Using random seed: 144407516
0:00:00 load avg: 2.25 Run 1 test sequentially
0:00:00 load avg: 2.25 [1/1] test_peg_generator
/Users/lysnikolaou/repos/python/cpython-versions/3.11-dev/Lib/test/support/__init__.py:1687: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils import ccompiler, sysconfig, spawn, errors
/Users/lysnikolaou/repos/python/cpython-versions/3.11-dev/Lib/test/support/__init__.py:1687: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  from distutils import ccompiler, sysconfig, spawn, errors

== Tests result: SUCCESS ==

1 test OK.

Total duration: 13.7 sec
Total tests: run=91
Total test files: run=1/1
Result: SUCCESS

I'm not sure if we want to take any further action here, and if we do, it's probably something that should be fixed on the setuptools side.

[smontanaro]

Ah, that rings a bell! I only recently installed my 3.11 build because I wanted to tox run in the cherry-picker repo. Previously, I'd just built the interpreter and run the test suite without installing...

[smontanaro]

I removed my locally installed Python 3.11 and tried again. The test suite ran without error.