[BUG] Setuptools failure with venv

[RyanSiu1995]

setuptools version

60.0.0

Python version

Python 3.9.5

OS

Debian

Additional environment information

No response

Description

I have an environment with virtualenv installed at ~/venv.
When we try to rerun the setup command,
python -m venv ~/venv.
If setuptools is 60.0.0, it will return the following error.

Error: Command '['/root/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

When I tried to run the command directly, I got the following error.

Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/usr/local/lib/python3.9/runpy.py", line 206, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/usr/local/lib/python3.9/runpy.py", line 130, in _get_module_details
    spec = importlib.util.find_spec(mod_name)
  File "/usr/local/lib/python3.9/importlib/util.py", line 103, in find_spec
    return _find_spec(fullname, parent_path)
  File "<frozen importlib._bootstrap>", line 925, in _find_spec
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 83, in find_spec
    return method()
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 104, in spec_for_pip
    if self.pip_imported_during_build():
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 115, in pip_imported_during_build
    return any(
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 116, in <genexpr>
    frame.f_globals['__file__'].endswith('setup.py')
KeyError: '__file__'
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/usr/local/lib/python3.9/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/usr/local/lib/python3.9/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/usr/local/lib/python3.9/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/venv/bin/python3', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmp1s3i_m4v/setuptools-56.0.0-py3-none-any.whl\', \'/tmp/tmp1s3i_m4v/pip-21.1.1-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmp1s3i_m4v\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.

Expected behavior

No error like setuptools<60

How to Reproduce

python -m venv ~/venv
. ~/venv/bin/activate
pip install -U setuptools
python -m venv ~/venv

Output

Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/usr/local/lib/python3.9/runpy.py", line 206, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/usr/local/lib/python3.9/runpy.py", line 130, in _get_module_details
    spec = importlib.util.find_spec(mod_name)
  File "/usr/local/lib/python3.9/importlib/util.py", line 103, in find_spec
    return _find_spec(fullname, parent_path)
  File "<frozen importlib._bootstrap>", line 925, in _find_spec
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 83, in find_spec
    return method()
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 104, in spec_for_pip
    if self.pip_imported_during_build():
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 115, in pip_imported_during_build
    return any(
  File "/root/venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 116, in <genexpr>
    frame.f_globals['__file__'].endswith('setup.py')
KeyError: '__file__'
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/usr/local/lib/python3.9/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/usr/local/lib/python3.9/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/usr/local/lib/python3.9/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/venv/bin/python3', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmp1s3i_m4v/setuptools-56.0.0-py3-none-any.whl\', \'/tmp/tmp1s3i_m4v/pip-21.1.1-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmp1s3i_m4v\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

It looks to me like the build workaround for the pip workaround is getting tripped up on the python -c command that ensurepip triggers. That is:

• first install of setuptools installs the distutils hack, enabled by default.
• the second attempt to run ensurepip invokes a command with python -c, creating a frame without a __file__ in globals.
• runmodule('pip') triggers an import of pip.
• the distutils hack kicks in and checks to see if this is a setuptools build that happens to import pip, inspecting the call stack.
• while inspecting the call stack, the code assumes every frame has a __file__, but that's not true when the top frame is created by python -c.

[jaraco]

Released as 60.0.3. Please test and report back if the issue persists.

[ogrisel]

Released as 60.0.3. Please test and report back if the issue persists.

I confirm 60.0.3 fixed the KeyError we observed on our CI. Thanks.

[RyanSiu1995]

Confirmed the fix working on our CI system. Thank you for the prompt action.