Windows installations crashing due to unavailability of standard output

[david-cortes]

I’m not entirely sure what’s the problem here, but I have this package in PyPI:
https://pypi.org/project/nonnegcg/

Whenever I try to install it Windows machines through pip or setuptools (e.g. pip install nonnegcg), I get an error at the moment something tries to write to standard output (which is bound to happen during installation), bet it print , warning, or something else, which crashes the installation:

ValueError: underlying buffer has been detached
Exception ignored in: <_io.TextIOWrapper mode='w' encoding='cp1252'>
ValueError: underlying buffer has been detached

Installing it with distutils from the downloaded file does not result in any problems (python setup.py install, modifying setup.py to import setup from distutils rather than from setuptoos), and it install just fine from pip and setup.py + setuptools in linux, but not on windows.

[anntzer]

I can repro a similar problem with the following minimal setup:
pyproject.toml:

[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"

[project]
name = "mod"
version = "42"

with an empty mod.py next to it.

Running pip wheel -v . or pip install -ve . for example succeeds, but prints out large amounts of tracebacks like

  --- Logging error ---
  Traceback (most recent call last):
    File "C:\ProgramData\Miniconda3\envs\myenv\lib\logging\__init__.py", line 1103, in emit
      stream.write(msg + self.terminator)
  ValueError: underlying buffer has been detached
  Call stack:
<elided>
  Message: 'removing build\\bdist.win-amd64\\wheel'
  Arguments: ()

[abravalheri]

Hi @anntzer, I am afraid I cannot reproduce the error in the Windows machine I have available (I am using Windows 10, PowerShell 5.1.19041.1682, Python 3.9.13):

mkdir test-stdout-detached
cd test-stdout-detached
# Write the given pyproject.toml example to a file using UTF-8/unix line ending encodings
echo "" > mod.py
python -m pip wheel .
# ...
# Successfully built mod

# (file mod-42-py3-none-any.whl is created successfully)

This is probably related to a specific setup you have in your environment.

I recommend trying again using an isolated virtual environment (python -m venv, not the conda ones).

[anntzer]

I have repro'd again this with a clean, entirely new, python.org python 3.11 install in Windows 10 Pro 21H2, whether outside of any venv or inside a clean, entirely new venv, whether in cmd.exe or in PowerShell 7.3.

[abravalheri]

I have repro'd again this with a clean, entirely new, python.org python 3.11 install in Windows 10 Pro 21H2, whether outside of any venv or inside a clean, entirely new venv, whether in cmd.exe or in PowerShell 7.3.

I still cannot reproduce it I am afraid 😅
Could you please share a windows docker container with the reproducer?

[anntzer]

Sorry, setting up a Windows docker container is a bit beyond my expertise...

[abravalheri]

No problems @anntzer, thank you anyway for having attempted to propose a reproducer.

I wrote a test using Dockerfile (see bellow). Everything goes well and I cannot observe the faulty behaviour described in this issue.

To keep investigating, it would be necessary that a member of the community that is experiencing this problem provide a minimal reproducer that can be executed in an isolated environment (ideally via a windows container, as shown below).

If there is no reproducer available, I think it would be safe to say that this problem is more likely to be related on how an specific environment is setup (e.g. how Python is being compiled/installed) than to setuptools itself (and therefore we can close this issue).

Successful example

rm -R isolated-test-in-container
mkdir isolated-test-in-container
cd isolated-test-in-container
new-item pyproject.toml
new-item Dockerfile

@"
[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"

[project]
name = "mod"
version = "42"
"@ | add-content -Path pyproject.toml

@"
FROM python:3.11-windowsservercore-1809

RUN mkdir C:/pkg
WORKDIR C:/pkg
COPY pyproject.toml .

RUN python -m pip install -U pip setuptools wheel

CMD pip wheel -v .
"@ | add-content -Path Dockerfile
docker build -t isolated-test .
docker run --rm -it isolated-test

Output of `docker run`

Processing c:\pkg
  Running command pip subprocess to install build dependencies
  Collecting setuptools>=61
    Using cached setuptools-65.5.1-py3-none-any.whl (1.2 MB)
  Installing collected packages: setuptools
  Successfully installed setuptools-65.5.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  running egg_info
  creating mod.egg-info
  writing mod.egg-info\PKG-INFO
  writing dependency_links to mod.egg-info\dependency_links.txt
  writing top-level names to mod.egg-info\top_level.txt
  writing manifest file 'mod.egg-info\SOURCES.txt'
  reading manifest file 'mod.egg-info\SOURCES.txt'
  writing manifest file 'mod.egg-info\SOURCES.txt'
  Getting requirements to build wheel ... done
  Running command pip subprocess to install backend dependencies
  Collecting wheel
    Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
  Installing collected packages: wheel
  Successfully installed wheel-0.38.4
  Installing backend dependencies ... done
  Running command Preparing metadata (pyproject.toml)
  running dist_info
  creating C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info
  writing C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info\PKG-INFO
  writing dependency_links to C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info\dependency_link
s.txt
  writing top-level names to C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info\top_level.txt
  writing manifest file 'C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info\SOURCES.txt'
  reading manifest file 'C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info\SOURCES.txt'
  writing manifest file 'C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod.egg-info\SOURCES.txt'
  creating 'C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-modern-metadata-4yyajyko\mod-42.dist-info'
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: mod
  Running command Building wheel for mod (pyproject.toml)
  running bdist_wheel
  running build
  installing to build\bdist.win-amd64\wheel
  running install
  running install_egg_info
  running egg_info
  writing mod.egg-info\PKG-INFO
  writing dependency_links to mod.egg-info\dependency_links.txt
  writing top-level names to mod.egg-info\top_level.txt
  reading manifest file 'mod.egg-info\SOURCES.txt'
  writing manifest file 'mod.egg-info\SOURCES.txt'
  Copying mod.egg-info to build\bdist.win-amd64\wheel\.\mod-42-py3.11.egg-info
  running install_scripts
  creating build\bdist.win-amd64\wheel\mod-42.dist-info\WHEEL
  creating 'C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-wheel-uk07kl01\tmpm7j7vnk0\mod-42-py3-none-any.whl' and adding 'build\b
dist.win-amd64\wheel' to it
  adding 'mod-42.dist-info/METADATA'
  adding 'mod-42.dist-info/WHEEL'
  adding 'mod-42.dist-info/top_level.txt'
  adding 'mod-42.dist-info/RECORD'
  removing build\bdist.win-amd64\wheel
  Building wheel for mod (pyproject.toml) ... done
  Created wheel for mod: filename=mod-42-py3-none-any.whl size=862 sha256=79a2c8146078715f5f6cf53f0a8fa88330938625d50f530e8711a4f6990da82a

  Stored in directory: C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-ephem-wheel-cache-dlh4ev4r\wheels\7a\c7\0a\dfe41c75aa52ca4e3
17402d3e007a60a5e660ca93de44e656d
Successfully built mod

[abravalheri]

Please also note that I tried to run in a container the original suggestion (pip install nonnegcg - #1631 (comment)) and that fails, but because of compilation problems, not because of underlying buffer has been detached. Instead, Python/pip/setuptools seem to be handling the stdout quite well:

rm -R isolated-test-in-container
mkdir isolated-test-in-container
cd isolated-test-in-container

new-item Dockerfile
@"
FROM python:3.11-windowsservercore-1809

RUN python -m pip install -U pip setuptools wheel numpy findblas Cython mkl-devel
CMD python -m pip install nonnegcg --no-build-isolation --no-use-pep517
"@ | add-content -Path Dockerfile
docker build -t isolated-test .
docker run --rm -it isolated-test

Output of `docker run`

Collecting nonnegcg
  Downloading nonnegcg-0.1.6.tar.gz (30 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in c:\python\lib\site-packages (from nonnegcg) (1.23.4)
Collecting scipy
  Downloading scipy-1.9.3-cp311-cp311-win_amd64.whl (39.9 MB)
     ---------------------------------------- 39.9/39.9 MB 4.9 MB/s eta 0:00:00
Requirement already satisfied: cython in c:\python\lib\site-packages (from nonnegcg) (0.29.32)
Requirement already satisfied: findblas in c:\python\lib\site-packages (from nonnegcg) (0.1.20)
Building wheels for collected packages: nonnegcg
  Building wheel for nonnegcg (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [46 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\nonnegcg
      copying nonnegcg\__init__.py -> build\lib.win-amd64-cpython-311\nonnegcg
      running egg_info
      writing nonnegcg.egg-info\PKG-INFO
      writing dependency_links to nonnegcg.egg-info\dependency_links.txt
      writing requirements to nonnegcg.egg-info\requires.txt
      writing top-level names to nonnegcg.egg-info\top_level.txt
      reading manifest file 'nonnegcg.egg-info\SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE'
      writing manifest file 'nonnegcg.egg-info\SOURCES.txt'
      copying nonnegcg\nonnegcg.pxd -> build\lib.win-amd64-cpython-311\nonnegcg
      copying nonnegcg\pywrapper.pyx -> build\lib.win-amd64-cpython-311\nonnegcg
      running build_ext
      INFO: No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
      C:\Python\Lib\site-packages\numpy\distutils\system_info.py:2077: UserWarning:
          Optimized (vendor) Blas libraries are not found.
          Falls back to netlib Blas library which has worse performance.
          A better performance should be easily gained by switching
          Blas library.
        if self._calc_info(blas):
      C:\Python\Lib\site-packages\numpy\distutils\system_info.py:2077: UserWarning:
          Blas (http://www.netlib.org/blas/) libraries not found.
          Directories to search for the libraries can be specified in the
          numpy/distutils/site.cfg file (section [blas]) or by setting
          the BLAS environment variable.
        if self._calc_info(blas):
      C:\Python\Lib\site-packages\numpy\distutils\system_info.py:2077: UserWarning:
          Blas (http://www.netlib.org/blas/) sources not found.
          Directories to search for the sources can be specified in the
          numpy/distutils/site.cfg file (section [blas_src]) or by setting
          the BLAS_SRC environment variable.
        if self._calc_info(blas):
      WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
      Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
      To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
      cythoning nonnegcg/pywrapper.pyx to nonnegcg\pywrapper.c
      C:\Python\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (
Py2). This will change in a later release! File: C:\Users\ContainerAdministrator\AppData\Local\Temp\pip-install-p4yynn08\nonnegcg_5d76d50f
1d9e4cef988095affeecd76b\nonnegcg\pywrapper.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)
      building 'nonnegcg._minimize' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com
/visual-cpp-build-tools/
      [end of output]
  ERROR: Failed building wheel for nonnegcg
  Running setup.py clean for nonnegcg
Failed to build nonnegcg
Installing collected packages: scipy, nonnegcg
  Running setup.py install for nonnegcg ... error
  error: subprocess-exited-with-error

  × Running setup.py install for nonnegcg did not run successfully.
  │ exit code: 1
  ╰─> [46 lines of output]
      running install
      C:\Python\Lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use buil
d and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\nonnegcg
      copying nonnegcg\__init__.py -> build\lib.win-amd64-cpython-311\nonnegcg
      running egg_info
      writing nonnegcg.egg-info\PKG-INFO
      writing dependency_links to nonnegcg.egg-info\dependency_links.txt
      writing requirements to nonnegcg.egg-info\requires.txt
      writing top-level names to nonnegcg.egg-info\top_level.txt
      reading manifest file 'nonnegcg.egg-info\SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE'
      writing manifest file 'nonnegcg.egg-info\SOURCES.txt'
      copying nonnegcg\nonnegcg.pxd -> build\lib.win-amd64-cpython-311\nonnegcg
      copying nonnegcg\pywrapper.pyx -> build\lib.win-amd64-cpython-311\nonnegcg
      running build_ext
      INFO: No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
      C:\Python\Lib\site-packages\numpy\distutils\system_info.py:2077: UserWarning:
          Optimized (vendor) Blas libraries are not found.
          Falls back to netlib Blas library which has worse performance.
          A better performance should be easily gained by switching
          Blas library.
        if self._calc_info(blas):
      C:\Python\Lib\site-packages\numpy\distutils\system_info.py:2077: UserWarning:
          Blas (http://www.netlib.org/blas/) libraries not found.
          Directories to search for the libraries can be specified in the
          numpy/distutils/site.cfg file (section [blas]) or by setting
          the BLAS environment variable.
        if self._calc_info(blas):
      C:\Python\Lib\site-packages\numpy\distutils\system_info.py:2077: UserWarning:
          Blas (http://www.netlib.org/blas/) sources not found.
          Directories to search for the sources can be specified in the
          numpy/distutils/site.cfg file (section [blas_src]) or by setting
          the BLAS_SRC environment variable.
        if self._calc_info(blas):
      WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
      Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
      To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
      skipping 'nonnegcg\pywrapper.c' Cython extension (up-to-date)
      building 'nonnegcg._minimize' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com
/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> nonnegcg

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[dlaugt]

Hi @abravalheri,

I have the same errors by executing the @anntzer's simple steps:

• Windows 10 version 21H2
• Install python 3.11.0 (which I have never installed before)
• Create the pyproject.toml
• Create the empty mod.py
• From a cmd window, pip wheel -v .

The file mod-42-py3-none-any.whl is generated but there are these ugly error messages in the standard output like:

  --- Logging error ---
  Traceback (most recent call last):
    File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
      stream.write(msg + self.terminator)
  ValueError: underlying buffer has been detached
  Call stack:
    File "C:\dev\Python311\Lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 351, in <module>
      main()
    File "C:\dev\Python311\Lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 333, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\dev\Python311\Lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 249, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\build_meta.py", line 412, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 1, in <module>
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
      return run_commands(dist)
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
      dist.run_commands()
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
      super().run_command(command)
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
      cmd_obj.run()
    File "C:\Users\daniel\AppData\Local\Temp\pip-build-env-pg7a2f7d\normal\Lib\site-packages\wheel\bdist_wheel.py", line 399, in run
      log.info(f"removing {self.bdist_dir}")
  Message: 'removing build\\bdist.win-amd64\\wheel'
  Arguments: ()

It is not linked to a python version as I have the same with python 3.9 and 3.10. I suspect something wrong in the logging of setuptools arround the following line?

setuptools/setuptools/dist.py

Lines 1202 to 1210 in a0e8e53

	sys.stdout = io.TextIOWrapper(
	sys.stdout.detach(), 'utf-8', errors, newline, line_buffering
	)
	try:
	return _Distribution.handle_display_options(self, option_order)
	finally:
	sys.stdout = io.TextIOWrapper(
	sys.stdout.detach(), encoding, errors, newline, line_buffering
	)

[dlaugt]

If it can help, here is the full traces:
traces.log

[dlaugt]

Hi,

To be able to debug the use case, I have changed the steps as follow:

• Windows 10 version 21H2
• Install a fresh python 3.11.0
• pip install setuptools
• pip install wheel
• Create empty mod.py
• Create pyproject.toml
• Create setup.py
• python setup.py bdist_wheel

pyproject.toml:

[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"

[project]
name = "mod"
version = "42"

setup.py:

from setuptools import setup
setup()

Output of `python setup.py bdist_wheel`

running bdist_wheel
running build
running build_py
creating build
creating build\lib
copying mod.py -> build\lib
running egg_info
creating mod.egg-info
writing mod.egg-info\PKG-INFO
writing dependency_links to mod.egg-info\dependency_links.txt
writing top-level names to mod.egg-info\top_level.txt
writing manifest file 'mod.egg-info\SOURCES.txt'
reading manifest file 'mod.egg-info\SOURCES.txt'
writing manifest file 'mod.egg-info\SOURCES.txt'
C:\dev\Python311\Lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 358, in run
    log.info(f"installing to {self.bdist_dir}")
Message: 'installing to build\\bdist.win-amd64\\wheel'
Arguments: ()
running install
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\wheel
copying build\lib\mod.py -> build\bdist.win-amd64\wheel\.
running install_egg_info
Copying mod.egg-info to build\bdist.win-amd64\wheel\.\mod-42-py3.11.egg-info
running install_scripts
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 379, in run
    self.write_wheelfile(distinfo_dir)
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 423, in write_wheelfile
    log.info(f"creating {wheelfile_path}")
Message: 'creating build\\bdist.win-amd64\\wheel\\mod-42.dist-info\\WHEEL'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 387, in run
    wf.write_files(archive_root)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 122, in write_files
    log.info(f"creating '{self.filename}' and adding '{base_dir}' to it")
Message: "creating 'dist\\mod-42-py3-none-any.whl' and adding 'build\\bdist.win-amd64\\wheel' to it"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 387, in run
    wf.write_files(archive_root)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 137, in write_files
    self.write(path, arcname)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 153, in write
    self.writestr(zinfo, data, compress_type)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 165, in writestr
    log.info(f"adding '{fname}'")
Message: "adding 'mod.py'"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 387, in run
    wf.write_files(archive_root)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 141, in write_files
    self.write(path, arcname)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 153, in write
    self.writestr(zinfo, data, compress_type)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 165, in writestr
    log.info(f"adding '{fname}'")
Message: "adding 'mod-42.dist-info/METADATA'"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 387, in run
    wf.write_files(archive_root)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 141, in write_files
    self.write(path, arcname)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 153, in write
    self.writestr(zinfo, data, compress_type)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 165, in writestr
    log.info(f"adding '{fname}'")
Message: "adding 'mod-42.dist-info/WHEEL'"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 387, in run
    wf.write_files(archive_root)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 141, in write_files
    self.write(path, arcname)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 153, in write
    self.writestr(zinfo, data, compress_type)
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 165, in writestr
    log.info(f"adding '{fname}'")
Message: "adding 'mod-42.dist-info/top_level.txt'"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 386, in run
    with WheelFile(wheel_path, "w", self.compression) as wf:
  File "C:\dev\Python311\Lib\zipfile.py", line 1342, in __exit__
    self.close()
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 189, in close
    self.writestr(zinfo, data.getvalue())
  File "C:\dev\Python311\Lib\site-packages\wheel\wheelfile.py", line 165, in writestr
    log.info(f"adding '{fname}'")
Message: "adding 'mod-42.dist-info/RECORD'"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\dev\Python311\Lib\logging\__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
  File "C:\src\python\test\setup.py", line 3, in <module>
    setup()
  File "C:\dev\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\dev\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\dev\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\dev\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 399, in run
    log.info(f"removing {self.bdist_dir}")
Message: 'removing build\\bdist.win-amd64\\wheel'
Arguments: ()

The standard output is fixed if I change the following in setuptools/dist.py:
from:

        sys.stdout = io.TextIOWrapper(
            sys.stdout.detach(), 'utf-8', errors, newline, line_buffering
        )
        try:
            return _Distribution.handle_display_options(self, option_order)
        finally:
            sys.stdout = io.TextIOWrapper(
                sys.stdout.detach(), encoding, errors, newline, line_buffering
            )

to:

        sys.stdout.reconfigure(encoding='utf-8')
        try:
            return _Distribution.handle_display_options(self, option_order)
        finally:
            sys.stdout.reconfigure(encoding=encoding)

Output of `python setup.py bdist_wheel`

running bdist_wheel
running build
running build_py
creating build
creating build\lib
copying mod.py -> build\lib
running egg_info
creating mod.egg-info
writing mod.egg-info\PKG-INFO
writing dependency_links to mod.egg-info\dependency_links.txt
writing top-level names to mod.egg-info\top_level.txt
writing manifest file 'mod.egg-info\SOURCES.txt'
reading manifest file 'mod.egg-info\SOURCES.txt'
writing manifest file 'mod.egg-info\SOURCES.txt'
C:\dev\Python311\Lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
installing to build\bdist.win-amd64\wheel
running install
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\wheel
copying build\lib\mod.py -> build\bdist.win-amd64\wheel\.
running install_egg_info
Copying mod.egg-info to build\bdist.win-amd64\wheel\.\mod-42-py3.11.egg-info
running install_scripts
creating build\bdist.win-amd64\wheel\mod-42.dist-info\WHEEL
creating 'dist\mod-42-py3-none-any.whl' and adding 'build\bdist.win-amd64\wheel' to it
adding 'mod.py'
adding 'mod-42.dist-info/METADATA'
adding 'mod-42.dist-info/WHEEL'
adding 'mod-42.dist-info/top_level.txt'
adding 'mod-42.dist-info/RECORD'
removing build\bdist.win-amd64\wheel

I will use this tweak. However, the function reconfigure() is available only from python 3.7. But that's fine for me as I'm using python 3.10.

[dlaugt]

From pypi website, the latest setuptools requires at least python 3.7. So, perhaps, this kind of tweak can be committed to the repo.

[abravalheri]

Hi @dlaugt, thank you very much for investigating this.

What is puzzling me is that I could not reproduce this in my Windows machine or in an isolated Windows container (I admit that I am probably not using the exact version of Windows you guys are using because of licenses/availability).

Probably there is some information missing to create a reproducer. I wonder which console/terminal program you guys are using and which encoding it is configured to use... Maybe that is the missing piece?

If you have tested the proposed patch and it works for you, would you like to submit a PR?
Ideally we would like to have regression tests associated, but that might be tricky in this situation... Maybe just the fact that it does not fail the existing test suite is enough... (But please feel free to suggest a test suite if you are feeling creative). Once the PR is submitted, I will probably ask for the other maintainers to review.

[dlaugt]

I'm running this from a command prompt window (%windir%\system32\cmd.exe). The encoding is utf-8:

python -c "import sys; print(sys.stdout.encoding)"
utf-8

However, when I launch python setup.py bdist_wheel, it goes to setuptools.Distribution.handle_display_options(). Inside this function, the encoding is cp1252...

I will create a PR.

[dlaugt]

I guess I understood why you cannot reproduce it... In fact, I was redirecting the standard & error output into a file.

From cmd.exe, it worked fine when I run the following:

python -c "import sys; print(sys.stdout.encoding)"
utf-8

python setup.py bdist_wheel

From cmd.exe, it didn't work fine when I run the following:

python -c "import sys; print(sys.stdout.encoding)" > traces.log 2>&1
cp1252

python setup.py bdist_wheel > traces.log 2>&1

However, the command bdist_wheel should work also when you redirect the standard & error output into a file. So, the pull request is fixing the following use case:
python setup.py bdist_wheel > traces.log 2>&1