Windows: Process pool termination when running Black with Pipenv inside venv

[felix-hilden]

From #2106: I've encountered a peculiar error when installing and running Black.

Environment

• Windows 10
• Python 3.7.2
• CMD with admin privileges
• venv
• pipenv
• black dev install

Specifically, the issue does not appear when using pipenv only (without venv).

Reproducing

First I followed the contributing guide, which suggested using a pre-existing virtualenvironment is fine.

git clone https://github.com/psf/black black-mwe
cd black-mwe

python -m venv venv
"./venv/Scripts/activate.bat"

pip install pipenv
pipenv install --dev
pipenv shell

pre-commit install
pip install -r test_requirements.txt
pip install -e .[d]

Then both of these commands fail:

pre-commit run -a
black .

Pre-commit fails with:

this output

- hook id: black
- exit code: 123

Process SpawnProcess-1:
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap
    self.run()
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226, in _process_worker
    call_item = call_queue.get(block=True)
  File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in get
    with self._rlock:
  File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
PermissionError: [WinError 5] Access is denied

leading to a bunch of error: cannot format file.py: A process in the process pool was terminated abruptly while the future was running or pending. Other pre-commit tasks are run successfully.

And running Black simply leaves the process hanging. Interrupting it produces a similar error message for 4 processes (= core count?), with an additional message "Future exception was never retrieved".

---

I'm a bit over my head with this, since I've never used pipenv, and only used venv at the very surface level. Knowing the setup works without venv is completely acceptable to me, but I thought this could be worth addressing at least on the documentation level if nobody has the time to install Windows and start digging around, since it's explicitly mentioned in the contributing guide that existing virtualenvs are ok 😄

[felix-hilden]

I also managed to pull through with my WSL running Debian 10 and it had no such issues, so my guess is that this is specific to Windows.

[felix-hilden]

Well we ain't using pipenv anymore 😄