Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix process pool fallback on Python 3.10 #2631

Merged
merged 3 commits into from Nov 20, 2021
Merged

Fix process pool fallback on Python 3.10 #2631

merged 3 commits into from Nov 20, 2021

Conversation

MatthewScholefield
Copy link
Contributor

Description

In Python 3.10 the exception generated by creating a process pool on a Python build that doesn't support this is now NotImplementedError.

Here is the stack trace I received on Python 3.10 on Termux / Android 11:

  File "/data/data/com.termux/files/home/my-project/.venv/lib/python3.10/site-packages/black/__init__.py", line 624, in reformat_many
    executor = ProcessPoolExecutor(max_workers=worker_count)
  File "/data/data/com.termux/files/usr/lib/python3.10/concurrent/futures/process.py", line 594, in __init__
    _check_system_limits()
  File "/data/data/com.termux/files/usr/lib/python3.10/concurrent/futures/process.py", line 542, in _check_system_limits
    raise NotImplementedError(_system_limited)
NotImplementedError: This Python build lacks multiprocessing.synchronize, usually due to named semaphores being unavailable on this platform.

Checklist

  • Add a CHANGELOG entry if necessary?
  • Add / update tests if necessary?
    • We could parameterize the test that injects OSError into ProcessPoolExecutor but at that point we're just repeating code.
  • Add new / update outdated documentation?
    • n/a

In Python 3.10 the exception generated by creating a process pool on a Python build that doesn't support this is now `NotImplementedError`
CHANGES.md Outdated Show resolved Hide resolved
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
@JelleZijlstra
Copy link
Collaborator

The 3.9 fuzz failure is expected. Thanks for the PR! I'll merge it as soon as the rest of CI passes and I'm on my computer.

Copy link
Collaborator

@ichard26 ichard26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'mma beat you to it Jelle :p


Thank you so much for your contribution! This project is only possible by contributions like these 🖤. You're awesome, @MatthewScholefield. I do wonder how much black is really ran on systems without multiprocessing support like android 🤔

@ichard26 ichard26 merged commit 05954c0 into psf:main Nov 20, 2021
@MatthewScholefield MatthewScholefield deleted the patch-1 branch February 2, 2022 03:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants