Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport maintenance/2.15.x] Avoid hanging forever after a parallel job was killed #7930

Merged

Commits on Dec 12, 2022

  1. Avoid hanging forever after a parallel job was killed (#7834)

    * Replace multiprocessing.pool with concurrent.futures.ProcessPoolExecutor to avoid deadlocks.
    
    In a multiprocessing.pool, if a process terminates in a non-clean fashion
    (for example, due to OOM or a segmentation fault), the pool will silently
    replace said process, but the work that the process was supposed to do
    will never be done, causing pylint to hang indefinitely.
    The concurrent.futures.ProcessPoolExecutor will raise a
    BrokenProcessPool exception in that case, avoiding the hang.
    
    Co-authored-by: Daniël van Noord <13665637+DanielNoord@users.noreply.github.com>
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    (cherry picked from commit 5eca8ec)
    daniel-wer authored and github-actions[bot] committed Dec 12, 2022
    Copy the full SHA
    defd3a4 View commit details
    Browse the repository at this point in the history