gh-125451: Fix deadlock in ProcessPoolExecutor shutdown (#125492)

There was a deadlock when `ProcessPoolExecutor` shuts down at the same
time that a queueing thread handles an error processing a task.

Don't use `_shutdown_lock` to protect the `_ThreadWakeup` pipes -- use
an internal lock instead. This fixes the ordering deadlock where the
`ExecutorManagerThread` holds the `_shutdown_lock` and joins the
queueing thread, while the queueing thread is attempting to acquire the
`_shutdown_lock` while closing the `_ThreadWakeup`.
This commit is contained in:
Sam Gross 2024-10-16 11:39:17 -04:00 committed by GitHub
parent d83fcf8371
commit 760872efec
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 23 additions and 32 deletions

View file

@ -253,9 +253,6 @@ def test_cancel_futures_wait_false(self):
class ProcessPoolShutdownTest(ExecutorShutdownTest):
# gh-125451: 'lock' cannot be serialized, the test is broken
# and hangs randomly
@unittest.skipIf(True, "broken test")
def test_processes_terminate(self):
def acquire_lock(lock):
lock.acquire()