cpython/Misc/NEWS.d/next/Library
mpage 33da0e844c
gh-114271: Fix race in Thread.join() (#114839)
There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()`
and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution
involving threads A, B, and C:

1. A starts.
2. B joins A, blocking on its `_tstate_lock`.
3. C joins A, blocking on its `_tstate_lock`.
4. A finishes and releases its `_tstate_lock`.
5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped
   out before calling `_stop()`.
6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped
   out before releasing it.
7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held.
   However, C holds it, so the assertion fails.

The race can be reproduced[^3] by inserting sleeps at the appropriate points in
the threading code. To do so, run the `repro_join_race.py` from the linked repo.

There are two main parts to this PR:

1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`.
   The event is set by the runtime prior to the thread being cleared (in the same
   place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the
   event to be set.
2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all
   non-daemon threads to exit. To do so, an `is_daemon` predicate was added to
   `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()`
   now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on
   `_tstate_lock`s.

[^1]: 441affc9e7/Lib/threading.py (L1201)
[^2]: 441affc9e7/Lib/threading.py (L1115)
[^3]: 8194653279

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
2024-03-16 13:56:30 +01:00
..
2024-02-01-03-09-38.gh-issue-114271.raCkt5.rst gh-114271: Fix race in Thread.join() (#114839) 2024-03-16 13:56:30 +01:00
2024-02-26-10-06-50.gh-issue-113308.MbvOFt.rst gh-113308: Remove some internal parts of uuid module (#115934) 2024-03-14 13:01:41 +03:00
2024-03-01-20-23-57.gh-issue-90535.wXm-jC.rst gh-90535: Fix support of interval>1 in logging.TimedRotatingFileHandler (GH-116220) 2024-03-16 12:29:42 +02:00
2024-03-06-18-30-37.gh-issue-116401.3Wcda2.rst gh-116401: Fix blocking os.fwalk() and shutil.rmtree() on opening a named pipe (GH-116421) 2024-03-13 11:40:28 +02:00
2024-03-08-11-31-49.gh-issue-116484.VMAsU7.rst gh-116484: Fix collisions between Checkbutton and ttk.Checkbutton default names (GH-116495) 2024-03-16 13:31:19 +02:00
2024-03-13-15-45-54.gh-issue-63283.OToJnG.rst gh-63283: IDNA prefix should be case insensitive (GH-17726) 2024-03-15 15:38:13 +01:00
2024-03-14-10-01-23.gh-issue-116811._h5iKP.rst gh-116811: Ensure MetadataPathFinder.invalidate_caches is reachable when delegated through PathFinder. (#116812) 2024-03-14 17:59:00 -04:00
2024-03-14-14-01-46.gh-issue-116764.moB3Lc.rst gh-116764: Fix regressions in urllib.parse.parse_qsl() (GH-116801) 2024-03-16 12:36:05 +02:00
2024-03-14-17-24-59.gh-issue-106531.9ehywi.rst gh-106531: Refresh zipfile._path with zipp 3.18. (#116835) 2024-03-14 21:53:50 +00:00
2024-03-14-20-59-28.gh-issue-90095.7UaJ1U.rst gh-90095: Ignore empty lines and comments in .pdbrc (#116834) 2024-03-15 09:36:04 +00:00
README.rst Link to blurb on PyPI in the NEWS.d READMEs. (#3323) 2017-09-05 10:38:05 -07:00

Put news entry `blurb`_ files for the *Library* section in this directory.

.. _blurb: https://pypi.org/project/blurb/