[3.13] gh-142048: Fix lost gc allocations count on thread cleanup (GH-142233)
(cherry picked from commit 49b1fb43f6)
Co-authored-by: Kevin Wang <kevmo314@gmail.com>
The GC for the free threaded build would get slower with each collection due
to effectively double counting objects freed by the GC.
(cherry picked from commit eb892868b3)
Co-authored-by: Kevin Wang <kevmo314@gmail.com>
If we overflowed the global version counter (i.e., after 2*24 calls to
`_PyMonitoring_SetEvents`), we bailed out after setting global monitoring
events but before instrumenting code objects, which led to assertion errors
later on.
Also add a `time.sleep()` to `test_free_threading.test_monitoring` to avoid
overflowing the global version counter.
(cherry picked from commit e457d60daa)
gh-140530: fix a reference leak in an error path for `raise exc from cause` (#140908)
Fix a reference leak in `raise E from T` when `T` is an exception
subtype for which `T.__new__` does not return an exception instance.
(cherry picked from commit 0c77e7c23b)
gh-140815: Fix faulthandler for invalid/freed frame (#140921)
faulthandler now detects if a frame or a code object is invalid or
freed.
Add helper functions:
* _PyCode_SafeAddr2Line()
* _PyFrame_SafeGetCode()
* _PyFrame_SafeGetLasti()
_PyMem_IsPtrFreed() now detects pointers in [-0xff, 0xff] range
as freed.
(cherry picked from commit a84181c31b)
Revert GH-131993.
Fix swallowing some syntax warnings in different modules if they accidentally
have the same message and are emitted from the same line.
Fix duplicated warnings in the "finally" block.
(cherry picked from commit 279db6bede)
There were a few thread-safety issues when profiling or tracing all
threads via PyEval_SetProfileAllThreads or PyEval_SetTraceAllThreads:
* The loop over thread states could crash if a thread exits concurrently
(in both the free threading and default build)
* The modification of `c_profilefunc` and `c_tracefunc` wasn't
thread-safe on the free threading build.
(cherry picked from commit a10152f8fd)
* [3.13] gh-139748: fix leaks in AC error paths when using unicode FS-based converters (GH-139765)
(cherry picked from commit b04a57deef)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
fix comment reference from man 7 signal to man 7 signal-safety (GH-138554)
(cherry picked from commit 2f5ace780b)
docs: fix comment reference from man 7 signal to man 7 signal-safety
Co-authored-by: mqudah <mohghq@gmail.com>
* [3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805)
Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.
(cherry picked from commit 8cc5aa47ee)
Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* state "3.13.7 and earlier"
* backport: do not add the deprecated marker
* fix Py_IsFinalizing doc ref
---------
Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
The `PyEval_SetProfileAllThreads` function and other related functions
had a race condition on `tstate->c_profilefunc` that could lead to a
crash when disable profiling or tracing on all threads while another
thread is starting to profile or trace a a call.
There are still potential crashes when threads exit concurrently with
profiling or tracing be enabled/disabled across all threads.
(cherry picked from commit 362692852f)
gh-137440: Update comment in Python/hamt.c on importing for testing (GH-137441)
Switch from `_testcapi` to `_testinternalcapi`.
(cherry picked from commit 7ab68cd506)
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
gh-58124: Avoid CP_UTF8 in UnicodeDecodeError (GH-137415)
Fix name of the Python encoding in Unicode errors of the code page
codec: use "cp65000" and "cp65001" instead of "CP_UTF7" and "CP_UTF8"
which are not valid Python code names.
(cherry picked from commit ce1b747ff6)
Co-authored-by: Victor Stinner <vstinner@python.org>
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.
Take into account the size of the memory block when deciding when to
process QSBR requests.
Also track the amount of memory being held by QSBR for mimalloc pages.
Advance the write sequence if this memory exceeds a limit. Advancing
the sequence will allow it to be freed more quickly.
Process the held QSBR items from the "eval breaker", rather than from
`_PyMem_FreeDelayed()`. This gives a higher chance that the global read
sequence has advanced enough so that items can be freed.
(cherry picked from commit 113de8545f)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-134411: assert `PyLong_FromLong(x) != NULL` when `x` is known to be small (GH-134415)
Since `PyLong_From Long(PY_MONITORING_DEBUGGER_ID)` falls to `small_int` case and can't return `NULL`. Added `assert`s for extra confidence.
https://github.com/python/cpython/issues/134411#issuecomment-2897653868
(cherry picked from commit cf19b6435d)
Co-authored-by: Sergey Muraviov <smurav@mail.ru>
This commit fixes the following problems:
* The x86_64 trampolines are not preserving frame pointers
* The hardcoded offsets to the code segment from the FDE only worked properly for x64_64
* The CIE data was not following conventions of aarch64
* The eh_frame for aarch64 was not fully correct
(cherry picked from commit 236f733d8f)
The free threading build could spin unnecessarily on `_Py_yield()` if the initial
compare and swap failed.
(cherry picked from commit cbfaf41caf)
Co-authored-by: Joseph Tibbertsma <josephtibbertsma@gmail.com>
Apply Intel Control-flow Technology for x86-64 on asm_trampoline.S.
Required for mitigation against return-oriented programming (ROP)
and Call or Jump Oriented Programming (COP/JOP) attacks.
Manual application is required for the assembly files.
See also: https://sourceware.org/annobin/annobin.html/Test-cf-protection.html
[3.14] gh-128605: Revert "Add branch protections for x86_64 in asm_tr…ampoline.S (GH-128606) (GH-135077)" (GH-135175)
This reverts commit 899cca6dbf,
which broke buildbots.
(cherry picked from commit b477e21d5c)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
[3.14] gh-128605: Add branch protections for x86_64 in asm_trampoline.S (GH-128606) (GH-135077)
Apply Intel Control-flow Technology for x86-64 on asm_trampoline.S.
Required for mitigation against return-oriented programming (ROP)
and Call or Jump Oriented Programming (COP/JOP) attacks.
Manual application is required for the assembly files.
See also: https://sourceware.org/annobin/annobin.html/Test-cf-protection.html
(cherry picked from commit 899cca6dbf)
Co-authored-by: stratakis <cstratak@redhat.com>
gh-130070: Fix `exec(<string>, closure=<non-None>)` unexpected path (#130071)
Fixed an assertion error (so, it could be reproduced only in builds with assertions enabled)
for `exec` when the `source` argument is a string and the `closure` argument is not `None`.
Co-authored-by: sobolevn <mail@sobolevn.me>
(cherry picked from commit 954b2cf031)
(cherry picked from commit ac3c439cdf)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Mark Shannon <mark@hotpy.org>
