gh-106560: Fix redundant declarations in Include/ (#112611)
Don't declare PyBool_Type and PyLong_Type twice, but only once.
Compiler warnings seen by building Python with gcc -Wredundant-decls.
This should have been done in gh-104148.
(A similar fix has already be done for that slot's value macros, and backported to 3.12. See gh-110968.)
(cherry picked from commit 836e0a75d5)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Fixes GH-109894
* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init
---------
(cherry picked from commit 47d3e2ed93)
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
We do the following:
* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()
The per-interpreter registry helps preserve isolation between interpreters. This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.
(cherry-picked from commit 80dc39e1dc)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.
We add PyInterpreterState.threads.main, with some internal C-API functions.
(cherry-picked from commit 1dd9dee45d)
We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.
Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
(cherry-picked from commit b72947a8d2)
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all.
This change does not address the broader concerns from gh-109853.
(cherry-picked from commit a040a32ea2)
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.
thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.
Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
(cherry picked from commit 517cd82ea7)
* gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808)
(cherry picked from commit 1b8f2366b3)
---------
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
gh-104690: thread_run() checks for tstate dangling pointer (#109056)
thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.
Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
(cherry picked from commit f63d37877a)
gh-109496: Detect Py_DECREF() after dealloc in debug mode (GH-109539)
On a Python built in debug mode, Py_DECREF() now calls
_Py_NegativeRefcount() if the object is a dangling pointer to
deallocated memory: memory filled with 0xDD "dead byte" by the debug
hook on memory allocators. The fix is to check the reference count
*before* checking for _Py_IsImmortal().
Add test_decref_freed_object() to test_capi.test_misc.
(cherry picked from commit 0bb0d88e2d)
Co-authored-by: Victor Stinner <vstinner@python.org>
* GH-108390: Prevent non-local events being set with `sys.monitoring.set_local_events()` (GH-108420)
* Restore generated objects
* Restore size of monitoring arrays in code object for 3.12 ABI compatibility.
* Update ABI file
* gh-91051: fix segfault when using all 8 type watchers (GH-107853)
(cherry picked from commit 66e4edd734)
Co-authored-by: Carl Meyer <carl@oddbird.net>
GH-107724: Fix the signature of `PY_THROW` callback functions. (GH-107725)
(cherry picked from commit 52fbcf61b5)
Co-authored-by: Mark Shannon <mark@hotpy.org>
GH-107263: Increase C stack limit for most functions, except `_PyEval_EvalFrameDefault()` (GH-107535)
* Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
(cherry picked from commit fa45958450)
Co-authored-by: Mark Shannon <mark@hotpy.org>
gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
(cherry picked from commit 58ef741867)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL. It turns out it's tricky to use the same thread state for different threads. Who could have guessed?
We solve the problem by eliminating the one object we were still sharing between interpreters. We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.
We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
(cherry picked from commit 8ba4df91ae)
The _xxsubinterpreters module was meant to only use public API. Some internal C-API usage snuck in over the last few years (e.g. gh-28969). This fixes that.
(cherry picked from commit e6373c0d8b)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-105059: Fix MSCV compiler warning on PyObject union (GH-107239)
Use pragma to ignore the MSCV compiler warning on the PyObject
nameless union.
(cherry picked from commit 1c8fe9bdb6)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-105059: Use GCC/clang extension for PyObject union (GH-107232)
Anonymous union is new in C11. To prevent compiler warning
when using -pedantic compiler option, use Clang and GCC
extension on C99 and older.
(cherry picked from commit 6261585d63)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-98608: Move PyInterpreterConfig to pylifecycle.h (GH-107191)
Move PyInterpreterConfig structure and associated macros from
initconfig.h to pylifecycle.h: it's not related to the Python
Initialization Configuration.
(cherry picked from commit e717b47ed8)
Co-authored-by: Victor Stinner <vstinner@python.org>
GH-96803: Move PyUnstable_InterpreterFrame_GetCode() to Python.h (GH-107188)
Declare the following 3 PyUnstable functions in
Include/cpython/pyframe.h rather than Include/cpython/frameobject.h,
so they are now provided by the standard "GH-include <Python.h>".
(cherry picked from commit 837fa5c0cd)
Co-authored-by: Victor Stinner <vstinner@python.org>
GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069)
Rename private C API constants:
* Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS
* Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS
(cherry picked from commit 0927a2b25c)
gh-105227: Add PyType_GetDict() (GH-105747)
This compensates for static builtin types having `tp_dict` set to `NULL`.
(cherry picked from commit a840806d33)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-105340: include hidden fast-locals in locals() (GH-105715)
* gh-105340: include hidden fast-locals in locals()
(cherry picked from commit 104d7b760f)
Co-authored-by: Carl Meyer <carl@oddbird.net>
