## Summary
- Move the `runtime->initialized = 1` store from before `site.py` import to the end of `init_interp_main()`, so `Py_IsInitialized()` only returns true after initialization has fully completed
- Access `initialized` and `core_initialized` through new inline accessors using acquire/release atomics, to also protect from data race undefined behavior
- `PySys_AddAuditHook()` now uses the accessor, so with the flag move it correctly skips audit hook invocation during all init phases (matching the documented "after runtime initialization" behavior) ... We could argue that running these earlier would be good even if the intent was never explicitly expressed, but that'd be its own issue.
## Motivation
`Py_IsInitialized()` returned 1 while `Py_InitializeEx()` was still running — specifically, before `site.py` had been imported. See https://github.com/PyO3/pyo3/issues/5900 where a second thread could acquire the GIL and start executing Python with an incomplete `sys.path` because `site.py` hadn't finished.
The flag was also a plain `int` with no atomic operations, making concurrent reads a C-standard data race, though unlikely to manifest.
## Regression test:
The added test properly fails on `main` with `ERROR: Py_IsInitialized() was true during site import`.
---
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
the lazy imports PEP initial implementation (3.15 alpha) inadvertently incremented the length of the sys.flags tuple. In a way that did not do anything useful or related to the lazy imports setting (it exposed sys.flags.gil in the tuple). This fixes that to hard code the length to the 3.13 & 3.14 released length of 18 and have our tests and code comments make it clear that we've since stopped making new sys.flags attributes available via sequence index.
Changing the values requires forking and patching, which is intentional. Simply rebuilding from source does not change the implementation enough to justify changing these values - they would still be `cpython` and compatible with existing `.pyc` files. But people who maintain forks are better served by being able to easily override these values in a place that can be forward-ported reliably.
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Allow the --enable-pystats build option to be used with free-threading. The
stats are now stored on a per-interpreter basis, rather than process global.
For free-threaded builds, the stats structure is allocated per-thread and
then periodically merged into the per-interpreter stats structure (on thread
exit or when the reporting function is called). Most of the pystats related
code has be moved into the file Python/pystats.c.
Follow-up refactoring after GH-133143, where we use `_Py_ID` for "big" and "little"
in `abi_info.byteorder`.
This uses `_Py_ID` for `sys.byteorder`, but also `float_repr_style` and a module name.
This makes information about the interpreter ABI more accessible.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
This partially reverts #137047, keeping the tests for GC collectability of the
original class that dataclass adds `__slots__` to.
The reference leaks solved there are instead solved by having the `__dict__` &
`__weakref__` descriptors not tied to (and referencing) their class.
Instead, they're shared between all classes that need them (within
an interpreter).
The `__objclass__` ol the descriptors is set to `object`, since these
descriptors work with *any* object. (The appropriate checks were already
made in the get/set code, so the `__objclass__` check was redundant.)
The repr of these descriptors (and any others whose `__objclass__` is `object`)
now doesn't mention the objclass.
This change required adjustment of introspection code that checks
`__objclass__` to determine an object's “own” (i.e. not inherited) `__dict__`.
Third-party code that does similar introspection of the internals will also
need adjusting.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
There were a few thread-safety issues when profiling or tracing all
threads via PyEval_SetProfileAllThreads or PyEval_SetTraceAllThreads:
* The loop over thread states could crash if a thread exits concurrently
(in both the free threading and default build)
* The modification of `c_profilefunc` and `c_tracefunc` wasn't
thread-safe on the free threading build.
Remove the `__dict__` and `__weakref__` descriptors from the original class when creating a dataclass from it.
An interesting hack, but more localized in scope than gh-135230.
This may be a breaking change if people intentionally keep the original class around
when using `@dataclass(slots=True)`, and then use `__dict__` or `__weakref__` on the
original class.
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Make `warnings.catch_warnings()` use a context variable for holding
the warning filtering state if the `sys.flags.context_aware_warnings`
flag is set to true. This makes using the context manager thread-safe in
multi-threaded programs.
Add the `sys.flags.thread_inherit_context` flag. If true, starting a new
thread with `threading.Thread` will use a copy of the context
from the caller of `Thread.start()`.
Both these flags are set to true by default for the free-threaded build
and false for the default build.
Move the Python implementation of warnings.py into _py_warnings.py.
Make _contextvars a builtin module.
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
The use of PySys_GetObject() and _PySys_GetAttr(), which return a borrowed
reference, has been replaced by using one of the following functions, which
return a strong reference and distinguish a missing attribute from an error:
_PySys_GetOptionalAttr(), _PySys_GetOptionalAttrString(),
_PySys_GetRequiredAttr(), and _PySys_GetRequiredAttrString().
CPython current temporarily changes `PYMEM_DOMAIN_RAW` to the default
allocator during initialization and shutdown. The motivation is to
ensure that core runtime structures are allocated and freed using the
same allocator. However, modifying the current allocator changes global
state and is not thread-safe even with the GIL. Other threads may be
allocating or freeing objects use PYMEM_DOMAIN_RAW; they are not
required to hold the GIL to call PyMem_RawMalloc/PyMem_RawFree.
This adds new internal-only functions like `_PyMem_DefaultRawMalloc`
that aren't affected by calls to `PyMem_SetAllocator()`, so they're
appropriate for Python runtime initialization and finalization. Use
these calls in places where we previously swapped to the default raw
allocator.
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.
Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.
Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
Temporarily ignore warnings about JIT deactivation when perf support is active.
This will be reverted as soon as a way is found to determine at run time whether the interpreter was built with JIT. Currently, this is not possible on Windows.
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
