The PyObject header reference count fields must be initialized using
atomic operations because they may be concurrently read by another
thread (e.g., from `_Py_TryIncref`).
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Brandt Bucher <brandt@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
The use of memmove and _Py_memory_repeat were not thread-safe in the
free threading build in some cases. In theory, memmove and
_Py_memory_repeat can copy byte-by-byte instead of pointer-by-pointer,
so concurrent readers could see uninitialized data or tearing.
Additionally, we should be using "release" (or stronger) ordering to be
compliant with the C11 memory model when copying objects within a list.
This makes generator frame state transitions atomic in the free
threading build, which avoids segfaults when trying to execute
a generator from multiple threads concurrently.
There are still a few operations that aren't thread-safe and may crash
if performed concurrently on the same generator/coroutine:
* Accessing gi_yieldfrom/cr_await/ag_await
* Accessing gi_frame/cr_frame/ag_frame
* Async generator operations
Now that we specialize range iteration in the interpreter for the common
case where the iterator has only one reference, there's not a
significant performance cost to making the iteration thread-safe.
JIT: Fix crash due to incorrect caching on side exits when exiting jitted code.
* Make sure that stack is in correct state at side exits with TOS cached values
* Simplify choice of cached items for side exits
* WIP: ENH: Pixi package definitions for downstream development
[skip ci]
* linux-64 support
* tidy gitignore
* respond to review
- switch cases on `PYTHON_VARIANT`
- remove `minor_version` by using `python3`
- remove runtime-only asan options
* README updates
* use `.md` to preview rendering
* Apply suggestions from code review
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Apply suggestion from @FFY00
Co-authored-by: Filipe Laíns <filipe.lains@gmail.com>
* Apply suggestion from @FFY00
Co-authored-by: Filipe Laíns <filipe.lains@gmail.com>
* Apply suggestion from @FFY00
Co-authored-by: Filipe Laíns <filipe.lains@gmail.com>
* Apply suggestion from @lucascolley
---------
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Filipe Laíns <filipe.lains@gmail.com>
* Include LDVERSION and EXE in base_interpreter value
In Fedora, build-details.json created and installed for python3.14t
contains "/usr/bin/python3.14" as the base_interpreter value.
Create a correct string, taking into account both LDVERSION and EXE
config variables, similarly to how it's defined in altbininstall in Makefile.
* Add news
Signed-off-by: Filipe Laíns <lains@riseup.net>
---------
Signed-off-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Filipe Laíns <lains@riseup.net>
JIT: Remove redundant branches to jump in the assembly optimizer
* Refactor JIT assembly optimizer making instructions instances not just strings
* Remove redundant jumps and branches where legal to do so
* Modifies _BINARY_OP_SUBSCR_STR_INT to avoid excessive inlining depth
This PR implements frame caching in the RemoteUnwinder class to significantly reduce memory reads when profiling remote processes with deep call stacks.
When cache_frames=True, the unwinder stores the frame chain from each sample and reuses unchanged portions in subsequent samples. Since most profiling samples capture similar call stacks (especially the parent frames), this optimization avoids repeatedly reading the same frame data from the target process.
The implementation adds a last_profiled_frame field to the thread state that tracks where the previous sample stopped. On the next sample, if the current frame chain reaches this marker, the cached frames from that point onward are reused instead of being re-read from remote memory.
The sampling profiler now enables frame caching by default.
It should make it easier when you need to rebuild just the e.g. host Python, but it requires ./configure to run.
Co-authored-by: Emma Smith <emma@emmatyping.dev>
Loading a small data which does not even involve arbitrary code execution
could consume arbitrary large amount of memory. There were three issues:
* PUT and LONG_BINPUT with large argument (the C implementation only).
Since the memo is implemented in C as a continuous dynamic array, a single
opcode can cause its resizing to arbitrary size. Now the sparsity of
memo indices is limited.
* BINBYTES, BINBYTES8 and BYTEARRAY8 with large argument. They allocated
the bytes or bytearray object of the specified size before reading into
it. Now they read very large data by chunks.
* BINSTRING, BINUNICODE, LONG4, BINUNICODE8 and FRAME with large
argument. They read the whole data by calling the read() method of
the underlying file object, which usually allocates the bytes object of
the specified size before reading into it. Now they read very large data
by chunks.
Also add comprehensive benchmark suite to measure performance and memory
impact of chunked reading optimization in PR #119204.
Features:
- Normal mode: benchmarks legitimate pickles (time/memory metrics)
- Antagonistic mode: tests malicious pickles (DoS protection)
- Baseline comparison: side-by-side comparison of two Python builds
- Support for truncated data and sparse memo attack vectors
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Factor out bodies of the largest uops, to reduce jit code size.
* Factor out common assert, also reducing jit code size.
* Limit size of jitted code for a single executor to 1MB.
This introduces a Wasmtime configuration file to get some CLI options out of the code for easier manipulation. It also allows for easier tweaking after the Makefile is generated.
As well, cut back on the flexibility of specifying HOSTRUNNER for simpler code. The flexibility was never used and so it didn't make sense to keep it around.
This commit updates CI and configuration from wasi-sdk-25 to wasi-sdk-29
which was released recently. This notably includes stubs for pthreads
which all return errors, so some adjustment in logic is necessary to
retain knowledge that WASI cannot yet spawn threads for example.
This additionally increases the wasm stack allowance to 32MiB from 16MiB
to accomodate the `test_recursive_pickle` test in the
`test_functools.py` file. It looks like the Clang/LLVM update that
happened in wasi-sdk-29 relative to wasi-sdk-25 is likely the cause of
this where presumably functions have more locals than before and/or a
slightly adjusted stack space requirement which overflows the stack.
The dataclasses `__init__` function is generated dynamically by a call to `exec()` and so doesn't have deferred reference counting enabled. Enable deferred reference counting on functions when assigned as an attribute to type objects to avoid reference count contention when creating dataclass instances.
