Commit graph

10368 commits

Author SHA1 Message Date
Irit Katriel
c650b51c32
gh-148973: fix segfault on mismatch between consts size and oparg in compiler (#148974) 2026-04-25 10:47:41 +01:00
Gregory P. Smith
665b7dfcfa
Improve hash() builtin docstring with caveats. (GH-125229)
Improve  `hash()` builtin docstring with caveats.

Mention its return type and that the value can be expected to change between
processes (hash randomization).

Why? The `hash` builtin gets reached for and used by a lot of people whether it
is the right tool or not. IDEs surface docstrings and people use pydoc and
`help(hash)`.
2026-04-24 16:36:46 +00:00
Hai Zhu
618b726d68
gh-146073: Add fitness/exit quality mechanism for JIT trace frontend (GH-148089)
* Replaces ad-hoc logic for ending traces with a simple inequality: `fitness < exit_quality`
* Fitness starts high and is reduced for branches, backward edges, calls and trace length
* Exit quality reflect how good a spot that instruction is to end a trace. Closing a loop is very, specializable instructions are very low and the others in between.
2026-04-24 10:37:01 +01:00
Diego Russo
9633c5239d
GH-126910: Build/link the JIT shim in the Python interpreter (#148872) 2026-04-23 12:23:18 +01:00
Sam Gross
ad3c5b7958
gh-148820: Fix _PyRawMutex use-after-free on spurious semaphore wakeup (gh-148852)
_PyRawMutex_UnlockSlow CAS-removes the waiter from the list and then
calls _PySemaphore_Wakeup, with no handshake. If _PySemaphore_Wait
returns Py_PARK_INTR, the waiter can destroy its stack-allocated
semaphore before the unlocker's Wakeup runs, causing a fatal error from
ReleaseSemaphore / sem_post.

Loop in _PyRawMutex_LockSlow until _PySemaphore_Wait returns Py_PARK_OK,
which is only signalled when a matching Wakeup has been observed.

Also include GetLastError() and the handle in the Windows fatal messages
in _PySemaphore_Init, _PySemaphore_Wait, and _PySemaphore_Wakeup to make
similar races easier to diagnose in the future.
2026-04-22 14:31:19 -04:00
Gabriele N. Tornetta
858e69eab0
gh-142186: Allow all PEP-669 events to be per-code object and disableable (GH-146182)
* Make the `PY_UNWIND` monitoring event available as a code-local
event to allow trapping on function exit events when an exception
bubbles up. This complements the PY_RETURN event by allowing to
catch any function exit event.

* Allow `PY_UNWIND`  to be `DISABLE`d; disabling it disables the event for the whole code object.

* Do the above for `PY_THROW`, `RAISE`, `EXCEPTION_HANDLED`, and `RERAISE` events.
2026-04-22 09:08:23 +01:00
AraHaan
789120e826
gh-148790: Eliminate redundant call to _PyRuntime_Initialize in Py_InitializeEx (GH-121628) 2026-04-20 10:01:06 -04:00
Dino Viehland
d81599eeb7
gh-148659: Export a few more functions required for external JITs (#148704)
Export a few more functions required for external JITs
2026-04-18 11:32:22 -07:00
Serhiy Storchaka
2e37d83641
gh-148653: Fix some marshal errors related to recursive immutable objects (GH-148698)
Forbid marshalling recursive code, slice and frozendict objects which
cannot be correctly unmarshalled.
Reject invalid marshal data produced by marshalling recursive frozendict
objects which was previously incorrectly unmarshalled.
Add multiple tests for recursive data structures.
2026-04-18 11:24:33 +03:00
Pieter Eendebak
db3e990b98
gh-146393: Remove special character in optimizer_bytecodes.c (#148693) 2026-04-17 16:52:16 +00:00
Dino Viehland
c0af5c024b
gh-146031: Allow keeping specialization enabled when specifying eval frame function (#146032)
Allow keeping specialization enabled when specifying eval frame function
2026-04-16 09:44:26 -07:00
Mark Shannon
cecf564073
GH-146128: Fix AArch64 multi-instruction constants and relocations (GH-148598)
Fix AArch64 multi-instruction constants and relocations

* Elimates rendundant orr xN, xN, 0xffff after 16 or 32 bit loads
* Merges adrp (21rx) and ldr (12) relocations into single 33rx relocation, when safe to do so.
2026-04-16 15:33:09 +01:00
Mark Shannon
600f4dbd54
GH-145668: Add FOR_ITER specialization for virtual iterators. Specialize GET_ITER. (GH-147967)
* Add FOR_ITER_VIRTUAL to specialize FOR_ITER for virtual iterators
* Add GET_ITER_SELF to specialize GET_ITER for iterators (including generators)
* Add GET_ITER_VIRTUAL to specialize GET_ITER for iterables as virtual iterators
* Add new (internal) _tp_iteritem function slot to PyTypeObject
* Put limited RESUME at start of genexpr for free-threading. Fix up exception handling in genexpr
2026-04-16 15:22:22 +01:00
Pieter Eendebak
1f6a09fb36
gh-100239: Specialize more binary operations using BINARY_OP_EXTEND (GH-128956) 2026-04-16 09:22:41 +01:00
Joshua Root
5f28e6b818
gh-148474: Fix _Py_hexlify_simd compilation with older clang (#148475) 2026-04-16 11:21:23 +05:30
Jelle Zijlstra
5b8cd314e2
gh-137814: Fix __qualname__ of __annotate__ (#137842) 2026-04-15 21:52:30 -07:00
Neko Asakura
e998eb9a83
gh-148604: change ADD_OP(_POP_TOP, ...) to optimize_pop_top in optimizer_bytecodes.c (GH-148619) 2026-04-16 01:08:29 +08:00
Carey Metcalfe
cb339d3c9e
gh-143886: Ensure function annotations are returned in order of definition (#143888)
Ensure function annotations are returned in order of definition

Previously, when getting type annotations of a function, normal
arguments were returned before positional-only ones in the dictionary.
Since `functools.singledispatch` relies on this ordering being correct
to dispatch based on the type of the first argument, this issue was
causing incorrect registrations for functions with positional-only
arguments.

This commit updates how annotations are generated so that
positional-only arguments are generated and added to the dictionary
before normal arguments.
2026-04-15 06:24:28 -07:00
Ken Jin
ab45919812
gh-148609: Remove unicode character in bytecodes.c (GH-148611)
Remove unicode character in bytecodes.c
2026-04-15 12:58:35 +00:00
Wulian233
c7e9919df0
gh-131798: Fix _ITER_CHECK_RANGE type in the JIT (#148607) 2026-04-15 12:22:55 +00:00
Pieter Eendebak
5c3decad66
gh-146393: Use recorded type instead of instance in BINARY_OP (#148569) 2026-04-14 20:11:42 +00:00
Filipe Laíns
0012686d92
GH-145278: freeze encodings (partially) and linecache (#148347) 2026-04-14 21:01:23 +01:00
Pieter Eendebak
95cbd4a232
gh-146393: Optimize float division operations by mutating uniquely-referenced operands in place (JIT only) (GH-146397)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 02:08:04 +08:00
Kumar Aditya
3cb7eaec85
gh-131798: constant fold special method lookups in JIT (#148432) 2026-04-14 16:02:23 +00:00
Kumar Aditya
1aa7e7ee6d
gh-gh-131798: optimize LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN in the JIT (#148555) 2026-04-14 21:00:32 +05:30
Wulian233
e0b56f006c
gh-131798: Constant-fold _CONTAINS_OP_DICT for frozendict (GH-148548) 2026-04-14 23:04:28 +08:00
Neko Asakura
52a7f1b7f8
gh-148510: restore func_version check in _LOAD_ATTR_PROPERTY_FRAME (GH-148528) 2026-04-14 22:44:39 +08:00
Hai Zhu
5ce0fe8b6c
gh-148378: Allow multiple consecutive recording ops per macro op (GH-148496) 2026-04-14 19:26:53 +08:00
Neko Asakura
e02ac1d907
gh-148515: make optimizer_generator respect multiple caches (#148524) 2026-04-14 12:51:05 +08:00
Wulian233
afaf58b573
gh-131798: JIT: Optimize _CHECK_IS_NOT_PY_CALLABLE_EX and _CHECK_IS_NOT_PY_CALLABLE_KW (GH-148494) 2026-04-13 22:41:06 +08:00
Kumar Aditya
88e378cc1c
gh-131798: optimize through keyword and bound method calls in the JIT (GH-148466)
Co-authored-by: Ken Jin <kenjin4096@gmail.com>
2026-04-13 21:14:48 +08:00
Charlie Lin
10d275fdf8
gh-148483: Use Py_GCC_ATTRIBUTE(unused) for stop_tracing label (GH-148481) 2026-04-13 21:05:34 +08:00
Neko Asakura
63492628be
gh-148438: implement _RECORD_BOUND_METHOD in JIT (GH-148457) 2026-04-13 02:57:55 +08:00
Sacul
18d7d90ef9
gh-131798: Split _CHECK_AND_ALLOCATE_OBJECT into smaller uops (GH-148433)
Co-authored-by: Hai Zhu <haiizhu@outlook.com>
Co-authored-by: Ken Jin <kenjin4096@gmail.com>
2026-04-13 02:31:24 +08:00
Ken Jin
6f7bb297db
gh-146261: JIT: protect against function version changes (#146300)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2026-04-13 02:23:47 +08:00
Sam Gross
3ab94d6842
gh-148393: Use atomic ops on _ma_watcher_tag in free threading build (gh-148397)
Fixes data races between dict mutation and watch/unwatch on the same dict.
2026-04-12 10:40:41 -04:00
Wulian233
03d2f035d6
gh-131798: Add _CHECK_IS_NOT_PY_CALLABLE to the JIT optimizer (GH-148434) 2026-04-12 21:07:19 +08:00
Neko Asakura
30c698a655
gh-148398: add type watcher in _CHECK_ATTR_CLASS (GH-148399) 2026-04-12 20:54:03 +08:00
Gregory P. Smith
64afa947f4
gh-146302: make Py_IsInitialized() thread-safe and reflect true init completion (GH-146303)
## Summary

- Move the `runtime->initialized = 1` store from before `site.py` import to the end of `init_interp_main()`, so `Py_IsInitialized()` only returns true after initialization has fully completed
- Access `initialized` and `core_initialized` through new inline accessors using acquire/release atomics, to also protect from data race undefined behavior
- `PySys_AddAuditHook()` now uses the accessor, so with the flag move it correctly skips audit hook invocation during all init phases (matching the documented "after runtime initialization" behavior) ... We could argue that running these earlier would be good even if the intent was never explicitly expressed, but that'd be its own issue.

## Motivation

`Py_IsInitialized()` returned 1 while `Py_InitializeEx()` was still running — specifically, before `site.py` had been imported. See https://github.com/PyO3/pyo3/issues/5900 where a second thread could acquire the GIL and start executing Python with an incomplete `sys.path` because `site.py` hadn't finished.

The flag was also a plain `int` with no atomic operations, making concurrent reads a C-standard data race, though unlikely to manifest.

## Regression test:

The added test properly fails on `main` with `ERROR: Py_IsInitialized() was true during site import`.

---

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 21:54:23 +00:00
Kumar Aditya
ba1e1c192b
gh-131798: do not watch immutable types in JIT (#148383) 2026-04-11 15:43:53 +00:00
Sacul
83f33dccf2
gh-148374: Fix a bug in _Py_uop_sym_get_type (GH-148375) 2026-04-11 23:03:13 +08:00
Wulian233
a059e85866
gh-131798: Add _IS_NONE to the JIT optimizer (GH-148369) 2026-04-11 23:02:46 +08:00
Pieter Eendebak
1c89817f51
gh-148276: Optimize object creation and method calls in the JIT by resolving __init__ at trace optimization time (GH-148277)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ken Jin <kenjin4096@gmail.com>
2026-04-11 22:22:42 +08:00
Donghee Na
a71b043356
gh-148171: Convert CALL_BUILTIN_CLASS to leave arguments on the stack (gh-148381) 2026-04-11 23:01:25 +09:00
Neko Asakura
9831dea3bf
gh-148211: decompose _POP_TWO/_POP_CALL(_ONE/_TWO) in JIT (GH-148377) 2026-04-11 20:46:56 +08:00
Neko Asakura
72006a71b2
gh-148211: decompose [_POP_TWO/_INSERT_2]_LOAD_CONST_INLINE_BORROW in JIT (GH-148357) 2026-04-11 18:27:51 +08:00
Sacul
e872c19922
gh-148171: Convert variadic argument opcodes to leave their arguments on the stack (CALL_BUILTIN_FAST_WITH_KEYWORDS) (#148366) 2026-04-11 13:27:24 +08:00
Kumar Aditya
2b439da972
gh-148171: convert more variadic uops to leave input on stack in JIT (#148361) 2026-04-11 10:29:38 +05:30
Kumar Aditya
8f17140fc1
gh-131798: split _CALL_BUILTIN_CLASS to smaller uops (#148094) 2026-04-10 17:28:20 +00:00
Ken Jin
266247c9a6
gh-148171: Convert CALL_BUILTIN_FAST to leave inputs on the stack for refcount elimination in JIT (GH-148172) 2026-04-10 23:11:18 +08:00