Commit graph

10067 commits

Author SHA1 Message Date
Pablo Galindo Salgado
ac80f2d978 Fix some crashes
```
import types
types.LazyImportType({}, "3", 0)
```

```
def f():
    exec("lazy import json")
f()
```
2025-12-08 01:07:05 +00:00
T. Wouters
a0a28c23b4
Merge branch 'main' into lazy 2025-12-07 18:58:30 -05:00
Bénédikt Tran
5000033162
Fix missing lazy-related symbols for the JIT (#34)
* fix missing `PyLazyImport_CheckExact`

* fix missing lazy-related symbols

* fix missing lazy-related symbols

* `extern` seems enough

* Export _PyImport_LoadLazyImportTstate for JIT
2025-12-08 00:57:22 +01:00
Bénédikt Tran
c4ed3c4119
Various UAFs & cosmetic fixes in Python/{bltinmodule,bytecode,ceval}.c (#33)
* fix UAF in `bltinmodule.c`

* fix UAF in `bytecode.c`

* run `make regen-all`

* various improvements to `ceval.c`

* fix CI
2025-12-08 00:55:48 +01:00
Bénédikt Tran
f96a99c222 fix error path in _imp__set_lazy_attributes_impl 2025-12-07 18:51:17 -05:00
Bénédikt Tran
301481631f fix error path in _PyImport_LazyImportModuleLevelObject 2025-12-07 18:51:17 -05:00
Bénédikt Tran
74f35c42cb fix UAFs in register_lazy_on_parent 2025-12-07 18:51:17 -05:00
Bénédikt Tran
2a514d92a9 fix error path in _PyImport_LoadLazyImportTstate 2025-12-07 18:51:17 -05:00
Bénédikt Tran
093f08b8ab document return value for PyImport_GetLazyImportsFilter 2025-12-07 18:47:26 -05:00
Pablo Galindo Salgado
d6d850df89
gh-138122: Don't sample partial frame chains (#141912) 2025-12-07 15:53:48 +00:00
Pablo Galindo Salgado
572c780aa8
gh-138122: Implement frame caching in RemoteUnwinder to reduce memory reads (#142137)
This PR implements frame caching in the RemoteUnwinder class to significantly reduce memory reads when profiling remote processes with deep call stacks.

When cache_frames=True, the unwinder stores the frame chain from each sample and reuses unchanged portions in subsequent samples. Since most profiling samples capture similar call stacks (especially the parent frames), this optimization avoids repeatedly reading the same frame data from the target process.

The implementation adds a last_profiled_frame field to the thread state that tracks where the previous sample stopped. On the next sample, if the current frame chain reaches this marker, the cached frames from that point onward are reused instead of being re-read from remote memory.

The sampling profiler now enables frame caching by default.
2025-12-06 22:37:34 +00:00
Pablo Galindo Salgado
470b9e4362 TSAN fixes 2025-12-06 19:16:52 +00:00
Pablo Galindo Salgado
b0adc30759 Free threading fixes 2025-12-06 19:11:21 +00:00
Kir Chou
35142b18ae
gh-142168: explicitly initialize stack_array in _PyEval_Vector and _PyEvalFramePushAndInit_Ex (#142192)
Co-authored-by: Kir Chou <note351@hotmail.com>
2025-12-06 19:59:52 +01:00
Pablo Galindo Salgado
80133a59ff Fix error paths and edge cases 2025-12-06 18:56:06 +00:00
Pablo Galindo Salgado
4c3477ba9a moar fixes much refleaks wow 2025-12-06 18:28:28 +00:00
Pablo Galindo Salgado
4cc6905fa6 PeRfoRManCe 2025-12-06 18:23:09 +00:00
Pablo Galindo Salgado
8e4f292575 brrrrrrrrrrrrrrrrrrrrrr 2025-12-06 18:15:07 +00:00
Pablo Galindo Salgado
f70d4df39c Fix smelly 2025-12-06 18:13:41 +00:00
Pablo Galindo Salgado
a39cd25f15 Fix smelly 2025-12-06 18:13:24 +00:00
Pablo Galindo Salgado
70e5ce7425 More jit fixes 2025-12-06 18:08:33 +00:00
Pablo Galindo Salgado
8e1b20a77a JIT fixes 2025-12-06 18:08:09 +00:00
Pablo Galindo Salgado
76846fedf4 Address feedback 2025-12-06 17:52:23 +00:00
Pablo Galindo
ea120fc6e9 Regen stuff 2025-12-06 15:44:07 +00:00
Pablo Galindo
db151a5192 Merge remote-tracking branch 'upstream/main' into lazy 2025-12-06 15:40:02 +00:00
Pablo Galindo
e6cb131a53 Implement more of PEP 810 2025-12-06 15:21:17 +00:00
Serhiy Storchaka
706fdda8b3
gh-141370: Fix undefined behavior when using Py_ABS() (GH-141548)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
2025-12-05 16:24:35 +02:00
Ken Jin
b3bf212898
gh-141976: Check stack bounds in JIT optimizer (GH-142201) 2025-12-04 20:28:08 +00:00
Mark Shannon
6825d5c11d
GH-139757: Fix reference leaks introduced in GH-140800 (GH-142257) 2025-12-04 12:27:15 +00:00
Mark Shannon
62423c9c36
GH-141794: Limit size of generated machine code. (GH-142228)
* Factor out bodies of the largest uops, to reduce jit code size.
* Factor out common assert, also reducing jit code size.
* Limit size of jitted code for a single executor to 1MB.
2025-12-03 17:43:35 +00:00
Victor Stinner
7e5fcae09b
gh-142217: Remove internal _Py_Identifier functions (#142219)
Remove internal functions:

* _PyDict_ContainsId()
* _PyDict_DelItemId()
* _PyDict_GetItemIdWithError()
* _PyDict_SetItemId()
* _PyEval_GetBuiltinId()
* _PyObject_CallMethodIdNoArgs()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_CallMethodIdOneArg()
* _PyObject_VectorcallMethodId()
* _PyUnicode_EqualToASCIIId()

These functions were not exported and so no usable outside CPython.
2025-12-03 14:33:32 +01:00
Kevin Wang
eb892868b3
gh-142048: Fix quadratically increasing GC delays (gh-142051)
The GC for the free threaded build would get slower with each collection due
to effectively double counting objects freed by the GC.
2025-12-01 19:04:47 -05:00
Victor Stinner
d5d9e89dde
gh-116008: Detect freed thread state in faulthandler (#141988)
Add _PyMem_IsULongFreed() function.
2025-11-27 12:35:00 +01:00
Victor Stinner
83d8134c5b
gh-127635: Use flexible array in tracemalloc (#141991)
Replace frames[1] with frames[] in tracemalloc_traceback structure.
2025-11-27 12:32:31 +01:00
Sergey Miryanov
2ea67caf31
GH-141861: Fix TRACE_RECORD if full (GH-141959) 2025-11-26 14:32:30 +00:00
Itamar Oren
27f62eb711
gh-140011: Delete importdl assertion that prevents importing embedded modules from packages (GH-141605) 2025-11-26 14:12:49 +01:00
Pablo Galindo Salgado
d07d3a3c57
gh-138122: Split Modules/_remote_debugging_module.c into multiple files (#141934)
gh-1381228: Split Modules/_remote_debugging_module.c into multiple files
2025-11-25 12:51:24 +00:00
Sergey Miryanov
dc62b62252
GH-141861: Fix invalid memory read in the ENTER_EXECUTOR (GH-141921) 2025-11-24 22:07:45 +00:00
Petr Viktorin
bf66bce4ee
gh-141780: Make PyModule_FromSlotsAndSpec enable GIL if needed (GH-141785) 2025-11-24 13:26:35 +01:00
Sam Gross
e457d60daa
gh-120158: Fix inconsistent monitoring state when setting events too frequently (gh-141845)
If we overflowed the global version counter (i.e., after 2*24 calls to
`_PyMonitoring_SetEvents`), we bailed out after setting global monitoring
events but before instrumenting code objects, which led to assertion errors
later on.

Also add a `time.sleep()` to `test_free_threading.test_monitoring` to avoid
overflowing the global version counter.
2025-11-23 10:07:17 -05:00
Brandt Bucher
227b9d326e
GH-140638: Add a GC "candidates" stat (GH-141814) 2025-11-22 21:59:14 +00:00
Sam Gross
2d50dd242e
gh-137422: Fix race condition in PyImport_AddModuleRef (gh-141822) 2025-11-21 13:30:33 -05:00
Kumar Aditya
49ff8b6cc0
gh-140795: fetch thread state once on fast path for critical sections (#141406) 2025-11-21 19:49:53 +05:30
Brandt Bucher
598d4c64de
GH-140638: Add a GC "duration" stat (GH-141720) 2025-11-19 08:51:39 -08:00
Mark Shannon
c25a070759
GH-139653: Only raise an exception (or fatal error) when the stack pointer is about to overflow the stack. (GH-141711)
Only raises if the stack pointer is both below the limit *and* above the stack base.
This prevents false positives for user-space threads, as the stack pointer will be outside those bounds
if the stack has been swapped.
2025-11-19 10:16:24 +00:00
Shamil
daafacf005
gh-42400: Fix buffer overflow in _Py_wrealpath() for very long paths (#141529)
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-11-18 17:34:58 +01:00
Victor Stinner
600f3feb23
gh-141070: Add PyUnstable_Object_Dump() function (#141072)
* Promote _PyObject_Dump() as a public function.
* Keep _PyObject_Dump() alias to PyUnstable_Object_Dump()
  for backward compatibility.
* Replace _PyObject_Dump() with PyUnstable_Object_Dump().

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-11-18 16:13:13 +00:00
Mark Shannon
b420f6be53
GH-139109: Support switch/case dispatch with the tracing interpreter. (GH-141703) 2025-11-18 13:31:48 +00:00
Stefano Rivera
f6dd9c12a8
GH-139914: Handle stack growth direction on HPPA (GH-140028)
Adapted from a patch for Python 3.14 submitted to the Debian BTS by John
https://bugs.debian.org/1105111#20

Co-authored-by: John David Anglin <dave.anglin@bell.net>
2025-11-17 14:41:22 +01:00
Brandt Bucher
336366fd7c
GH-140643: Add <native> and <GC> frames to the sampling profiler (#141108)
- Introduce a new field in the GC state to store the frame that initiated garbage collection.
- Update RemoteUnwinder to include options for including "<native>" and "<GC>" frames in the stack trace.
- Modify the sampling profiler to accept parameters for controlling the inclusion of native and GC frames.
- Enhance the stack collector to properly format and append these frames during profiling.
- Add tests to verify the correct behavior of the profiler with respect to native and GC frames, including options to exclude them.

Co-authored-by: Pablo Galindo Salgado <pablogsal@gmail.com>
2025-11-17 13:39:00 +00:00