Fix race condition in test_external_inspection thread status tests
The tests test_thread_status_detection and test_thread_status_gil_detection
had a race condition where the test could sample thread status between when
the sleeper thread sends its "ready" message and when it actually calls
time.sleep(). This caused intermittent test failures where the sleeper
thread would show as running (status=0) instead of idle (status=1 or 2).
The fix moves the thread status collection inside the retry loop and
specifically waits for the expected thread states before proceeding with
assertions. The retry loop now continues until:
- The sleeper thread shows as idle (status=1 for CPU mode, status=2 for GIL mode)
- The busy thread shows as running (status=0)
- Both thread IDs are found in the status collection
This ensures the test waits for threads to settle into their expected states
before making assertions, eliminating the race condition.
Implement a statistical sampling profiler that can profile external
Python processes by PID. Uses the _remote_debugging module and converts
the results to pstats-compatible format for analysis.
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Reorder result tuple of parse_code_object
The standard followed by APIs like pstat.Stats is to take a file, line,
function triplet. The parse_code_object function (and callers exposing
this in Python like RemoteUnwinder.get_stack_trace) return function,
file, line triplets which requires the caller to reorder these when
using it in classes like pstat.Stats.
Completely refactor Modules/_remote_debugging_module.c with improved
code organization, replacing scattered reference counting and error
handling with centralized goto error paths. This cleanup improves
maintainability and reduces code duplication throughout the module while
preserving the same external API.
Implement memory page caching optimization in Python/remote_debug.h to
avoid repeated reads of the same memory regions during debugging
operations. The cache stores previously read memory pages and reuses
them for subsequent reads, significantly reducing system calls and
improving performance.
Add code object caching mechanism with a new code_object_generation
field in the interpreter state that tracks when code object caches need
invalidation. This allows efficient reuse of parsed code object metadata
and eliminates redundant processing of the same code objects across
debugging sessions.
Optimize memory operations by replacing multiple individual structure
copies with single bulk reads for the same data structures. This reduces
the number of memory operations and system calls required to gather
debugging information from the target process.
Update Makefile.pre.in to include Python/remote_debug.h in the headers
list, ensuring that changes to the remote debugging header force proper
recompilation of dependent modules and maintain build consistency across
the codebase.
Also, make the module compatible with the free threading build as an extra :)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
[tests] fix test_external_inspection race assert
either line could be where the inspection finds the foo()
function as after ready is sent, the process may not have made progress
onto the next line yet. "solve" by putting the statements on the same
line.
