Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2026-01-06 15:32:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
cpython/Include/internal
History
Neil Schemenauer 113de8545f
GH-133136: Revise QSBR to reduce excess memory held (gh-135473) 
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.

Take into account the size of the memory block when deciding when to
process QSBR requests.

Also track the amount of memory being held by QSBR for mimalloc pages.  Advance the write sequence if this memory exceeds a limit.  Advancing the sequence will allow it to be freed more quickly.

Process the held QSBR items from the "eval breaker", rather than from `_PyMem_FreeDelayed()`.  This gives a higher chance that the global read sequence has advanced enough so that items can be freed.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2025-06-25 00:06:32 -07:00
..
mimalloc gh-134875: Fix mimallc build error for the old compilers (gh-134994) 2025-06-03 08:40:40 +09:00
pycore_abstract.h gh-106320: Remove private AC converter functions (#108505) 2023-08-26 04:05:17 +02:00
pycore_asdl.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
pycore_ast.h gh-132661: Implement PEP 750 (#132662) 2025-04-30 11:46:41 +02:00
pycore_ast_state.h gh-132661: Implement PEP 750 (#132662) 2025-04-30 11:46:41 +02:00
pycore_atexit.h gh-131238: Remove pycore_lock.h includes (#131483) 2025-03-19 23:46:25 +00:00
pycore_audit.h gh-125604: Move _Py_AuditHookEntry, etc. Out of pycore_runtime.h (gh-125605) 2024-10-18 09:26:08 -06:00
pycore_backoff.h GH-131798: Optimize cached class attributes and methods in the JIT (GH-134403) 2025-05-22 11:15:03 -04:00
pycore_bitutils.h gh-108216: Cleanup #include in internal header files (#108228) 2023-08-21 18:05:59 +00:00
pycore_blocks_output_buffer.h gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221) 2023-08-21 19:15:52 +02:00
pycore_brc.h gh-131238: Remove pycore_lock.h includes (#131483) 2025-03-19 23:46:25 +00:00
pycore_bytes_methods.h gh-117431: Adapt bytes and bytearray .find() and friends to Argument Clinic (#117502) 2024-04-12 07:40:55 +00:00
pycore_bytesobject.h gh-133767: Fix use-after-free in the unicode-escape decoder with an error handler (GH-129648) 2025-05-12 20:42:23 +03:00
pycore_c_array.h gh-130080: move _Py_EnsureArrayLargeEnough to a separate header so it can be used outside of the compiler (#130930) 2025-03-13 16:02:58 +00:00
pycore_call.h gh-131238: Remove pycore_runtime.h from pycore_pystate.h (#131356) 2025-03-19 17:33:24 +01:00
pycore_capsule.h gh-108240: Add pycore_capsule.h internal header file (#108596) 2023-08-29 01:20:02 +00:00
pycore_cell.h gh-123358: Use _PyStackRef in LOAD_DEREF (gh-130064) 2025-03-26 12:08:20 -04:00
pycore_ceval.h gh-135443: Sometimes Fall Back to __main__.__dict__ For Globals (gh-135491) 2025-06-16 17:34:19 -06:00
pycore_ceval_state.h gh-131238: Remove pycore_lock.h includes (#131483) 2025-03-19 23:46:25 +00:00
pycore_code.h GH-132554: "Virtual" iterators (GH-132555) 2025-05-27 15:59:45 +01:00
pycore_codecs.h gh-131238: Remove pycore_lock.h includes (#131483) 2025-03-19 23:46:25 +00:00
pycore_compile.h GH-132554: "Virtual" iterators (GH-132555) 2025-05-27 15:59:45 +01:00
pycore_complexobject.h gh-128813: hide mixed-mode functions for complex arithmetic from C-API (#131703) 2025-04-22 14:18:18 +02:00
pycore_condvar.h gh-131082: Add missing guards for WIN32_LEAN_AND_MEAN (#131044) 2025-03-11 12:33:01 +01:00
pycore_context.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_critical_section.h gh-135380: enhance critical section held assertions (#135381) 2025-06-20 22:43:23 +05:30
pycore_crossinterp.h gh-132775: Clean Up Cross-Interpreter Error Handling (gh-135369) 2025-06-13 16:45:21 -06:00
pycore_crossinterp_data_registry.h gh-132775: Support Fallbacks in _PyObject_GetXIData() (gh-133482) 2025-05-21 07:23:48 -06:00
pycore_debug_offsets.h gh-91048: Refactor and optimize remote debugging module (#134652) 2025-05-25 20:19:29 +00:00
pycore_descrobject.h gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
pycore_dict.h gh-132641: fix race in lru_cache under free-threading (#133787) 2025-05-13 17:38:57 +00:00
pycore_dict_state.h gh-124296: Remove private dictionary version tag (PEP 699) (#124472) 2024-10-01 12:39:56 -04:00
pycore_dtoa.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_emscripten_signal.h GH-108614: Unbreak emscripten build (GH-109132) 2023-09-08 17:54:45 +01:00
pycore_emscripten_trampoline.h gh-132097: use a macro for semantically casting function pointers (#132406) 2025-04-18 12:24:34 +02:00
pycore_exceptions.h gh-129668: Fix thread-safety of MemoryError freelist in free threaded build (gh-129704) 2025-02-06 12:38:12 -05:00
pycore_faulthandler.h gh-127604: Add C stack dumps to faulthandler (#128159) 2025-04-21 20:48:02 +01:00
pycore_fileutils.h gh-131238: Use pycore_interp_structs.h header (#131481) 2025-03-19 23:13:25 +00:00
pycore_fileutils_windows.h gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221) 2023-08-21 19:15:52 +02:00
pycore_floatobject.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_flowgraph.h gh-119744: move a few functions from compile.c to flowgraph.c (#119745) 2024-05-30 21:55:06 +01:00
pycore_format.h gh-104018: remove unused format "z" handling in string formatfloat() (#104107) 2023-05-07 10:11:42 +05:30
pycore_frame.h gh-130704: Strength reduce LOAD_FAST{_LOAD_FAST} (#130708) 2025-04-01 10:18:42 -07:00
pycore_freelist.h gh-131238: Remove includes from pycore_interp.h (#131495) 2025-03-20 11:35:23 +00:00
pycore_freelist_state.h gh-126703: Add freelist for PyComplexObject's (gh-135233) 2025-06-13 22:32:26 +09:00
pycore_function.h gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133528) 2025-05-08 00:00:33 +00:00
pycore_gc.h gh-131238: Move _Py_VISIT_STACKREF() to pycore_stackref.h (#131560) 2025-03-21 23:24:14 +01:00
pycore_genobject.h gh-131238: Move _Py_VISIT_STACKREF() to pycore_stackref.h (#131560) 2025-03-21 23:24:14 +01:00
pycore_getopt.h bpo-36763: Cleanup precmdline in _PyCoreConfig_Read() (GH-13371) 2019-05-17 03:15:12 +02:00
pycore_gil.h gh-116322: Enable the GIL while loading C extension modules (#118560) 2024-05-06 23:07:23 -04:00
pycore_global_objects.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_global_objects_fini_generated.h gh-91048: Add better error messages for remote debugging for CI builds (#134682) 2025-05-26 15:31:47 +01:00
pycore_global_strings.h gh-91048: Add better error messages for remote debugging for CI builds (#134682) 2025-05-26 15:31:47 +01:00
pycore_hamt.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_hashtable.h gh-107211: No longer export internal functions (5) (#108423) 2023-08-24 16:06:53 +00:00
pycore_import.h gh-132775: Drop PyUnstable_InterpreterState_GetMainModule() (gh-132978) 2025-04-28 12:46:22 -06:00
pycore_importdl.h gh-131942: Use the Python-specific Py_DEBUG macro rather than _DEBUG in Windows-related C code (GH-131944) 2025-05-08 15:01:25 +00:00
pycore_index_pool.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_initconfig.h gh-130931: Add pycore_typedefs.h internal header (#131396) 2025-03-19 15:23:32 +01:00
pycore_instruction_sequence.h gh-130907: Treat all module-level annotations as conditional (#131550) 2025-04-28 06:10:28 -07:00
pycore_instruments.h gh-130931: Add pycore_typedefs.h internal header (#131396) 2025-03-19 15:23:32 +01:00
pycore_interp.h gh-131238: Remove includes from pycore_interp.h (#131495) 2025-03-20 11:35:23 +00:00
pycore_interp_structs.h gh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (GH-135209) 2025-06-11 12:44:58 +02:00
pycore_interpframe.h gh-133713: Compare the f->stackpointer to the result of _PyFrame_Stackbase(f) (GH-133714) 2025-05-10 11:04:32 +08:00
pycore_interpframe_structs.h gh-131238: Add pycore_interpframe_structs.h header (#131553) 2025-03-21 17:19:47 +00:00
pycore_interpolation.h gh-132661: Implement PEP 750 (#132662) 2025-04-30 11:46:41 +02:00
pycore_intrinsics.h gh-116126: Implement PEP 696 (#116129) 2024-05-03 06:17:32 -07:00
pycore_jit.h GH-131498: Remove conditional stack effects (GH-131499) 2025-03-20 15:39:38 +00:00
pycore_list.h gh-100239: specialize BINARY_OP/SUBSCR for list-slice (#132626) 2025-05-01 10:28:52 +00:00
pycore_llist.h gh-111964: Implement stop-the-world pauses (gh-112471) 2024-01-23 11:08:23 -07:00
pycore_lock.h gh-134745: Change PyThread_allocate_lock() implementation to PyMutex (#134747) 2025-05-30 10:15:47 +00:00
pycore_long.h gh-135379: Move PyLong_CheckCompact to private header and rename it (GH-135707) 2025-06-19 13:09:09 +00:00
pycore_magic_number.h gh-134889: Fix handling of a few opcodes when optimizing LOAD_FAST (#134958) 2025-06-04 16:07:58 -07:00
pycore_memoryobject.h gh-132776: Revert Moving memoryview XIData Code to memoryobject.c (gh-132960) 2025-04-25 16:43:50 +00:00
pycore_mimalloc.h gh-122584: Import mimalloc headers in a C++ context (#122587) 2024-08-15 09:01:01 -04:00
pycore_modsupport.h gh-129594: Remove redundant check on varargs in _PyArg_CheckPositional (#129595) 2025-05-26 10:51:12 +02:00
pycore_moduleobject.h gh-132775: Add _PyModule_GetFilenameObject() and _PyModule_GetFilenameUTF8() (gh-132979) 2025-04-28 12:41:32 -06:00
pycore_namespace.h gh-107211: No longer export internal functions (4) (#107217) 2023-07-25 03:16:28 +00:00
pycore_object.h gh-133931: Introduce _PyObject_XSetRefDelayed to replace Py_XSETREF (gh-134377) 2025-06-18 08:36:02 +09:00
pycore_object_alloc.h gh-112529: Use GC heaps for GC allocations in free-threaded builds (gh-114157) 2024-01-21 01:14:45 +09:00
pycore_object_deferred.h gh-117139: Convert the evaluation stack to stack refs (#118450) 2024-06-27 03:10:43 +08:00
pycore_object_stack.h gh-125859: Fix crash when gc.get_objects is called during GC (#125882) 2024-10-24 09:33:11 -04:00
pycore_object_state.h gh-125604: Move _Py_AuditHookEntry, etc. Out of pycore_runtime.h (gh-125605) 2024-10-18 09:26:08 -06:00
pycore_obmalloc.h gh-92953: Improve nextpool/prevpool comment. (gh-125545) 2024-10-15 11:47:20 -07:00
pycore_obmalloc_init.h gh-113055: Use pointer for interp->obmalloc state (gh-113412) 2024-01-26 19:38:14 -08:00
pycore_opcode_metadata.h GH-135379: Specialize int operations for compact ints only (GH-135668) 2025-06-19 11:10:29 +01:00
pycore_opcode_utils.h gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133528) 2025-05-08 00:00:33 +00:00
pycore_optimizer.h GH-135379: Specialize int operations for compact ints only (GH-135668) 2025-06-19 11:10:29 +01:00
pycore_parking_lot.h gh-110850: Cleanup pycore_time.h includes (#115724) 2024-02-20 16:50:43 +00:00
pycore_parser.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_pathconfig.h gh-107211: No longer export internal functions (6) (#108424) 2023-08-24 17:28:35 +02:00
pycore_pyarena.h gh-107211: Fix test_peg_generator (#108435) 2023-08-24 17:47:44 +00:00
pycore_pyatomic_ft_wrappers.h gh-131757: allow lru_cache functions to execute concurrently (#131758) 2025-04-14 18:31:19 +02:00
pycore_pybuffer.h gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791) 2024-04-24 16:18:24 +00:00
pycore_pyerrors.h gh-132775: Use _PyFunction_VerifyStateless() and _PyCode_VerifyStateless() (gh-134439) 2025-05-21 14:16:55 -06:00
pycore_pyhash.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_pylifecycle.h gh-131238: Cleanup pycore_runtime.h includes (#131486) 2025-03-20 00:47:30 +00:00
pycore_pymath.h gh-120026: soft deprecate Py_HUGE_VAL macro (#120027) 2024-11-01 22:04:31 +00:00
pycore_pymem.h GH-133136: Revise QSBR to reduce excess memory held (gh-135473) 2025-06-25 00:06:32 -07:00
pycore_pymem_init.h gh-115103: Implement delayed free mechanism for free-threaded builds (#115367) 2024-02-20 13:04:37 -05:00
pycore_pystate.h gh-133261: Use __builtin_frame_address() on GCC 9 and older (#133269) 2025-05-01 19:13:03 +02:00
pycore_pystats.h gh-108753: Enhance pystats (#108754) 2023-09-06 15:54:59 +00:00
pycore_pythonrun.h gh-132775: Add _PyCode_GetScriptXIData() (gh-133480) 2025-05-08 15:07:46 +00:00
pycore_pythread.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_qsbr.h GH-133136: Revise QSBR to reduce excess memory held (gh-135473) 2025-06-25 00:06:32 -07:00
pycore_range.h gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
pycore_runtime.h gh-131238: Use pycore_interp_structs.h header (#131481) 2025-03-19 23:13:25 +00:00
pycore_runtime_init.h gh-131185: Use a proper thread-local for cached thread states (gh-132510) 2025-05-21 07:01:25 -06:00
pycore_runtime_init_generated.h gh-91048: Add better error messages for remote debugging for CI builds (#134682) 2025-05-26 15:31:47 +01:00
pycore_runtime_structs.h gh-131185: Use a proper thread-local for cached thread states (gh-132510) 2025-05-21 07:01:25 -06:00
pycore_semaphore.h gh-131082: Add missing guards for WIN32_LEAN_AND_MEAN (#131044) 2025-03-11 12:33:01 +01:00
pycore_setobject.h gh-130312: SET_ADD should not lock (#130136) 2025-03-21 15:58:32 -07:00
pycore_signal.h gh-109693: Use pyatomic.h for signal module (gh-110480) 2023-10-10 08:26:29 +09:00
pycore_sliceobject.h GH-115802: JIT "small" code for Windows (GH-115964) 2024-02-29 08:11:28 -08:00
pycore_stackref.h GH-135379: Remove types from stack items in code generator. (GH-135384) 2025-06-11 15:52:25 +01:00
pycore_stats.h GH-131238: More refactoring of core header files (GH-131351) 2025-03-17 14:41:05 +00:00
pycore_strhex.h gh-107211: No longer export pycore_strhex.h functions (#108229) 2023-08-21 18:12:22 +00:00
pycore_structs.h GH-131498: Cases generator: manage stacks automatically (GH-132074) 2025-04-04 17:59:36 +01:00
pycore_structseq.h gh-107211: No longer export internal functions (4) (#107217) 2023-07-25 03:16:28 +00:00
pycore_symtable.h gh-130924: Do not create cells for usages of names in local annotations (#131843) 2025-03-29 21:15:48 +00:00
pycore_sysmodule.h gh-108512: Add and use new replacements for PySys_GetObject() (GH-111035) 2025-05-28 20:11:09 +03:00
pycore_template.h gh-132661: Implement PEP 750 (#132662) 2025-04-30 11:46:41 +02:00
pycore_time.h gh-131238: Use pycore_interp_structs.h header (#131481) 2025-03-19 23:13:25 +00:00
pycore_token.h gh-132661: Implement PEP 750 (#132662) 2025-04-30 11:46:41 +02:00
pycore_traceback.h gh-127604: Add C stack dumps to faulthandler (#128159) 2025-04-21 20:48:02 +01:00
pycore_tracemalloc.h gh-129185: Use PyMutex in tracemalloc (#129246) 2025-01-24 11:25:24 +01:00
pycore_tstate.h gh-131586: Avoid refcount contention in some "special" calls (#131588) 2025-03-26 14:38:47 -04:00
pycore_tuple.h gh-131525: Cache the result of tuple_hash (#131529) 2025-03-27 09:57:06 -04:00
pycore_typedefs.h gh-130931: Add pycore_typedefs.h internal header (#131396) 2025-03-19 15:23:32 +01:00
pycore_typeobject.h gh-127266: avoid data races when updating type slots (gh-133177) 2025-05-27 18:27:41 -07:00
pycore_typevarobject.h gh-119180: Add evaluate functions for type params and type aliases (#122212) 2024-07-27 17:24:10 +00:00
pycore_ucnhash.h gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
pycore_unicodeobject.h gh-133767: Fix use-after-free in the unicode-escape decoder with an error handler (GH-129648) 2025-05-12 20:42:23 +03:00
pycore_unicodeobject_generated.h gh-91048: Add better error messages for remote debugging for CI builds (#134682) 2025-05-26 15:31:47 +01:00
pycore_unionobject.h gh-105499: Merge typing.Union and types.UnionType (#105511) 2025-03-04 11:44:19 -08:00
pycore_uniqueid.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_uop_ids.h gh-134584: Specialize POP_TOP by reference and type in JIT (GH-135761) 2025-06-24 00:57:14 +08:00
pycore_uop_metadata.h gh-134584: Specialize POP_TOP by reference and type in JIT (GH-135761) 2025-06-24 00:57:14 +08:00
pycore_warnings.h GH-131238: Core header refactor (GH-131250) 2025-03-17 09:19:04 +00:00
pycore_weakref.h gh-131238: Remove pycore_lock.h includes (#131483) 2025-03-19 23:46:25 +00:00