Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2026-01-08 00:12:42 +00:00
Code Issues Projects Releases Packages Wiki Activity
cpython/Include/internal
History
Eric Snow dcd6f226d6
gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters (gh-103084) 
Sharing mutable (or non-immortal) objects between interpreters is generally not safe.  We can work around that but not easily. 
 There are two restrictions that are critical for objects that break interpreter isolation.

The first is that the object's state be guarded by a global lock.  For now the GIL meets this requirement, but a granular global lock is needed once we have a per-interpreter GIL.

The second restriction is that the object (and, for a container, its items) be deallocated/resized only when the interpreter in which it was allocated is the current one.  This is because every interpreter has (or will have, see gh-101660) its own object allocator.  Deallocating an object with a different allocator can cause crashes.

The dict for the cache of module defs is completely internal, which simplifies what we have to do to meet those requirements.  To do so, we do the following:

* add a mechanism for re-using a temporary thread state tied to the main interpreter in an arbitrary thread
   * add _PyRuntime.imports.extensions.main_tstate` 
   * add _PyThreadState_InitDetached() and _PyThreadState_ClearDetached() (pystate.c)
   * add _PyThreadState_BindDetached() and _PyThreadState_UnbindDetached() (pystate.c)
* make sure the cache dict (_PyRuntime.imports.extensions.dict) and its items are all owned by the main interpreter)
* add a placeholder using for a granular global lock

Note that the cache is only used for legacy extension modules and not for multi-phase init modules.

https://github.com/python/cpython/issues/100227
2023-03-29 17:15:43 -06:00
..
pycore_abstract.h bpo-45636: Simplify BINARY_OP (GH-29565) 2021-11-16 05:53:57 -08:00
pycore_asdl.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
pycore_ast.h bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
pycore_ast_state.h gh-95185: Check recursion depth in the AST constructor (#95186) 2022-07-24 15:58:52 +01:00
pycore_atomic.h Fix typo in internal/pycore_atomic.h (GH-95939) 2022-08-13 11:40:41 +08:00
pycore_atomic_funcs.h bpo-39465: Add pycore_atomic_funcs.h header (GH-20766) 2020-12-23 03:41:08 +01:00
pycore_bitutils.h bpo-29882: Fix portability bug introduced in GH-30774 (#30794) 2022-01-23 09:59:34 +00:00
pycore_blocks_output_buffer.h bpo-44458: Ensure BUFFER_BLOCK_SIZE symbol is statically allocated. (GH-26808) 2021-06-21 23:36:36 -07:00
pycore_bytes_methods.h bpo-35081: Move bytes_methods.h to the internal C API (GH-18492) 2020-02-12 22:32:34 +01:00
pycore_bytesobject.h bpo-47070: Add _PyBytes_Repeat() (GH-31999) 2022-03-28 04:43:45 -04:00
pycore_call.h GH-90230: Add stats to breakdown the origin of calls to PyEval_EvalFrame (GH-93284) 2022-05-27 16:31:41 +01:00
pycore_ceval.h GH-91079: Decouple C stack overflow checks from Python recursion checks. (GH-96510) 2022-10-05 01:34:03 +01:00
pycore_ceval_state.h gh-81057: Move Ceval Trampoline Globals to _PyRuntimeState (gh-100083) 2022-12-08 17:17:20 -07:00
pycore_code.h GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
pycore_compile.h gh-102371: move _Py_Mangle from compile.c to symtable.c (#102372) 2023-03-02 18:38:22 +00:00
pycore_condvar.h bpo-46315: Add ifdef HAVE_ feature checks for WASI compatibility (GH-30507) 2022-01-13 09:46:04 +01:00
pycore_context.h gh-81057: Move contextvars-related Globals to _PyRuntimeState (gh-99400) 2022-11-16 09:54:28 -07:00
pycore_descrobject.h gh-93911: Specialize LOAD_ATTR_PROPERTY (GH-93912) 2022-06-17 23:13:17 +08:00
pycore_dict.h gh-100227: Move dict_state.global_version to PyInterpreterState (gh-102338) 2023-03-09 08:16:30 -07:00
pycore_dict_state.h gh-100227: Move dict_state.global_version to PyInterpreterState (gh-102338) 2023-03-09 08:16:30 -07:00
pycore_dtoa.h gh-100227: Move the dtoa State to PyInterpreterState (gh-102331) 2023-02-28 13:14:40 -07:00
pycore_emscripten_signal.h bpo-47176: Interrupt handling for wasm32-emscripten builds without pthreads (GH-32209) 2022-04-03 22:58:52 +02:00
pycore_exceptions.h bpo-46417: Factorize _PyExc_InitTypes() code (GH-30804) 2022-01-22 21:48:56 +01:00
pycore_faulthandler.h gh-81057: Move faulthandler Globals to _PyRuntimeState (gh-100152) 2022-12-12 09:58:46 -07:00
pycore_fileutils.h gh-99726: Improves correctness of stat results for Windows, and uses faster API when available (GH-102149) 2023-03-16 17:27:21 +00:00
pycore_fileutils_windows.h gh-99726: Fix order of recently added fields for FILE_STAT_BASIC_INFORMATION (GH-102976) 2023-03-23 23:27:46 +00:00
pycore_floatobject.h gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516) 2022-11-16 09:37:14 -07:00
pycore_format.h bpo-45995: add "z" format specifer to coerce negative 0 to zero (GH-30049) 2022-04-11 15:34:18 +01:00
pycore_frame.h GH-100719: Remove redundant gi_code field from generator object. (GH-100749) 2023-02-23 10:19:01 +00:00
pycore_function.h gh-100227: Move func_state.next_version to PyInterpreterState (gh-102334) 2023-03-08 15:56:36 -07:00
pycore_gc.h gh-97922: Run the GC only on eval breaker (#97920) 2022-10-08 07:57:09 -07:00
pycore_genobject.h GH-99005: More intrinsics (GH-100774) 2023-01-06 14:47:57 +00:00
pycore_getopt.h bpo-36763: Cleanup precmdline in _PyCoreConfig_Read() (GH-13371) 2019-05-17 03:15:12 +02:00
pycore_gil.h bpo-38353: Cleanup includes in the internal C API (GH-16548) 2019-10-02 23:51:20 +02:00
pycore_global_objects.h gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339) 2023-03-28 12:52:28 -06:00
pycore_global_objects_fini_generated.h gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779) 2023-03-18 11:47:11 +00:00
pycore_global_strings.h gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779) 2023-03-18 11:47:11 +00:00
pycore_hamt.h gh-81057: Move contextvars-related Globals to _PyRuntimeState (gh-99400) 2022-11-16 09:54:28 -07:00
pycore_hashtable.h Use _Py_RVALUE() in macros (#99844) 2022-11-28 17:42:22 +01:00
pycore_import.h gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters (gh-103084) 2023-03-29 17:15:43 -06:00
pycore_initconfig.h gh-98608: Stop Treating All Errors from _Py_NewInterpreterFromConfig() as Fatal (gh-102657) 2023-03-21 10:49:12 -06:00
pycore_interp.h gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545) 2023-03-21 11:46:09 -06:00
pycore_interpreteridobject.h bpo-35081: Move interpreteridobject.h to Include/internal/ (GH-28969) 2021-10-15 11:56:34 +02:00
pycore_intrinsics.h gh-90110: Fix the c-analyzer Tool (#102483) 2023-03-06 19:40:09 -07:00
pycore_list.h gh-100146: Steal references from stack when building a list (#100147) 2023-01-03 10:49:49 -08:00
pycore_long.h GH-101291: Rearrange the size bits in PyLongObject (GH-102464) 2023-03-22 14:49:51 +00:00
pycore_moduleobject.h bpo-45459: Use type names in the internal C API (GH-31669) 2022-03-03 23:08:07 +01:00
pycore_namespace.h bpo-45482: Rename namespaceobject.h to pycore_namespace.h (GH-28975) 2021-10-15 15:21:21 +02:00
pycore_object.h GH-101291: Rearrange the size bits in PyLongObject (GH-102464) 2023-03-22 14:49:51 +00:00
pycore_object_state.h gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545) 2023-03-21 11:46:09 -06:00
pycore_obmalloc.h gh-81057: Move More Globals to _PyRuntimeState (gh-100092) 2022-12-07 15:56:31 -07:00
pycore_obmalloc_init.h gh-81057: Move More Globals to _PyRuntimeState (gh-100092) 2022-12-07 15:56:31 -07:00
pycore_opcode.h GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
pycore_parser.h gh-81057: Move the Cached Parser Dummy Name to _PyRuntimeState (#100277) 2022-12-16 13:48:03 +00:00
pycore_pathconfig.h bpo-45582: Port getpath[p].c to Python (GH-29041) 2021-12-03 00:08:42 +00:00
pycore_pyarena.h bpo-43244: Remove the pyarena.h header (GH-25007) 2021-03-24 02:23:01 +01:00
pycore_pyerrors.h GH-101578: Normalize the current exception (GH-101607) 2023-02-08 09:31:12 +00:00
pycore_pyhash.h gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516) 2022-11-16 09:37:14 -07:00
pycore_pylifecycle.h gh-102755: Add PyErr_DisplayException(exc) (#102756) 2023-03-16 22:18:04 +00:00
pycore_pymath.h Remove unused internal macros (#102415) 2023-03-04 12:20:14 +00:00
pycore_pymem.h gh-81057: Move tracemalloc Globals to _PyRuntimeState (gh-100151) 2022-12-12 08:44:23 -07:00
pycore_pymem_init.h gh-81057: Move the Allocators to _PyRuntimeState (gh-99217) 2022-11-11 16:30:46 -07:00
pycore_pystate.h gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters (gh-103084) 2023-03-29 17:15:43 -06:00
pycore_pythread.h gh-81057: Fix the wasm32-wasi Buildbot (gh-100139) 2022-12-09 10:17:54 -07:00
pycore_range.h gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
pycore_runtime.h gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339) 2023-03-28 12:52:28 -06:00
pycore_runtime_init.h gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters (gh-103084) 2023-03-29 17:15:43 -06:00
pycore_runtime_init_generated.h gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779) 2023-03-18 11:47:11 +00:00
pycore_signal.h gh-81057: Move Signal-Related Globals to _PyRuntimeState (gh-100085) 2022-12-12 16:50:19 -07:00
pycore_sliceobject.h GH-94163: Add BINARY_SLICE and STORE_SLICE instructions. (GH-94168) 2022-06-27 12:24:23 +01:00
pycore_strhex.h gh-91768: C API no longer use "const PyObject*" type (#91769) 2022-04-21 22:07:19 +02:00
pycore_structseq.h gh-94673: Add _PyStaticType_InitBuiltin() (#95152) 2022-07-25 12:47:31 -06:00
pycore_symtable.h gh-102371: move _Py_Mangle from compile.c to symtable.c (#102372) 2023-03-02 18:38:22 +00:00
pycore_sysmodule.h gh-101758: Clean Up Uses of Import State (gh-101919) 2023-02-15 15:32:31 -07:00
pycore_time.h gh-81057: Move time Globals to _PyRuntimeState (gh-100122) 2022-12-08 16:46:09 -07:00
pycore_token.h gh-97669: Create Tools/build/ directory (#97963) 2022-10-17 12:01:00 +02:00
pycore_traceback.h bpo-45459: Use type names in the internal C API (GH-31669) 2022-03-03 23:08:07 +01:00
pycore_tracemalloc.h gh-77532: Minor tweaks to allow compiling with PlatformToolset=ClangCL on Windows (GH-101352) 2023-01-27 14:45:08 +00:00
pycore_tuple.h GH-100110: Specialize FOR_ITER for tuples (GH-100109) 2022-12-09 10:27:01 +00:00
pycore_typeobject.h gh-101476: Add _PyType_GetModuleState (GH-101477) 2023-02-23 13:42:15 -08:00
pycore_ucnhash.h bpo-42157: Rename unicodedata.ucnhash_CAPI (GH-22994) 2020-10-27 04:36:22 +01:00
pycore_unicodeobject.h gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339) 2023-03-28 12:52:28 -06:00
pycore_unicodeobject_generated.h gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339) 2023-03-28 12:52:28 -06:00
pycore_unionobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
pycore_warnings.h bpo-35134: Split warnings.h and weakrefobject.h (GH-29042) 2021-10-19 01:31:57 +02:00