Commit graph

1016 commits

Author SHA1 Message Date
Ken Jin
e28b0dc86d
gh-107557: Setup abstract interpretation (#107847)
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Jules <57632293+juliapoo@users.noreply.github.com>
2023-08-15 18:04:17 +00:00
Finn Womack
0932272431
gh-106242: Fix path truncation in os.path.normpath (GH-106816) 2023-08-15 16:33:00 +01:00
Irit Katriel
39745347f6
gh-105481: reduce repetition in opcode metadata generation code (#107942) 2023-08-14 18:36:29 +00:00
Serhiy Storchaka
3901c991e1
gh-84805: Autogenerate signature for METH_NOARGS and METH_O extension functions (GH-107794) 2023-08-11 18:08:38 +03:00
Mark Shannon
1d976b2da2
GH-106485: Handle dict subclasses correctly when dematerializing __dict__ (GH-107837) 2023-08-10 13:34:00 +01:00
Irit Katriel
bafedfbebd
gh-106149: move CFG and basicblock definitions into flowgraph.c, use them as opaque types in compile.c (#107639) 2023-08-10 13:03:47 +01:00
Brandt Bucher
326f0ba1c5
GH-106485: Dematerialize instance dictionaries when possible (GH-106539) 2023-08-09 19:14:50 +00:00
Brandt Bucher
a9caf9cf90
GH-105848: Simplify the arrangement of CALL's stack (GH-107788) 2023-08-09 18:19:39 +00:00
Mark Shannon
52fbcf61b5
GH-107724: Fix the signature of PY_THROW callback functions. (GH-107725) 2023-08-09 09:30:50 +01:00
Brandt Bucher
ea72c6fe3b
GH-107596: Specialize str[int] (GH-107597) 2023-08-08 13:42:43 -07:00
Eric Snow
430632d6f7
gh-107630: Initialize Each Interpreter's refchain Properly (gh-107733)
This finishes fixing the crashes in Py_TRACE_REFS builds.  We missed this part in gh-107567.
2023-08-07 13:14:56 -06:00
Ivin Lee
4e6fac7fcc
gh-106608: make uop trace variable length (#107531)
Executors are now more like tuples.
2023-08-04 21:10:46 -07:00
Brandt Bucher
05a824f294
GH-84436: Skip refcounting for known immortals (GH-107605) 2023-08-04 16:24:50 -07:00
Guido van Rossum
400835ea16
gh-106812: Refactor cases_generator to allow uops with array stack effects (#107564)
Introducing a new file, stacking.py, that takes over several responsibilities related to symbolic evaluation of push/pop operations, with more generality.
2023-08-04 09:35:56 -07:00
Mark Shannon
2ba7c7f7b1
Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Eric Snow
58ef741867
gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
2023-08-03 19:51:08 +00:00
Irit Katriel
6ef8f8ca88
gh-105481: the ENABLE_SPECIALIZATION flag does not need to be generated by the build script, or exposed in opcode.py (#107534) 2023-08-01 17:05:00 +00:00
Guido van Rossum
5eb80a61f5
GH-104909: Move unused cache entries from uops to macros (#107444)
There's no need to use a dummy uop to skip unused cache entries. The macro syntax lets you write `unused/1` instead.

Similarly, move `unused/5` from op `_LOAD_ATTR_INSTANCE_VALUE` to macro `LOAD_ATTR_INSTANCE_VALUE`.
2023-07-31 08:55:33 -07:00
Eric Snow
8ba4df91ae
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules.  The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL.  It turns out it's tricky to use the same thread state for different threads.  Who could have guessed?

We solve the problem by eliminating the one object we were still sharing between interpreters.  We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.

We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
2023-07-28 14:39:08 -06:00
Mark Shannon
a1b679572e
GH-104580: Put eval_breaker back at the start of the interpreter state. (GH-107383) 2023-07-28 13:55:25 +01:00
Eric Snow
8bdae1424b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)
The _xxsubinterpreters module should not rely on internal API.  Some of the functions it uses were recently moved there however.  Here we move them back (and expose them properly).
2023-07-27 15:30:16 -06:00
Eric Snow
b72947a8d2
gh-106931: Intern Statically Allocated Strings Globally (gh-107272)
We tried this before with a dict and for all interned strings.  That ran into problems due to interpreter isolation.  However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning.  Here we circle back to using a global cache, but only for statically allocated strings.  We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.

Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter.  Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
2023-07-27 13:56:59 -06:00
Mark Shannon
c6539b36c1
GH-106895: Raise a ValueError when attempting to disable events that cannot be disabled. (GH-107337) 2023-07-27 15:27:11 +01:00
Irit Katriel
d77d973335
gh-105481: remove dependency of _inline_cache_entries on opname (#107339) 2023-07-27 14:15:25 +01:00
Mark Shannon
766d2518ae
GH-106897: Add RERAISE event to sys.monitoring. (GH-107291)
* Ensures that exception handling events are balanced. Each [re]raise event has a matching unwind/handled event.
2023-07-27 13:32:30 +01:00
Pablo Galindo Salgado
da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer (#107018) 2023-07-26 16:34:15 +01:00
Irit Katriel
b0202a4e5d
gh-106149: Simplify stack depth calculation. Replace asserts by exceptions. (#107255) 2023-07-26 13:32:47 +01:00
Victor Stinner
188000ae4b
Remove unused internal _PyImport_GetModuleId() function (#107235) 2023-07-25 17:02:12 +00:00
Victor Stinner
1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner
3b309319cc
gh-107211: No longer export internal variables (#107218)
No longer export these 5 internal C API variables:

* _PyBufferWrapper_Type
* _PyImport_FrozenBootstrap
* _PyImport_FrozenStdlib
* _PyImport_FrozenTest
* _Py_SwappedOp

Fix the definition of these internal functions, replace PyAPI_DATA()
with PyAPI_FUNC():

* _PyImport_ClearExtension()
* _PyObject_IsFreed()
* _PyThreadState_GetCurrent()
2023-07-25 03:48:04 +00:00
Victor Stinner
c5b13d6f80
gh-107211: No longer export internal functions (4) (#107217)
No longer export these 2 internal C API functions:

* _PyEval_SignalAsyncExc()
* _PyEval_SignalReceived()

Add also comments explaining why some internal functions have to be
exported, and update existing comments.
2023-07-25 03:16:28 +00:00
Victor Stinner
0d0520af83
gh-107211: No longer export internal functions (3) (#107215)
No longer export these 14 internal C API functions:

* _PySys_Audit()
* _PySys_SetAttr()
* _PyTraceBack_FromFrame()
* _PyTraceBack_Print_Indented()
* _PyUnicode_FormatAdvancedWriter()
* _PyUnicode_ScanIdentifier()
* _PyWarnings_Init()
* _Py_DumpASCII()
* _Py_DumpDecimal()
* _Py_DumpHexadecimal()
* _Py_DumpTraceback()
* _Py_DumpTracebackThreads()
* _Py_WriteIndent()
* _Py_WriteIndentedMargin()
2023-07-25 02:25:45 +00:00
Victor Stinner
5a61692c6c
gh-107211: No longer export internal functions (2) (#107214)
No longer export these 43 internal C API functions:

* _PyDict_CheckConsistency()
* _PyErr_ChainStackItem()
* _PyErr_CheckSignals()
* _PyErr_CheckSignalsTstate()
* _PyErr_Clear()
* _PyErr_ExceptionMatches()
* _PyErr_Fetch()
* _PyErr_Format()
* _PyErr_FormatFromCauseTstate()
* _PyErr_GetExcInfo()
* _PyErr_GetHandledException()
* _PyErr_GetTopmostException()
* _PyErr_NoMemory()
* _PyErr_NormalizeException()
* _PyErr_Restore()
* _PyErr_SetHandledException()
* _PyErr_SetNone()
* _PyErr_SetObject()
* _PyErr_SetString()
* _PyErr_StackItemToExcInfoTuple()
* _PyExc_CreateExceptionGroup()
* _PyExc_PrepReraiseStar()
* _PyException_AddNote()
* _PyInterpreterState_Enable()
* _PyLong_FormatAdvancedWriter()
* _PyLong_FormatBytesWriter()
* _PyLong_FormatWriter()
* _PyMem_GetAllocatorName()
* _PyMem_SetDefaultAllocator()
* _PyMem_SetupAllocators()
* _PyOS_InterruptOccurred()
* _PyRuntimeState_Fini()
* _PyRuntimeState_Init()
* _PyRuntime_Finalize()
* _PyRuntime_Initialize()
* _PyThreadState_Bind()
* _PyThreadState_DeleteExcept()
* _PyThreadState_New()
* _PyThreadState_Swap()
* _PyType_CheckConsistency()
* _PyUnicodeTranslateError_Create()
* _Py_DumpExtensionModules()
* _Py_FatalErrorFormat()
2023-07-25 03:49:28 +02:00
Victor Stinner
11306a91bd
gh-107211: No longer export internal functions (1) (#107213)
No longer export these 49 internal C API functions:

* _PyArgv_AsWstrList()
* _PyCode_New()
* _PyCode_Validate()
* _PyFloat_DebugMallocStats()
* _PyFloat_FormatAdvancedWriter()
* _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* _PyImport_ClearExtension()
* _PyImport_GetModuleId()
* _PyImport_SetModuleString()
* _PyInterpreterState_IDDecref()
* _PyInterpreterState_IDIncref()
* _PyInterpreterState_IDInitref()
* _PyInterpreterState_LookUpID()
* _PyWideStringList_AsList()
* _PyWideStringList_CheckConsistency()
* _PyWideStringList_Clear()
* _PyWideStringList_Copy()
* _PyWideStringList_Extend()
* _Py_ClearArgcArgv()
* _Py_DecodeUTF8Ex()
* _Py_DecodeUTF8_surrogateescape()
* _Py_EncodeLocaleRaw()
* _Py_EncodeUTF8Ex()
* _Py_GetEnv()
* _Py_GetForceASCII()
* _Py_GetLocaleEncoding()
* _Py_GetLocaleEncodingObject()
* _Py_GetLocaleconvNumeric()
* _Py_ResetForceASCII()
* _Py_device_encoding()
* _Py_dg_dtoa()
* _Py_dg_freedtoa()
* _Py_dg_strtod()
* _Py_get_blocking()
* _Py_get_env_flag()
* _Py_get_inheritable()
* _Py_get_osfhandle_noraise()
* _Py_get_xoption()
* _Py_open()
* _Py_open_osfhandle()
* _Py_open_osfhandle_noraise()
* _Py_read()
* _Py_set_blocking()
* _Py_str_to_int()
* _Py_wfopen()
* _Py_wgetcwd()
* _Py_wreadlink()
* _Py_wrealpath()
* _Py_write()
2023-07-25 03:44:11 +02:00
Victor Stinner
2e0744955f
gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212)
Rename the internal PySymtable_Lookup() function to
_PySymtable_Lookup() and no longer export it.
2023-07-25 00:54:09 +00:00
Irit Katriel
7608fa8fd5
gh-106149: move _PyCfg_BasicblockLastInstr and make it local to flowgraph.c (#107180) 2023-07-24 22:08:59 +01:00
Victor Stinner
307186704d
gh-106320: Remove private _PyMem API (#107187)
Move private _PyMem functions to the internal C API (pycore_pymem.h):

* _PyMem_GetCurrentAllocatorName()
* _PyMem_RawStrdup()
* _PyMem_RawWcsdup()
* _PyMem_Strdup()

No longer export these functions.

Move pymem_getallocatorsname() function from _testcapi to _testinternalcapi,
since the API moved to the internal C API.
2023-07-24 18:48:06 +00:00
Victor Stinner
d27eb1e406
gh-106320: Remove private _PyUnicode C API (#107185)
Move private _PyUnicode functions to the internal C API
(pycore_unicodeobject.h):

* _PyUnicode_IsCaseIgnorable()
* _PyUnicode_IsCased()
* _PyUnicode_IsXidContinue()
* _PyUnicode_IsXidStart()
* _PyUnicode_ToFoldedFull()
* _PyUnicode_ToLowerFull()
* _PyUnicode_ToTitleFull()
* _PyUnicode_ToUpperFull()

No longer export these functions.
2023-07-24 18:26:29 +00:00
Victor Stinner
fd66baf34a
gh-106320: Remove private _PyDict C API (#107145)
Move private _PyDict functions to the internal C API (pycore_dict.h):

* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()

No longer export these functions.
2023-07-24 14:02:03 +00:00
Victor Stinner
1e50112287
gh-106320: Remove private _PyObject C API (#107159)
Move private _PyObject and private _PyType functions to the internal
C API (pycore_object.h):

* _PyObject_GetMethod()
* _PyObject_IsAbstract()
* _PyObject_NextNotImplemented()
* _PyType_CalculateMetaclass()
* _PyType_GetDocFromInternalDoc()
* _PyType_GetTextSignatureFromInternalDoc()

No longer export these functions.
2023-07-23 22:25:48 +00:00
Victor Stinner
7d41ead919
gh-106320: Remove _PyBytes_Join() C API (#107144)
Move private _PyBytes functions to the internal C API
(pycore_bytesobject.h):

* _PyBytes_DecodeEscape()
* _PyBytes_FormatEx()
* _PyBytes_FromHex()
* _PyBytes_Join()

No longer export these functions.
2023-07-23 20:10:12 +00:00
Victor Stinner
0d6dfd68d2
gh-106320: Remove private _PyObject C API (#107147)
Move private debug _PyObject functions to the internal C API
(pycore_object.h):

* _PyDebugAllocatorStats()
* _PyObject_CheckConsistency()
* _PyObject_DebugTypeStats()
* _PyObject_IsFreed()

No longer export most of these functions, except of
_PyObject_IsFreed().

Move test functions using _PyObject_IsFreed() from _testcapi to
_testinternalcapi. check_pyobject_is_freed() test no longer catch
_testcapi.error: the tested function cannot raise _testcapi.error.
2023-07-23 20:09:08 +00:00
Victor Stinner
0810b0c435
gh-106320: Remove _PyTuple_MaybeUntrack() C API (#107143)
Move _PyTuple_MaybeUntrack() and _PyTuple_DebugMallocStats() functions
to the internal C API (pycore_tuple.h). No longer export these functions.
2023-07-23 19:16:21 +00:00
Victor Stinner
adb27ea2d5
gh-106320: Remove _PyIsSelectable_fd() C API (#107142)
Move _PyIsSelectable_fd() macro to the internal C API
(pycore_fileutils.h).
2023-07-23 19:07:12 +00:00
Victor Stinner
889851ecc3
gh-106320: Remove _PyFunction_Vectorcall() API (#107071)
Move _PyFunction_Vectorcall() API to the internal C API.
No longer export the function.
2023-07-22 21:44:33 +00:00
Victor Stinner
c1331ad508
gh-106320: Remove private _PyModule API (#107070)
Move private _PyModule API to the internal C API
(pycore_moduleobject.h):

* _PyModule_Clear()
* _PyModule_ClearDict()
* _PyModuleSpec_IsInitializing()
* _PyModule_IsExtension()

No longer export these functions.
2023-07-22 21:41:11 +00:00
Victor Stinner
0927a2b25c
GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069)
Rename private C API constants:

* Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS
* Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS
2023-07-22 21:35:27 +00:00
Victor Stinner
ee15844db8
gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
Victor Stinner
22422e9d1a
gh-106320: Remove private _PyInterpreterID C API (#107053)
Move the private _PyInterpreterID C API to the internal C API: add a
new pycore_interp_id.h header file.

Remove Include/interpreteridobject.h and
Include/cpython/interpreteridobject.h header files.
2023-07-22 19:31:55 +00:00
Victor Stinner
6fbc717214
gh-106320: Remove _Py_SwappedOp from the C API (#107036)
Move _Py_SwappedOp to the internal C API (pycore_object.h).
2023-07-22 16:07:07 +00:00