gh-150319: Replace all documentation which says "See PEP 585" (GH-150325)
* Replace all documentation which says "See PEP 585"
The following classes in the stdlib get simple updates:
- array.array
- asyncio.Future
- asyncio.Task
- collections.defaultdict
- collections.deque
- contextvars.ContextVar
- contextvars.Token
- ctypes.Array
- os.DirEntry
- re.Match
- re.Pattern
- string.templatelib.Interpolation
- string.templatelib.Template
- types.MappingProxyType
- queue.SimpleQueue
- weakref.ref
The following classes are documented publicly as functions, and are
therefore updated internally (`__class_getitem__.__doc__`) but not in the
public docs:
- functools.partial
- itertools.chain
The following builtin types have updates to `__class_getitem__.__doc__`
but not to any documentation pages:
- BaseExceptionGroup
- coroutines (from generators)
- dict
- enumerate
- frozendict
- frozenset
- generators (and async generators)
- list
- memoryview
- set
- slice
- tuple
Special cases:
- union objects are now documented as "supporting class-level []",
rather than anything to do with generics.
- Templates might be generic over a single type (union, in theory) or
over a TypeVarTuple. As this is not currently fully settled, it is
marked with a comment and a mild hint that it is a single type is used
(namely, "type" is singular rather than "types", plural)
* Apply suggestions from code review
* Correct several class getitem docs
And expand the text for tuples.
* Add notes on generic typing of builtins
* Fix typo in tuple.__class_getitem__ docstring
* Typo fix: malformed refs
Fix `generic` links which weren't marked as `:ref:`.
* Strike unnecessary docs on generic-ness
* Apply suggestions from code review
These are applied at both the originally indicated locations and in the
corresponding docstring definitions.
* Update Doc/library/re.rst
* Update Objects/enumobject.c
* Remove tuple generic doc in 'stdtypes' page
This is covered in more detail in the cross-linked typing documentation.
The other copy of this documentation -- in the docstring for
`tuple.__class_getitem__` -- is left in place.
* Fix whitespace around new doc of generics
Per review, do not introduce or remove whitespace such that section
breaks are altered by the introduction of doc on various generic types.
In most cases, this is a removal of an extra line.
In one case (Arrays), it is the reintroduction of a line.
Additionally, two other minor fixes are included:
- incorrect indent on 'defaultdicts'
- make `mappingproxy.__class_getitem__.__doc__` consistent with other
mapping type generic docs
* Move placement of memoryview generic note
Previous placement was at the end of the main docstring, which is
consistent with other types but places it after a section on various
methods (which makes it read somewhat inconsistently). Moving it up
helps resolve.
* Ensure sphinxdoc does not start sentences lowercase
Lowercase class names at the start of sentences are marked out with the
`class` role. In the case of `deque`, documentation already refers to
these as `Deques`, so this form is preferred.
* Apply suggestions from code review
* Fix line endings and wrap more tightly
Line endings fixed by pre-commit ; also re-wrapped the MappingProxyType
text which was too long.
* Use 'ContextVars' style in sphinx doc
---------
(cherry picked from commit 50fe49c879)
Co-authored-by: Stephen Rosen <sirosen@globus.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Jelle Zijlstra <906600+JelleZijlstra@users.noreply.github.com>
Co-authored-by: Alex Waygood <66076021+AlexWaygood@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
This partially reverts #137047, keeping the tests for GC collectability of the
original class that dataclass adds `__slots__` to.
The reference leaks solved there are instead solved by having the `__dict__` &
`__weakref__` descriptors not tied to (and referencing) their class.
Instead, they're shared between all classes that need them (within
an interpreter).
The `__objclass__` ol the descriptors is set to `object`, since these
descriptors work with *any* object. (The appropriate checks were already
made in the get/set code, so the `__objclass__` check was redundant.)
The repr of these descriptors (and any others whose `__objclass__` is `object`)
now doesn't mention the objclass.
This change required adjustment of introspection code that checks
`__objclass__` to determine an object's “own” (i.e. not inherited) `__dict__`.
Third-party code that does similar introspection of the internals will also
need adjusting.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Fix undefined behavior warnings (UBSan -fsanitize=function), for example:
Python/generated_cases.c.h:3315:13: runtime error: call to function mappingproxy_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'
descrobject.c:1160: note: mappingproxy_dealloc defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3315:13 in
Move the following private functions and structures to
pycore_modsupport.h internal C API:
* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()
Changes:
* Python/getargs.h now includes pycore_modsupport.h to export
functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
is used.
* Add pycore_modsupport.h includes when a C extension uses one of
these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
directly or indirectly (via code generated by Argument Clinic)
pycore_modsupport.h:
* _csv
* _curses_panel
* _dbm
* _gdbm
* _multiprocessing.posixshmem
* _sqlite.row
* _statistics
* grp
* resource
* syslog
* _testcapi: bad_get() no longer uses METH_FASTCALL calling
convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
by _testcapi sub-modules which need the internal C API
(pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
watchers.c.
* Remove Include/cpython/modsupport.h header file.
Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
There is a WIP proposal to enable webassembly stack switching which have been
implemented in v8:
https://github.com/WebAssembly/js-promise-integration
It is not possible to switch stacks that contain JS frames so the Emscripten JS
trampolines that allow calling functions with the wrong number of arguments
don't work in this case. However, the js-promise-integration proposal requires
the [type reflection for Wasm/JS API](https://github.com/WebAssembly/js-types)
proposal, which allows us to actually count the number of arguments a function
expects.
For better compatibility with stack switching, this PR checks if type reflection
is available, and if so we use a switch block to decide the appropriate
signature. If type reflection is unavailable, we should use the current EMJS
trampoline.
We cache the function argument counts since when I didn't cache them performance
was negatively affected.
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: Brett Cannon <brett@python.org>
Remove the following private functions from the public C API:
* _Py_CheckFunctionResult()
* _PyObject_CallMethod()
* _PyObject_CallMethodId()
* _PyObject_CallMethodIdNoArgs()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_CallMethodIdOneArg()
* _PyObject_MakeTpCall()
* _PyObject_VectorcallMethodId()
* _PyStack_AsDict()
Move these functions to the internal C API (pycore_call.h).
No longer export the following functions:
* _PyObject_Call()
* _PyObject_CallMethod()
* _PyObject_CallMethodId()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_Call_Prepend()
* _PyObject_FastCallDictTstate()
* _PyStack_AsDict()
The following functions are still exported for stdlib shared
extensions:
* _Py_CheckFunctionResult()
* _PyObject_MakeTpCall()
Mark the following internal functions as extern:
* _PyStack_UnpackDict()
* _PyStack_UnpackDict_Free()
* _PyStack_UnpackDict_FreeNoDecRef()
Remove the following functions from the public C API:
* _PyObject_RealIsInstance()
* _PyObject_RealIsSubclass()
* _Py_add_one_to_index_F()
* _Py_add_one_to_index_C()
Move _PyObject_RealIsInstance() and _PyObject_RealIsSubclass() to the
internal C API (pycore_abstract.h) and no longer export their symbols
(in libpython).
Make _Py_add_one_to_index_F() and _Py_add_one_to_index_C() functions
static: no longer export them.
Remove old aliases which were kept backwards compatibility with
Python 3.8:
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_FastCallDict()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyVectorcall_Function()
Update code which used these aliases to use new names.
Currently, calling Py_EnterRecursiveCall() and
Py_LeaveRecursiveCall() may use a function call or a static inline
function call, depending if the internal pycore_ceval.h header file
is included or not. Use a different name for the static inline
function to ensure that the static inline function is always used in
Python internals for best performance. Similar approach than
PyThreadState_GET() (function call) and _PyThreadState_GET() (static
inline function).
* Rename _Py_EnterRecursiveCall() to _Py_EnterRecursiveCallTstate()
* Rename _Py_LeaveRecursiveCall() to _Py_LeaveRecursiveCallTstate()
* pycore_ceval.h: Rename Py_EnterRecursiveCall() to
_Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() and
_Py_LeaveRecursiveCall()
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
Change generated by the command:
sed -i -e \
's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \
$(find -name "*.c")
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules.
The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).
https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.
The core of the change is in:
* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers
I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config.
The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.
The following are not changed (yet):
* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init
https://bugs.python.org/issue46541
The docstrings for MappingProxyType's keys(), values(), and items()
methods were never updated to reflect the changes that Python 3 brought
to these APIs, namely returning views rather than lists.
