Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2026-04-19 18:31:10 +00:00
Code Issues Projects Releases Packages Wiki Activity

gh-145921: Add "_DuringGC" functions for tp_traverse (GH-145925)

Browse source 
There are newly documented restrictions on tp_traverse:

    The traversal function must not have any side effects.
    It must not modify the reference counts of any Python
    objects nor create or destroy any Python objects.

* Add several functions that are guaranteed side-effect-free,
  with a _DuringGC suffix.
* Use these in ctypes
* Consolidate tp_traverse docs in gcsupport.rst, moving unique
  content from typeobj.rst there

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
This commit is contained in:
Petr Viktorin 2026-04-08 09:15:11 +02:00 • committed by GitHub
parent 0b20bff386
commit 8923ca418c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
22 changed files with 455 additions and 181 deletions
Download patch file Download diff file Expand all files Collapse all files

255
Doc/c-api/gcsupport.rst
View file

@ -220,42 +220,6 @@ The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter o
detection; it's not expected that users will need to write their own
visitor functions.
The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type:
.. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg)
Traversal function for a container object. Implementations must call the
*visit* function for each object directly contained by *self*, with the
parameters to *visit* being the contained object and the *arg* value passed
to the handler. The *visit* function must not be called with a ``NULL``
object argument. If *visit* returns a non-zero value that value should be
returned immediately.
The traversal function must not have any side effects. Implementations
may not modify the reference counts of any Python objects nor create or
destroy any Python objects.
To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is
provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation
must name its arguments exactly *visit* and *arg*:
.. c:macro:: Py_VISIT(o)
If the :c:expr:`PyObject *` *o* is not ``NULL``, call the *visit* callback, with arguments *o*
and *arg*. If *visit* returns a non-zero value, then return it.
Using this macro, :c:member:`~PyTypeObject.tp_traverse` handlers
look like::
static int
my_traverse(Noddy *self, visitproc visit, void *arg)
{
Py_VISIT(self->foo);
Py_VISIT(self->bar);
return 0;
}
The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or ``NULL``
if the object is immutable.
@ -270,6 +234,225 @@ if the object is immutable.
in a reference cycle.
.. _gc-traversal:
Traversal
---------
The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type:
.. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg)
Traversal function for a garbage-collected object, used by the garbage
collector to detect reference cycles.
Implementations must call the
*visit* function for each object directly contained by *self*, with the
parameters to *visit* being the contained object and the *arg* value passed
to the handler. The *visit* function must not be called with a ``NULL``
object argument. If *visit* returns a non-zero value, that value should be
returned immediately.
A typical :c:member:`!tp_traverse` function calls the :c:func:`Py_VISIT`
convenience macro on each of the instance's members that are Python
objects that the instance owns.
For example, this is a (slightly outdated) traversal function for
the :py:class:`threading.local` class::
static int
local_traverse(PyObject *op, visitproc visit, void *arg)
{
localobject *self = (localobject *) op;
Py_VISIT(Py_TYPE(self));
Py_VISIT(self->args);
Py_VISIT(self->kw);
Py_VISIT(self->dict);
return 0;
}
.. note::
:c:func:`Py_VISIT` requires the *visit* and *arg* parameters to
:c:func:`!local_traverse` to have these specific names; don't name them just
anything.
Instances of :ref:`heap-allocated types <heap-types>` hold a reference to
their type. Their traversal function must therefore visit the type::
Py_VISIT(Py_TYPE(self));
Alternately, the type may delegate this responsibility by
calling ``tp_traverse`` of a heap-allocated superclass (or another
heap-allocated type, if applicable).
If they do not, the type object may not be garbage-collected.
If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
:c:member:`~PyTypeObject.tp_flags` field, the traverse function must call
:c:func:`PyObject_VisitManagedDict` like this::
int err = PyObject_VisitManagedDict((PyObject*)self, visit, arg);
if (err) {
return err;
}
Only the members that the instance *owns* (by having
:term:`strong references <strong reference>` to them) must be
visited. For instance, if an object supports weak references via the
:c:member:`~PyTypeObject.tp_weaklist` slot, the pointer supporting
the linked list (what *tp_weaklist* points to) must **not** be
visited as the instance does not directly own the weak references to itself.
The traversal function has a limitation:
.. warning::
The traversal function must not have any side effects. Implementations
may not modify the reference counts of any Python objects nor create or
destroy any Python objects, directly or indirectly.
This means that *most* Python C API functions may not be used, since
they can raise a new exception, return a new reference to a result object,
have internal logic that uses side effects.
Also, unless documented otherwise, functions that happen to not have side
effects may start having them in future versions, without warning.
For a list of safe functions, see a
:ref:`separate section <duringgc-functions>` below.
.. note::
The :c:func:`Py_VISIT` call may be skipped for those members that provably
cannot participate in reference cycles.
In the ``local_traverse`` example above, there is also a ``self->key``
member, but it can only be ``NULL`` or a Python string and therefore
cannot be part of a reference cycle.
On the other hand, even if you know a member can never be part of a cycle,
as a debugging aid you may want to visit it anyway just so the :mod:`gc`
module's :func:`~gc.get_referents` function will include it.
.. note::
The :c:member:`~PyTypeObject.tp_traverse` function can be called from any
thread.
.. impl-detail::
Garbage collection is a "stop-the-world" operation:
even in :term:`free threading` builds, only one thread state is
:term:`attached <attached thread state>` when :c:member:`!tp_traverse`
handlers run.
.. versionchanged:: 3.9
Heap-allocated types are expected to visit ``Py_TYPE(self)`` in
``tp_traverse``. In earlier versions of Python, due to
`bug 40217 <https://bugs.python.org/issue40217>`_, doing this
may lead to crashes in subclasses.
To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers,
a :c:func:`Py_VISIT` macro is provided.
In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse`
implementation must name its arguments exactly *visit* and *arg*:
.. c:macro:: Py_VISIT(o)
If the :c:expr:`PyObject *` *o* is not ``NULL``, call the *visit*
callback, with arguments *o* and *arg*.
If *visit* returns a non-zero value, then return it.
This corresponds roughly to::
#define Py_VISIT(o) \
if (op) { \
int visit_result = visit(o, arg); \
if (visit_result != 0) { \
return visit_result; \
} \
}
Traversal-safe functions
^^^^^^^^^^^^^^^^^^^^^^^^
The following functions and macros are safe to use in a
:c:member:`~PyTypeObject.tp_traverse` handler:
* the *visit* function passed to ``tp_traverse``
* :c:func:`Py_VISIT`
* :c:func:`Py_SIZE`
* :c:func:`Py_TYPE`: if called from a :c:member:`!tp_traverse` handler,
:c:func:`!Py_TYPE`'s result will be valid for the duration of the handler call
* :c:func:`PyObject_VisitManagedDict`
* :c:func:`PyObject_TypeCheck`, :c:func:`PyType_IsSubtype`,
:c:func:`PyType_HasFeature`
* :samp:`Py{<type>}_Check` and :samp:`Py{<type>}_CheckExact` -- for example,
:c:func:`PyTuple_Check`
* :ref:`duringgc-functions`
.. _duringgc-functions:
"DuringGC" functions
^^^^^^^^^^^^^^^^^^^^
The following functions should *only* be used in a
:c:member:`~PyTypeObject.tp_traverse` handler; calling them in other
contexts may have unintended consequences.
These functions act like their counterparts without the ``_DuringGC`` suffix,
but they are guaranteed to not have side effects, they do not set an exception
on failure, and they return/set :term:`borrowed references <borrowed reference>`
as detailed in the individual documentation.
Note that these functions may fail (return ``NULL`` or ``-1``),
but as they do not set an exception, no error information is available.
In some cases, failure is not distinguishable from a successful ``NULL`` result.
.. c:function:: void *PyObject_GetTypeData_DuringGC(PyObject *o, PyTypeObject *cls)
void *PyObject_GetItemData_DuringGC(PyObject *o)
void *PyType_GetModuleState_DuringGC(PyTypeObject *type)
void *PyModule_GetState_DuringGC(PyObject *module)
int PyModule_GetToken_DuringGC(PyObject *module, void** result)
See :ref:`duringgc-functions` for common information.
.. versionadded:: next
.. seealso::
:c:func:`PyObject_GetTypeData`,
:c:func:`PyObject_GetItemData`,
:c:func:`PyType_GetModuleState`,
:c:func:`PyModule_GetState`,
:c:func:`PyModule_GetToken`,
:c:func:`PyType_GetBaseByToken`
.. c:function:: int PyType_GetBaseByToken_DuringGC(PyTypeObject *type, void *tp_token, PyTypeObject **result)
See :ref:`duringgc-functions` for common information.
Sets *\*result* to a :term:`borrowed reference` rather than a strong one.
The reference is valid for the duration
of the :c:member:`!tp_traverse` handler call.
.. versionadded:: next
.. seealso:: :c:func:`PyType_GetBaseByToken`
.. c:function:: PyObject* PyType_GetModule_DuringGC(PyTypeObject *type)
PyObject* PyType_GetModuleByToken_DuringGC(PyTypeObject *type, const void *mod_token)
See :ref:`duringgc-functions` for common information.
These functions return a :term:`borrowed reference`, which is
valid for the duration of the :c:member:`!tp_traverse` handler call.
.. versionadded:: next
.. seealso::
:c:func:`PyType_GetModule`,
:c:func:`PyType_GetModuleByToken`
Controlling the Garbage Collector State
---------------------------------------

88
Doc/c-api/typeobj.rst
View file

@ -1563,93 +1563,9 @@ and :c:data:`PyType_Type` effectively act as defaults.)
.. corresponding-type-slot:: Py_tp_traverse
An optional pointer to a traversal function for the garbage collector. This is
only used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is::
only used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set.
int tp_traverse(PyObject *self, visitproc visit, void *arg);
More information about Python's garbage collection scheme can be found
in section :ref:`supporting-cycle-detection`.
The :c:member:`~PyTypeObject.tp_traverse` pointer is used by the garbage collector to detect
reference cycles. A typical implementation of a :c:member:`~PyTypeObject.tp_traverse` function
simply calls :c:func:`Py_VISIT` on each of the instance's members that are Python
objects that the instance owns. For example, this is function :c:func:`!local_traverse` from the
:mod:`!_thread` extension module::
static int
local_traverse(PyObject *op, visitproc visit, void *arg)
{
localobject *self = (localobject *) op;
Py_VISIT(self->args);
Py_VISIT(self->kw);
Py_VISIT(self->dict);
return 0;
}
Note that :c:func:`Py_VISIT` is called only on those members that can participate
in reference cycles. Although there is also a ``self->key`` member, it can only
be ``NULL`` or a Python string and therefore cannot be part of a reference cycle.
On the other hand, even if you know a member can never be part of a cycle, as a
debugging aid you may want to visit it anyway just so the :mod:`gc` module's
:func:`~gc.get_referents` function will include it.
Heap types (:c:macro:`Py_TPFLAGS_HEAPTYPE`) must visit their type with::
Py_VISIT(Py_TYPE(self));
It is only needed since Python 3.9. To support Python 3.8 and older, this
line must be conditional::
#if PY_VERSION_HEX >= 0x03090000
Py_VISIT(Py_TYPE(self));
#endif
If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
:c:member:`~PyTypeObject.tp_flags` field, the traverse function must call
:c:func:`PyObject_VisitManagedDict` like this::
PyObject_VisitManagedDict((PyObject*)self, visit, arg);
.. warning::
When implementing :c:member:`~PyTypeObject.tp_traverse`, only the
members that the instance *owns* (by having :term:`strong references
<strong reference>` to them) must be
visited. For instance, if an object supports weak references via the
:c:member:`~PyTypeObject.tp_weaklist` slot, the pointer supporting
the linked list (what *tp_weaklist* points to) must **not** be
visited as the instance does not directly own the weak references to itself
(the weakreference list is there to support the weak reference machinery,
but the instance has no strong reference to the elements inside it, as they
are allowed to be removed even if the instance is still alive).
.. warning::
The traversal function must not have any side effects. It must not
modify the reference counts of any Python objects nor create or destroy
any Python objects.
Note that :c:func:`Py_VISIT` requires the *visit* and *arg* parameters to
:c:func:`!local_traverse` to have these specific names; don't name them just
anything.
Instances of :ref:`heap-allocated types <heap-types>` hold a reference to
their type. Their traversal function must therefore either visit
:c:func:`Py_TYPE(self) <Py_TYPE>`, or delegate this responsibility by
calling ``tp_traverse`` of another heap-allocated type (such as a
heap-allocated superclass).
If they do not, the type object may not be garbage-collected.
.. note::
The :c:member:`~PyTypeObject.tp_traverse` function can be called from any
thread.
.. versionchanged:: 3.9
Heap-allocated types are expected to visit ``Py_TYPE(self)`` in
``tp_traverse``. In earlier versions of Python, due to
`bug 40217 <https://bugs.python.org/issue40217>`_, doing this
may lead to crashes in subclasses.
See :ref:`gc-traversal` for documentation.
**Inheritance:**

7
Doc/data/refcounts.dat
View file

@ -2430,10 +2430,17 @@ PyType_GetName:PyTypeObject*:type:0:
PyType_GetModule:PyObject*::0:
PyType_GetModule:PyTypeObject*:type:0:
PyType_GetModule_DuringGC:PyObject*::0:
PyType_GetModule_DuringGC:PyTypeObject*:type:0:
PyType_GetModuleByToken:PyObject*::+1:
PyType_GetModuleByToken:PyTypeObject*:type:0:
PyType_GetModuleByToken:PyModuleDef*:def::
PyType_GetModuleByToken_DuringGC:PyObject*::0:
PyType_GetModuleByToken_DuringGC:PyTypeObject*:type:0:
PyType_GetModuleByToken_DuringGC:PyModuleDef*:mod_token::
PyType_GetModuleByDef:PyObject*::0:
PyType_GetModuleByDef:PyTypeObject*:type:0:
PyType_GetModuleByDef:PyModuleDef*:def::

7
Doc/data/stable_abi.dat generated
View file

@ -495,7 +495,9 @@ func,PyModule_GetName,3.2,,
func,PyModule_GetNameObject,3.7,,
func,PyModule_GetState,3.2,,
func,PyModule_GetStateSize,3.15,,
func,PyModule_GetState_DuringGC,3.15,,
func,PyModule_GetToken,3.15,,
func,PyModule_GetToken_DuringGC,3.15,,
func,PyModule_New,3.2,,
func,PyModule_NewObject,3.7,,
func,PyModule_SetDocString,3.7,,
@ -598,6 +600,7 @@ func,PyObject_GetIter,3.2,,
func,PyObject_GetOptionalAttr,3.13,,
func,PyObject_GetOptionalAttrString,3.13,,
func,PyObject_GetTypeData,3.12,,
func,PyObject_GetTypeData_DuringGC,3.15,,
func,PyObject_HasAttr,3.2,,
func,PyObject_HasAttrString,3.2,,
func,PyObject_HasAttrStringWithError,3.13,,
@ -750,13 +753,17 @@ func,PyType_FromSpecWithBases,3.3,,
func,PyType_GenericAlloc,3.2,,
func,PyType_GenericNew,3.2,,
func,PyType_GetBaseByToken,3.14,,
func,PyType_GetBaseByToken_DuringGC,3.15,,
func,PyType_GetFlags,3.2,,
func,PyType_GetFullyQualifiedName,3.13,,
func,PyType_GetModule,3.10,,
func,PyType_GetModuleByDef,3.13,,
func,PyType_GetModuleByToken,3.15,,
func,PyType_GetModuleByToken_DuringGC,3.15,,
func,PyType_GetModuleName,3.13,,
func,PyType_GetModuleState,3.10,,
func,PyType_GetModuleState_DuringGC,3.15,,
func,PyType_GetModule_DuringGC,3.15,,
func,PyType_GetName,3.11,,
func,PyType_GetQualName,3.11,,
func,PyType_GetSlot,3.4,,

11
Doc/whatsnew/3.15.rst
View file

@ -1781,6 +1781,17 @@ New features
* Add :c:func:`PyTuple_FromArray` to create a :class:`tuple` from an array.
(Contributed by Victor Stinner in :gh:`111489`.)
* Add functions that are guaranteed to be safe for use in
:c:member:`~PyTypeObject.tp_traverse` handlers:
:c:func:`PyObject_GetTypeData_DuringGC`,
:c:func:`PyObject_GetItemData_DuringGC`,
:c:func:`PyType_GetModuleState_DuringGC`,
:c:func:`PyModule_GetState_DuringGC`, :c:func:`PyModule_GetToken_DuringGC`,
:c:func:`PyType_GetBaseByToken_DuringGC`,
:c:func:`PyType_GetModule_DuringGC`,
:c:func:`PyType_GetModuleByToken_DuringGC`.
(Contributed by Petr Viktorin in :gh:`145925`.)
* Add :c:func:`PyObject_Dump` to dump an object to ``stderr``.
It should only be used for debugging.
(Contributed by Victor Stinner in :gh:`141070`.)

1
Include/cpython/object.h
View file

@ -442,6 +442,7 @@ PyAPI_FUNC(void) _PyTrash_thread_destroy_chain(PyThreadState *tstate);
PyAPI_FUNC(void *) PyObject_GetItemData(PyObject *obj);
PyAPI_FUNC(void *) PyObject_GetItemData_DuringGC(PyObject *obj);
PyAPI_FUNC(int) PyObject_VisitManagedDict(PyObject *obj, visitproc visit, void *arg);
PyAPI_FUNC(void) PyObject_ClearManagedDict(PyObject *obj);

2
Include/internal/pycore_moduleobject.h
View file

@ -53,11 +53,13 @@ static inline PyModuleDef *_PyModule_GetDefOrNull(PyObject *arg) {
return NULL;
}
// Get md_token. Used in _DuringGC functions; must have no side effects.
static inline PyModuleDef *_PyModule_GetToken(PyObject *arg) {
PyModuleObject *mod = _PyModule_CAST(arg);
return (PyModuleDef *)mod->md_token;
}
// Get md_state. Used in _DuringGC functions; must have no side effects.
static inline void* _PyModule_GetState(PyObject* mod) {
return _PyModule_CAST(mod)->md_state;
}

2
Include/moduleobject.h
View file

@ -125,6 +125,8 @@ PyAPI_FUNC(PyObject *) PyModule_FromSlotsAndSpec(const PyModuleDef_Slot *slots,
PyAPI_FUNC(int) PyModule_Exec(PyObject *module);
PyAPI_FUNC(int) PyModule_GetStateSize(PyObject *module, Py_ssize_t *result);
PyAPI_FUNC(int) PyModule_GetToken(PyObject *module, void **result);
PyAPI_FUNC(void*) PyModule_GetState_DuringGC(PyObject*);
PyAPI_FUNC(int) PyModule_GetToken_DuringGC(PyObject *module, void **result);
#endif
#ifndef _Py_OPAQUE_PYOBJECT

8
Include/object.h
View file

@ -779,6 +779,14 @@ PyAPI_FUNC(int) PyType_Freeze(PyTypeObject *type);
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= _Py_PACK_VERSION(3, 15)
PyAPI_FUNC(PyObject *) PyType_GetModuleByToken(PyTypeObject *type,
const void *token);
PyAPI_FUNC(void *) PyObject_GetTypeData_DuringGC(PyObject *obj,
PyTypeObject *cls);
PyAPI_FUNC(void *) PyType_GetModuleState_DuringGC(PyTypeObject *);
PyAPI_FUNC(int) PyType_GetBaseByToken_DuringGC(PyTypeObject *,
void *, PyTypeObject **);
PyAPI_FUNC(PyObject *) PyType_GetModule_DuringGC(PyTypeObject *);
PyAPI_FUNC(PyObject *) PyType_GetModuleByToken_DuringGC(PyTypeObject *type,
const void *token);
#endif
#ifdef __cplusplus

7
Lib/test/test_stable_abi_ctypes.py generated
View file

@ -488,7 +488,9 @@ SYMBOL_NAMES = (
"PyModule_GetNameObject",
"PyModule_GetState",
"PyModule_GetStateSize",
"PyModule_GetState_DuringGC",
"PyModule_GetToken",
"PyModule_GetToken_DuringGC",
"PyModule_New",
"PyModule_NewObject",
"PyModule_SetDocString",
@ -586,6 +588,7 @@ SYMBOL_NAMES = (
"PyObject_GetOptionalAttr",
"PyObject_GetOptionalAttrString",
"PyObject_GetTypeData",
"PyObject_GetTypeData_DuringGC",
"PyObject_HasAttr",
"PyObject_HasAttrString",
"PyObject_HasAttrStringWithError",
@ -740,13 +743,17 @@ SYMBOL_NAMES = (
"PyType_GenericAlloc",
"PyType_GenericNew",
"PyType_GetBaseByToken",
"PyType_GetBaseByToken_DuringGC",
"PyType_GetFlags",
"PyType_GetFullyQualifiedName",
"PyType_GetModule",
"PyType_GetModuleByDef",
"PyType_GetModuleByToken",
"PyType_GetModuleByToken_DuringGC",
"PyType_GetModuleName",
"PyType_GetModuleState",
"PyType_GetModuleState_DuringGC",
"PyType_GetModule_DuringGC",
"PyType_GetName",
"PyType_GetQualName",
"PyType_GetSlot",

9
Misc/NEWS.d/next/C_API/2026-03-13-16-37-54.gh-issue-145921.ssA7HZ.rst Normal file
View file

@ -0,0 +1,9 @@
Add functions that are guaranteed to be safe for use in
:c:member:`~PyTypeObject.tp_traverse` handlers:
:c:func:`PyObject_GetTypeData_DuringGC`,
:c:func:`PyObject_GetItemData_DuringGC`,
:c:func:`PyType_GetModuleState_DuringGC`,
:c:func:`PyModule_GetState_DuringGC`, :c:func:`PyModule_GetToken_DuringGC`,
:c:func:`PyType_GetBaseByToken_DuringGC`,
:c:func:`PyType_GetModule_DuringGC`,
:c:func:`PyType_GetModuleByToken_DuringGC`.

14
Misc/stable_abi.toml
View file

@ -2666,6 +2666,20 @@
[function.Py_SET_SIZE]
# Before 3.15, this was a macro that accessed the PyObject member
added = '3.15'
[function.PyObject_GetTypeData_DuringGC]
added = '3.15'
[function.PyType_GetModuleState_DuringGC]
added = '3.15'
[function.PyModule_GetState_DuringGC]
added = '3.15'
[function.PyModule_GetToken_DuringGC]
added = '3.15'
[function.PyType_GetBaseByToken_DuringGC]
added = '3.15'
[function.PyType_GetModule_DuringGC]
added = '3.15'
[function.PyType_GetModuleByToken_DuringGC]
added = '3.15'
# PEP 757 import/export API.

18
Modules/_ctypes/_ctypes.c
View file

@ -468,11 +468,7 @@ class _ctypes.CType_Type "PyObject *" "clinic_state()->CType_Type"
static int
CType_Type_traverse(PyObject *self, visitproc visit, void *arg)
{
StgInfo *info = _PyStgInfo_FromType_NoState(self);
if (!info) {
PyErr_FormatUnraisable("Exception ignored while "
"calling ctypes traverse function %R", self);
}
StgInfo *info = _PyStgInfo_FromType_DuringGC(self);
if (info) {
Py_VISIT(info->proto);
Py_VISIT(info->argtypes);
@ -516,11 +512,7 @@ ctype_free_stginfo_members(StgInfo *info)
static int
CType_Type_clear(PyObject *self)
{
StgInfo *info = _PyStgInfo_FromType_NoState(self);
if (!info) {
PyErr_FormatUnraisable("Exception ignored while "
"clearing ctypes %R", self);
}
StgInfo *info = _PyStgInfo_FromType_DuringGC(self);
if (info) {
ctype_clear_stginfo(info);
}
@ -530,11 +522,7 @@ CType_Type_clear(PyObject *self)
static void
CType_Type_dealloc(PyObject *self)
{
StgInfo *info = _PyStgInfo_FromType_NoState(self);
if (!info) {
PyErr_FormatUnraisable("Exception ignored while "
"deallocating ctypes %R", self);
}
StgInfo *info = _PyStgInfo_FromType_DuringGC(self);
if (info) {
ctype_free_stginfo_members(info);
}

9
Modules/_ctypes/ctypes.h
View file

@ -614,15 +614,14 @@ PyStgInfo_FromAny(ctypes_state *state, PyObject *obj, StgInfo **result)
* state is torn down.
*/
static inline StgInfo *
_PyStgInfo_FromType_NoState(PyObject *type)
_PyStgInfo_FromType_DuringGC(PyObject *type)
{
PyTypeObject *PyCType_Type;
if (_PyType_GetBaseByToken_Borrow(Py_TYPE(type), &pyctype_type_spec, &PyCType_Type) < 0 ||
PyCType_Type == NULL) {
PyType_GetBaseByToken_DuringGC(Py_TYPE(type), &pyctype_type_spec, &PyCType_Type);
if (PyCType_Type == NULL) {
return NULL;
}
return PyObject_GetTypeData(type, PyCType_Type);
return PyObject_GetTypeData_DuringGC(type, PyCType_Type);
}
// Initialize StgInfo on a newly created type

24
Modules/_testcapi/heaptype.c
View file

@ -403,6 +403,7 @@ static PyObject *
pyobject_getitemdata(PyObject *self, PyObject *o)
{
void *pointer = PyObject_GetItemData(o);
assert(pointer == PyObject_GetItemData_DuringGC(o));
if (pointer == NULL) {
return NULL;
}
@ -485,17 +486,27 @@ pytype_getbasebytoken(PyObject *self, PyObject *args)
mro_save = type->tp_mro;
type->tp_mro = NULL;
}
void *token = PyLong_AsVoidPtr(py_token);
if (PyErr_Occurred()) {
return NULL;
}
void *result_duringgc;
int ret_duringgc = PyType_GetBaseByToken_DuringGC(
type, token, (PyTypeObject **)&result_duringgc);
assert(!PyErr_Occurred());
PyObject *result;
int ret;
if (need_result == Py_True) {
ret = PyType_GetBaseByToken(type, token, (PyTypeObject **)&result);
assert(result == result_duringgc);
}
else {
result = NULL;
ret = PyType_GetBaseByToken(type, token, NULL);
}
assert(ret == ret_duringgc);
if (use_mro != Py_True) {
type->tp_mro = mro_save;
@ -518,6 +529,7 @@ pytype_getbasebytoken(PyObject *self, PyObject *args)
error:
Py_XDECREF(py_ret);
Py_XDECREF(result);
assert(PyErr_Occurred());
return NULL;
}
@ -525,6 +537,7 @@ static PyObject *
pytype_getmodulebydef(PyObject *self, PyObject *type)
{
PyObject *mod = PyType_GetModuleByDef((PyTypeObject *)type, _testcapimodule);
assert(mod == PyType_GetModuleByToken_DuringGC((PyTypeObject *)type, _testcapimodule));
return Py_XNewRef(mod);
}
@ -540,7 +553,9 @@ pytype_getmodulebytoken(PyObject *self, PyObject *args)
if ((!token) && PyErr_Occurred()) {
return NULL;
}
return PyType_GetModuleByToken((PyTypeObject *)type, token);
PyObject *result = PyType_GetModuleByToken((PyTypeObject *)type, token);
assert(result == PyType_GetModuleByToken_DuringGC((PyTypeObject *)type, token));
return result;
}
static PyType_Slot HeapCTypeWithBasesSlotNone_slots[] = {
@ -820,6 +835,7 @@ heapctypesubclasswithfinalizer_finalize(PyObject *self)
PyObject *exc = PyErr_GetRaisedException();
PyObject *m = PyType_GetModule(Py_TYPE(self));
assert(m == PyType_GetModule_DuringGC(Py_TYPE(self)));
if (m == NULL) {
goto cleanup_finalize;
}
@ -1283,6 +1299,7 @@ HeapCCollection_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds)
goto finally;
}
PyObject **data = PyObject_GetItemData(self);
assert(data == PyObject_GetItemData_DuringGC(self));
if (!data) {
goto finally;
}
@ -1312,6 +1329,7 @@ HeapCCollection_item(PyObject *self, Py_ssize_t i)
return PyErr_Format(PyExc_IndexError, "index %zd out of range", i);
}
PyObject **data = PyObject_GetItemData(self);
assert(data == PyObject_GetItemData_DuringGC(self));
if (!data) {
return NULL;
}
@ -1322,6 +1340,7 @@ static int
HeapCCollection_traverse(PyObject *self, visitproc visit, void *arg)
{
PyObject **data = PyObject_GetItemData(self);
assert(data == PyObject_GetItemData_DuringGC(self));
if (!data) {
return -1;
}
@ -1335,6 +1354,7 @@ static int
HeapCCollection_clear(PyObject *self)
{
PyObject **data = PyObject_GetItemData(self);
assert(data == PyObject_GetItemData_DuringGC(self));
if (!data) {
return -1;
}

9
Modules/_testcapi/module.c
View file

@ -161,6 +161,8 @@ module_from_slots_token(PyObject *self, PyObject *spec)
return NULL;
}
assert(got_token == &test_token);
assert(PyModule_GetToken_DuringGC(mod, &got_token) >= 0);
assert(got_token == &test_token);
return mod;
}
@ -433,7 +435,12 @@ static PyObject *
pymodule_get_token(PyObject *self, PyObject *module)
{
void *token;
if (PyModule_GetToken(module, &token) < 0) {
int res = PyModule_GetToken(module, &token);
void *token_duringgc;
int res_duringgc = PyModule_GetToken_DuringGC(module, &token_duringgc);
assert(res == res_duringgc);
assert(token == token_duringgc);
if (res < 0) {
return NULL;
}
return PyLong_FromVoidPtr(token);

10
Modules/_testlimitedcapi/heaptype_relative.c
View file

@ -1,7 +1,7 @@
// Need limited C API version 3.12 for PyType_FromMetaclass()
// Need limited C API version 3.15 for _DuringGC functions
#include "pyconfig.h" // Py_GIL_DISABLED
#if !defined(Py_GIL_DISABLED) && !defined(Py_LIMITED_API)
# define Py_LIMITED_API 0x030c0000
# define Py_LIMITED_API 0x030f0000
#endif
#include "parts.h"
@ -55,6 +55,8 @@ make_sized_heaptypes(PyObject *module, PyObject *args)
goto finally;
}
char *data_ptr = PyObject_GetTypeData(instance, (PyTypeObject *)sub);
assert(data_ptr == PyObject_GetTypeData_DuringGC(instance,
(PyTypeObject *)sub));
if (!data_ptr) {
goto finally;
}
@ -80,6 +82,7 @@ var_heaptype_set_data_to_3s(
PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
void *data_ptr = PyObject_GetTypeData(self, defining_class);
assert(data_ptr == PyObject_GetTypeData_DuringGC(self, defining_class));
if (!data_ptr) {
return NULL;
}
@ -96,6 +99,7 @@ var_heaptype_get_data(PyObject *self, PyTypeObject *defining_class,
PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
void *data_ptr = PyObject_GetTypeData(self, defining_class);
assert(data_ptr == PyObject_GetTypeData_DuringGC(self, defining_class));
if (!data_ptr) {
return NULL;
}
@ -259,6 +263,7 @@ heapctypewithrelativedict_dealloc(PyObject* self)
{
PyTypeObject *tp = Py_TYPE(self);
HeapCTypeWithDictStruct *data = PyObject_GetTypeData(self, tp);
assert(data == PyObject_GetTypeData_DuringGC(self, tp));
Py_XDECREF(data->dict);
PyObject_Free(self);
Py_DECREF(tp);
@ -297,6 +302,7 @@ heapctypewithrelativeweakref_dealloc(PyObject* self)
{
PyTypeObject *tp = Py_TYPE(self);
HeapCTypeWithWeakrefStruct *data = PyObject_GetTypeData(self, tp);
assert(data == PyObject_GetTypeData_DuringGC(self, tp));
if (data->weakreflist != NULL) {
PyObject_ClearWeakRefs(self);
}

19
Modules/_testmultiphase.c
View file

@ -152,10 +152,13 @@ _testmultiphase_StateAccessType_get_defining_module_impl(StateAccessTypeObject *
{
PyObject *retval;
retval = PyType_GetModule(cls);
assert(retval == PyType_GetModule_DuringGC(cls));
if (retval == NULL) {
return NULL;
}
assert(PyType_GetModuleByDef(Py_TYPE(self), &def_meth_state_access) == retval);
assert(PyType_GetModuleByToken_DuringGC(Py_TYPE(self), &def_meth_state_access)
== retval);
return Py_NewRef(retval);
}
@ -172,9 +175,14 @@ _testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(StateAccessTypeObjec
PyTypeObject *cls)
/*[clinic end generated code: output=64509074dfcdbd31 input=edaff09aa4788204]*/
{
PyType_GetModuleByDef(Py_TYPE(self), &def_nonmodule); // should raise
// DuringGC: does not raise
assert(PyType_GetModuleByToken_DuringGC(Py_TYPE(self), &def_nonmodule) == NULL);
assert(!PyErr_Occurred());
// should raise:
PyObject *m = PyType_GetModuleByDef(Py_TYPE(self), &def_nonmodule);
assert(PyErr_Occurred());
return NULL;
assert(m == NULL);
return m;
}
/*[clinic input]
@ -200,6 +208,7 @@ _testmultiphase_StateAccessType_increment_count_clinic_impl(StateAccessTypeObjec
/*[clinic end generated code: output=3b34f86bc5473204 input=551d482e1fe0b8f5]*/
{
meth_state *m_state = PyType_GetModuleState(cls);
assert(m_state == PyType_GetModuleState_DuringGC(cls));
if (twice) {
n *= 2;
}
@ -249,6 +258,7 @@ _StateAccessType_increment_count_noclinic(PyObject *self,
n *= 2;
}
meth_state *m_state = PyType_GetModuleState(defining_class);
assert(m_state == PyType_GetModuleState_DuringGC(defining_class));
m_state->counter += n;
Py_RETURN_NONE;
@ -268,6 +278,7 @@ _testmultiphase_StateAccessType_get_count_impl(StateAccessTypeObject *self,
/*[clinic end generated code: output=64600f95b499a319 input=d5d181f12384849f]*/
{
meth_state *m_state = PyType_GetModuleState(cls);
assert(m_state == PyType_GetModuleState_DuringGC(cls));
return PyLong_FromLong(m_state->counter);
}
@ -889,6 +900,7 @@ meth_state_access_exec(PyObject *m)
meth_state *m_state;
m_state = PyModule_GetState(m);
assert(m_state == PyModule_GetState_DuringGC(m));
if (m_state == NULL) {
return -1;
}
@ -1158,6 +1170,7 @@ modexport_smoke_exec(PyObject *mod)
return 0;
}
int *state = PyModule_GetState(mod);
assert(state == PyModule_GetState_DuringGC(mod));
if (!state) {
return -1;
}
@ -1175,6 +1188,7 @@ static PyObject *
modexport_smoke_get_state_int(PyObject *mod, PyObject *arg)
{
int *state = PyModule_GetState(mod);
assert(state == PyModule_GetState_DuringGC(mod));
if (!state) {
return NULL;
}
@ -1204,6 +1218,7 @@ modexport_smoke_free(void *op)
{
PyObject *mod = (PyObject *)op;
int *state = PyModule_GetState(mod);
assert(state == PyModule_GetState_DuringGC(mod));
if (!state) {
PyErr_FormatUnraisable("Exception ignored in module %R free", mod);
}

1
Modules/_testsinglephase.c
View file

@ -254,6 +254,7 @@ get_module_state(PyObject *module)
}
else {
module_state *state = (module_state*)PyModule_GetState(module);
assert(state == PyModule_GetState_DuringGC(module));
assert(state != NULL);
return state;
}

20
Objects/moduleobject.c
View file

@ -910,6 +910,17 @@ PyModule_GetStateSize(PyObject *m, Py_ssize_t *size_p)
return 0;
}
int
PyModule_GetToken_DuringGC(PyObject *m, void **token_p)
{
*token_p = NULL;
if (!PyModule_Check(m)) {
return -1;
}
*token_p = _PyModule_GetToken(m);
return 0;
}
int
PyModule_GetToken(PyObject *m, void **token_p)
{
@ -1065,6 +1076,15 @@ PyModule_GetDef(PyObject* m)
return _PyModule_GetDefOrNull(m);
}
void*
PyModule_GetState_DuringGC(PyObject* m)
{
if (!PyModule_Check(m)) {
return NULL;
}
return _PyModule_GetState(m);
}
void*
PyModule_GetState(PyObject* m)
{

108
Objects/typeobject.c
View file

@ -5767,6 +5767,17 @@ PyType_GetSlot(PyTypeObject *type, int slot)
return *(void**)((char*)parent_slot + pyslot_offsets[slot].subslot_offset);
}
PyObject *
PyType_GetModule_DuringGC(PyTypeObject *type)
{
assert(PyType_Check(type));
if (!_PyType_HasFeature(type, Py_TPFLAGS_HEAPTYPE)) {
return NULL;
}
PyHeapTypeObject* et = (PyHeapTypeObject*)type;
return et->ht_module;
}
PyObject *
PyType_GetModule(PyTypeObject *type)
{
@ -5788,7 +5799,16 @@ PyType_GetModule(PyTypeObject *type)
return NULL;
}
return et->ht_module;
}
void *
PyType_GetModuleState_DuringGC(PyTypeObject *type)
{
PyObject *m = PyType_GetModule_DuringGC(type);
if (m == NULL) {
return NULL;
}
return _PyModule_GetState(m);
}
void *
@ -5801,19 +5821,18 @@ PyType_GetModuleState(PyTypeObject *type)
return _PyModule_GetState(m);
}
/* Return borrowed ref to the module of the first superclass where the module
* has the given token.
*/
static PyObject *
borrow_module_by_token(PyTypeObject *type, const void *token)
PyObject *
PyType_GetModuleByToken_DuringGC(PyTypeObject *type, const void *token)
{
assert(PyType_Check(type));
if (!_PyType_HasFeature(type, Py_TPFLAGS_HEAPTYPE)) {
// type_ready_mro() ensures that no heap type is
// contained in a static type MRO.
goto error;
return NULL;
}
else {
PyHeapTypeObject *ht = (PyHeapTypeObject*)type;
@ -5853,27 +5872,29 @@ borrow_module_by_token(PyTypeObject *type, const void *token)
}
END_TYPE_LOCK();
if (res != NULL) {
return res;
}
error:
PyErr_Format(
PyExc_TypeError,
"PyType_GetModuleByDef: No superclass of '%s' has the given module",
type->tp_name);
return NULL;
}
PyObject *
PyType_GetModuleByDef(PyTypeObject *type, PyModuleDef *def)
{
return borrow_module_by_token(type, def);
return res;
}
PyObject *
PyType_GetModuleByToken(PyTypeObject *type, const void *token)
{
return Py_XNewRef(borrow_module_by_token(type, token));
PyObject *mod = PyType_GetModuleByToken_DuringGC(type, token);
if (!mod) {
PyErr_Format(
PyExc_TypeError,
"PyType_GetModuleByDef: No superclass of '%s' has the given module",
type->tp_name);
return NULL;
}
return Py_NewRef(mod);
}
PyObject *
PyType_GetModuleByDef(PyTypeObject *type, PyModuleDef *def)
{
PyObject *mod = PyType_GetModuleByToken(type, def);
Py_XDECREF(mod); // return borrowed ref
return mod;
}
@ -5902,14 +5923,17 @@ get_base_by_token_recursive(PyObject *bases, void *token)
}
int
_PyType_GetBaseByToken_Borrow(PyTypeObject *type, void *token, PyTypeObject **result)
PyType_GetBaseByToken_DuringGC(PyTypeObject *type, void *token, PyTypeObject **result)
{
assert(token != NULL);
assert(PyType_Check(type));
if (result != NULL) {
*result = NULL;
}
if (token == NULL) {
return -1;
}
if (!PyType_Check(type)) {
return -1;
}
if (!_PyType_HasFeature(type, Py_TPFLAGS_HEAPTYPE)) {
// No static type has a heaptype superclass,
@ -5970,7 +5994,7 @@ PyType_GetBaseByToken(PyTypeObject *type, void *token, PyTypeObject **result)
return -1;
}
int res = _PyType_GetBaseByToken_Borrow(type, token, result);
int res = PyType_GetBaseByToken_DuringGC(type, token, result);
if (res > 0 && result) {
Py_INCREF(*result);
}
@ -5979,12 +6003,18 @@ PyType_GetBaseByToken(PyTypeObject *type, void *token, PyTypeObject **result)
void *
PyObject_GetTypeData(PyObject *obj, PyTypeObject *cls)
PyObject_GetTypeData_DuringGC(PyObject *obj, PyTypeObject *cls)
{
assert(PyObject_TypeCheck(obj, cls));
return (char *)obj + _align_up(cls->tp_base->tp_basicsize);
}
void *
PyObject_GetTypeData(PyObject *obj, PyTypeObject *cls)
{
return PyObject_GetTypeData_DuringGC(obj, cls);
}
Py_ssize_t
PyType_GetTypeDataSize(PyTypeObject *cls)
{
@ -5995,18 +6025,32 @@ PyType_GetTypeDataSize(PyTypeObject *cls)
return result;
}
void *
PyObject_GetItemData(PyObject *obj)
static inline void *
getitemdata(PyObject *obj, bool raise)
{
if (!PyType_HasFeature(Py_TYPE(obj), Py_TPFLAGS_ITEMS_AT_END)) {
PyErr_Format(PyExc_TypeError,
"type '%s' does not have Py_TPFLAGS_ITEMS_AT_END",
Py_TYPE(obj)->tp_name);
if (!_PyType_HasFeature(Py_TYPE(obj), Py_TPFLAGS_ITEMS_AT_END)) {
if (raise) {
PyErr_Format(PyExc_TypeError,
"type '%T' does not have Py_TPFLAGS_ITEMS_AT_END",
obj);
}
return NULL;
}
return (char *)obj + Py_TYPE(obj)->tp_basicsize;
}
void *
PyObject_GetItemData_DuringGC(PyObject *obj)
{
return getitemdata(obj, false);
}
void *
PyObject_GetItemData(PyObject *obj)
{
return getitemdata(obj, true);
}
/* Internal API to look for a name through the MRO, bypassing the method cache.
The result is stored as a _PyStackRef in `out`. It never set an exception.
Returns -1 if there was an error, 0 if the name was not found, and 1 if

7
PC/python3dll.c generated
View file

@ -437,8 +437,10 @@ EXPORT_FUNC(PyModule_GetFilenameObject)
EXPORT_FUNC(PyModule_GetName)
EXPORT_FUNC(PyModule_GetNameObject)
EXPORT_FUNC(PyModule_GetState)
EXPORT_FUNC(PyModule_GetState_DuringGC)
EXPORT_FUNC(PyModule_GetStateSize)
EXPORT_FUNC(PyModule_GetToken)
EXPORT_FUNC(PyModule_GetToken_DuringGC)
EXPORT_FUNC(PyModule_New)
EXPORT_FUNC(PyModule_NewObject)
EXPORT_FUNC(PyModule_SetDocString)
@ -523,6 +525,7 @@ EXPORT_FUNC(PyObject_GetIter)
EXPORT_FUNC(PyObject_GetOptionalAttr)
EXPORT_FUNC(PyObject_GetOptionalAttrString)
EXPORT_FUNC(PyObject_GetTypeData)
EXPORT_FUNC(PyObject_GetTypeData_DuringGC)
EXPORT_FUNC(PyObject_HasAttr)
EXPORT_FUNC(PyObject_HasAttrString)
EXPORT_FUNC(PyObject_HasAttrStringWithError)
@ -678,13 +681,17 @@ EXPORT_FUNC(PyType_FromSpecWithBases)
EXPORT_FUNC(PyType_GenericAlloc)
EXPORT_FUNC(PyType_GenericNew)
EXPORT_FUNC(PyType_GetBaseByToken)
EXPORT_FUNC(PyType_GetBaseByToken_DuringGC)
EXPORT_FUNC(PyType_GetFlags)
EXPORT_FUNC(PyType_GetFullyQualifiedName)
EXPORT_FUNC(PyType_GetModule)
EXPORT_FUNC(PyType_GetModule_DuringGC)
EXPORT_FUNC(PyType_GetModuleByDef)
EXPORT_FUNC(PyType_GetModuleByToken)
EXPORT_FUNC(PyType_GetModuleByToken_DuringGC)
EXPORT_FUNC(PyType_GetModuleName)
EXPORT_FUNC(PyType_GetModuleState)
EXPORT_FUNC(PyType_GetModuleState_DuringGC)
EXPORT_FUNC(PyType_GetName)
EXPORT_FUNC(PyType_GetQualName)
EXPORT_FUNC(PyType_GetSlot)