[3.11] gh-99952: [ctypes] fix refcount issues in from_param() result. (GH-100169)
Fixes a reference counting issue with `ctypes.Structure` when a `from_param()` method call is used and the structure size is larger than a C pointer `sizeof(void*)`.
This problem existed for a very long time, but became more apparent in 3.8+ by change likely due to garbage collection cleanup timing changes..
(cherry picked from commit dfad678d70)
(cherry picked from commit fa7c37af49)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Yukihiro Nakadaira <yukihiro.nakadaira@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
gh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (GH-5576)
The itemsize returned in a memoryview of a ctypes array is now computed from the item type, instead of dividing the total size by the length and assuming that the length is not zero.
(cherry picked from commit 84bc6a4f25)
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
Previously, when built on older macOS systems, `find_library` was not able to find macOS system libraries when running on Big Sur due to changes in how system libraries are stored.
(cherry picked from commit 71853a7302)
Co-authored-by: Tobias Bergkvist <tobias@bergkv.ist>
From the commit message:
> When the structure is packed we should always expand when needed,
> otherwise we will add some padding between the fields. This patch makes
> sure we always merge bitfields together. It also changes the field merging
> algorithm so that it handles bitfields correctly.
Automerge-Triggered-By: GH:jaraco
No longer use deprecated aliases to functions:
* Replace PyObject_MALLOC() with PyObject_Malloc()
* Replace PyObject_REALLOC() with PyObject_Realloc()
* Replace PyObject_FREE() with PyObject_Free()
* Replace PyObject_Del() with PyObject_Free()
* Replace PyObject_DEL() with PyObject_Free()
Explicitly cast PyExc_Exception to PyTypeObject* to fix the warning:
modules\_ctypes\_ctypes.c(5748): warning C4133: '=':
incompatible types - from 'PyObject *' to '_typeobject *'
It is no longer possible to build the _ctypes extension module
without wchar_t type: remove CTYPES_UNICODE macro. Anyway, the
wchar_t type is required to build Python.
Fix reference leaks in the error path of the initialization function
the _ctypes extension module: call Py_DECREF(mod) on error.
Change PyCFuncPtr_Type name from _ctypes.PyCFuncPtr to
_ctypes.CFuncPtr to be consistent with the name exposed in the
_ctypes namespace (_ctypes.CFuncPtr).
Split PyInit__ctypes() function into sub-functions and add macros for
readability.
Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com>
* Add support for macOS 11 and Apple Silicon (aka arm64)
As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy
* Support building on recent versions of macOS while deploying to older versions
This allows building installers on macOS 11 while still supporting macOS 10.9.
Use _PyLong_GetZero() and _PyLong_GetOne() in Modules/ directory.
_cursesmodule.c and zoneinfo.c are now built with
Py_BUILD_CORE_MODULE macro defined.
If PyDict_GetItemWithError is only used to check whether the key is in dict,
it is better to use PyDict_Contains instead.
And if it is used in combination with PyDict_SetItem, PyDict_SetDefault can
replace the combination.
On Windows, #include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.
PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.
Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
* Rename pycore_byteswap.h to pycore_bitutils.h.
* Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
* _Py_popcount32() uses GCC and clang builtin function if available.
* Add unit tests to _Py_popcount32().
If ctypes fails to convert the result of a callback or if a ctypes
callback function raises an exception, sys.unraisablehook is now
called with an exception set. Previously, the error was logged into
stderr by PyErr_Print().
ctypes now raises an ArgumentError when a callback
is invoked with more than 1024 arguments.
The ctypes module allocates arguments on the stack in
ctypes_callproc() using alloca(), which is problematic
when large numbers of arguments are passed. Instead
of a stack overflow, this commit raises an ArgumentError
if more than 1024 parameters are passed.
Add a new internal pycore_byteswap.h header file with the following
functions:
* _Py_bswap16()
* _Py_bswap32()
* _Py_bswap64()
Use these functions in _ctypes, sha256 and sha512 modules,
and also use in the UTF-32 encoder.
sha256, sha512 and _ctypes modules are now built with the internal
C API.
The bulk of this patch was generated automatically with:
for name in \
PyObject_Vectorcall \
Py_TPFLAGS_HAVE_VECTORCALL \
PyObject_VectorcallMethod \
PyVectorcall_Function \
PyObject_CallOneArg \
PyObject_CallMethodNoArgs \
PyObject_CallMethodOneArg \
;
do
echo $name
git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
done
old=_PyObject_FastCallDict
new=PyObject_VectorcallDict
git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"
and then cleaned up:
- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
Although the underlying libffi issue remains open, adding these
checks have caused problems in third-party projects which are in
widespread use. See the issue for examples.
The corresponding tests have also been skipped.
