Commit graph

666 commits

Author SHA1 Message Date
Serhiy Storchaka
c98600bed4
gh-111789: Use PyDict_GetItemRef() in _ctypes (GH-111828) 2023-11-14 11:28:34 +02:00
Serhiy Storchaka
26c0e5e03a
gh-108082: Remove _PyErr_WriteUnraisableMsg() (GH-111643)
Replace the remaining calls with PyErr_FormatUnraisable().
2023-11-03 09:45:53 +02:00
Serhiy Storchaka
970e719a7a
gh-108082: Use PyErr_FormatUnraisable() (GH-111580)
Replace most of calls of _PyErr_WriteUnraisableMsg() and some
calls of PyErr_WriteUnraisable(NULL) with PyErr_FormatUnraisable().

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-11-02 09:16:34 +00:00
Victor Stinner
ef83b3fc00
gh-110968: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED was added to 3.12 (#111584) 2023-11-01 04:24:33 +01:00
Sam Gross
6dfb8fe023
gh-110481: Implement biased reference counting (gh-110764) 2023-10-30 16:06:09 +00:00
Serhiy Storchaka
59ea0f523e
gh-110093: Replace trivial Py_BuildValue() with direct C API call (GH-110094) 2023-10-20 18:08:41 +03:00
Victor Stinner
2ba6f68890
gh-85283: Fix _ctypes_test build on Windows in release mode (#111005)
Define Py_BUILD_CORE to not attempt to link the extension to
python3.lib (which fails).
2023-10-17 23:45:24 +02:00
Victor Stinner
be5e8a0103
gh-110964: Remove private _PyArg functions (#110966)
Move the following private functions and structures to
pycore_modsupport.h internal C API:

* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()

Changes:

* Python/getargs.h now includes pycore_modsupport.h to export
  functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
  is used.
* Add pycore_modsupport.h includes when a C extension uses one of
  these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
  directly or indirectly (via code generated by Argument Clinic)
  pycore_modsupport.h:

  * _csv
  * _curses_panel
  * _dbm
  * _gdbm
  * _multiprocessing.posixshmem
  * _sqlite.row
  * _statistics
  * grp
  * resource
  * syslog

* _testcapi: bad_get() no longer uses METH_FASTCALL calling
  convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
  PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
  by _testcapi sub-modules which need the internal C API
  (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
  watchers.c.
* Remove Include/cpython/modsupport.h header file.
  Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
2023-10-17 14:30:31 +02:00
Victor Stinner
db15fc23c6
gh-110968: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED new in 3.13. (#110969)
* Only add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED to limited C API
  version 3.13.
* errno, xxlimited and _ctypes_test extensions now need the limited C
  API version 3.13 to get Py_MOD_PER_INTERPRETER_GIL_SUPPORTED.  They
  now include standard header files explicitly: <errno.h>, <string.h>
  and <stdio.h>.
* xxlimited_35: Remove Py_mod_multiple_interpreters slot,
  incompatible with limited C API version 3.5.
2023-10-17 12:27:16 +02:00
Victor Stinner
37bd8726b8
gh-85283: Build errno and _ctypes_test with limited C API (#110955)
_testimportmultiple is now built with limited C API version 3.2.
2023-10-17 01:05:20 +00:00
Serhiy Storchaka
add16f1a5e
gh-108511: Add C API functions which do not silently ignore errors (GH-109025)
Add the following functions:

* PyObject_HasAttrWithError()
* PyObject_HasAttrStringWithError()
* PyMapping_HasKeyWithError()
* PyMapping_HasKeyStringWithError()
2023-09-17 14:23:31 +03:00
Nikita Sobolev
47af188593
Add missing PyDoc_STR calls (#109393)
In files:

* Modules/_ctypes/cfield.c
* Modules/_struct.c
* Objects/dictobject.c
* Objects/typevarobject.c
* Objects/unionobject.c
2023-09-15 15:10:48 +02:00
Victor Stinner
676593859e
gh-106320: Remove private _PyErr_WriteUnraisableMsg() (#108863)
Move the private _PyErr_WriteUnraisableMsg() functions to the
internal C API (pycore_pyerrors.h).

Move write_unraisable_exc() from _testcapi to _testinternalcapi.
2023-09-05 01:54:55 +02:00
Victor Stinner
5a79d2ae57
Revert "gh-46376: Return existing pointer when possible in ctypes (#1… (#108688)
This reverts commit 08447b5deb.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626646.
2023-09-04 11:21:47 +02:00
Victor Stinner
594b00057e
gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
Victor Stinner
ad73674283
gh-107603: Argument Clinic: Only include pycore_gc.h if needed (#108726)
Argument Clinic now only includes pycore_gc.h if PyGC_Head is needed,
and only includes pycore_runtime.h if _Py_ID() is needed.

* Add 'condition' optional argument to Clinic.add_include().
* deprecate_keyword_use() includes pycore_runtime.h when using
  the _PyID() function.
* Fix rendering of includes: comments start at the column 35.
* Mark PC/clinic/_wmimodule.cpp.h and
  "Objects/stringlib/clinic/*.h.h" header files as generated in
  .gitattributes.

Effects:

* 42 header files generated by AC no longer include the internal C
  API, instead of 4 header files before. For example,
  Modules/clinic/_abc.c.h no longer includes the internal C API.
* Fix _testclinic_depr.c.h: it now always includes pycore_runtime.h
  to get _Py_ID().
2023-08-31 23:42:34 +02:00
Victor Stinner
546cab8444
gh-106320: Remove private _PyTraceback functions (#108453)
Move private functions to the internal C API (pycore_traceback.h):

* _Py_DisplaySourceLine()
* _PyTraceback_Add()
2023-08-24 23:35:47 +00:00
Victor Stinner
b32d4cad15
gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
Change generated by the command:

sed -i -e 's!_PyLong_AsInt!PyLong_AsInt!g' \
    $(find -name "*.c" -o -name "*.h")
2023-08-25 01:01:30 +02:00
Victor Stinner
26893016a7
gh-106320: Remove private _PyDict functions (#108449)
Move private functions to the internal C API (pycore_dict.h):

* _PyDictView_Intersect()
* _PyDictView_New()
* _PyDict_ContainsId()
* _PyDict_DelItemId()
* _PyDict_DelItem_KnownHash()
* _PyDict_GetItemIdWithError()
* _PyDict_GetItem_KnownHash()
* _PyDict_HasSplitTable()
* _PyDict_NewPresized()
* _PyDict_Next()
* _PyDict_Pop()
* _PyDict_SetItemId()
* _PyDict_SetItem_KnownHash()
* _PyDict_SizeOf()

No longer export most of these functions.

Move also the _PyDictViewObject structure to the internal C API.

Move dict_getitem_knownhash() function from _testcapi to the
_testinternalcapi extension. Update test_capi.test_dict for this
change.
2023-08-24 20:01:50 +00:00
Victor Stinner
6726626646
gh-108314: Add PyDict_ContainsString() function (#108323) 2023-08-24 15:59:12 +02:00
Kostya Farber
0f2fb6efb4
gh-107689: Add docstring to ctypes.Array (#107697) 2023-08-10 06:39:14 +00:00
Tomas R
de72677f8a
gh-107659: Add docstrings for ctypes.pointer and ctypes.POINTER (#107660) 2023-08-08 08:20:10 +00:00
Tomas R
62a3a15119
gh-107455: ctypes: Improve error messages when converting to an incompatible type (#107456) 2023-08-03 10:59:03 +05:30
Konstantin
08447b5deb
gh-46376: Return existing pointer when possible in ctypes (#107131) 2023-07-31 09:10:53 +02:00
Victor Stinner
1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Serhiy Storchaka
a293fa5915
gh-86493: Use PyModule_Add() instead of PyModule_AddObjectRef() (GH-106860) 2023-07-18 23:59:53 +03:00
Serhiy Storchaka
be1b968dc1
gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) 2023-07-12 08:57:10 +03:00
Inada Naoki
d5bd32fb48
gh-104922: remove PY_SSIZE_T_CLEAN (#106315) 2023-07-02 15:07:46 +09:00
Victor Stinner
48d107a87d
gh-105927: _ctypes use PyWeakref_GetRef() (#105964)
Rename PyDict_GetItemProxy() to _PyDict_GetItemProxy() and mark it as
static. _PyDict_GetItemProxy() now returns a strong reference,
instead of a borrowed reference: replace PyWeakref_GET_OBJECT() with
_PyWeakref_GET_REF().
2023-06-21 16:34:27 +02:00
Victor Stinner
fb655e0c45
_ctypes callbacks.c uses _Py_COMP_DIAG_IGNORE_DEPR_DECLS (#105732)
Replace #pragma with _Py_COMP_DIAG_PUSH,
_Py_COMP_DIAG_IGNORE_DEPR_DECLS and _Py_COMP_DIAG_POP to ease Python
maintenance. Also add a comment explaining why callbacks.c ignores a
deprecation warning.
2023-06-14 12:12:25 +02:00
Erlend E. Aasland
e8998e46a7
gh-105375: Improve error handling in _ctypes (#105593)
Prevent repeated PyLong_FromVoidPtr() from possibly overwriting the
current exception.
2023-06-11 19:46:19 +00:00
Kirill Podoprigora
2cd1c87d2a
gh-104456: Fix ref leak in _ctypes.COMError (#104457) 2023-05-14 21:05:35 +00:00
Hugo van Kemenade
d513ddee94
Trim trailing whitespace and test on CI (#104275)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-08 17:03:52 +03:00
Eric Snow
a9c6e0618f
gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)
Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules.  We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
2023-05-05 21:11:27 +00:00
Erlend E. Aasland
4f524da484
gh-103092: Port _ctypes.COMError to heap type (#104020) 2023-05-04 13:03:24 +00:00
chgnrdv
2a884ceb36
Fix typo in "expected" word in few source files (#104034) 2023-05-01 09:45:50 -06:00
Erlend E. Aasland
e9c777208f
gh-103092: Isolate _ctypes, part 1 (#103893)
Establish global state and port the following types to heap types:

- DictRemover_Type
- PyCArg_Type
- PyCThunk_Type
- PyCField_Type
- StructParam_Type
2023-04-27 14:57:54 +02:00
Irit Katriel
2999e02836
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives in _ctypes (#102477) 2023-03-10 14:32:32 +05:30
Mark Dickinson
46416b9004
gh-76961: Fix buildbot failures in test_pep3118 (#101587)
This PR fixes the buildbot failures introduced by the merge of #5561, by restricting the relevant tests to something that should work on both 32-bit and 64-bit platforms. It also silences some compiler warnings introduced in that PR.
2023-02-06 12:25:31 +00:00
Eric Wieser
90d85a9b41
gh-76961: Fix the PEP3118 format string for ctypes.Structure (#5561)
The summary of this diff is that it:

* adds a `_ctypes_alloc_format_padding` function to append strings like `37x` to a format string to indicate 37 padding bytes
* removes the branches that amount to "give up on producing a valid format string if the struct is packed"
* combines the resulting adjacent `if (isStruct) {`s now that neither is `if (isStruct && !isPacked) {`
* invokes `_ctypes_alloc_format_padding` to add padding between structure fields, and after the last structure field. The computation used for the total size is unchanged from ctypes already used.

This patch does not affect any existing aligment computation; all it does is use subtraction to deduce the amount of paddnig introduced by the existing code.

---

Without this fix, it would never include padding bytes - an assumption that was only
valid in the case when `_pack_` was set - and this case was explicitly not implemented.

This should allow conversion from ctypes structs to numpy structs

Fixes https://github.com/numpy/numpy/issues/10528
2023-02-05 17:10:53 +00:00
Yukihiro Nakadaira
dfad678d70
gh-99952: [ctypes] fix refcount issues in from_param() result. (#100169)
Fixes a reference counting issue with `ctypes.Structure` when a `from_param()` method call is used and the structure size is larger than a C pointer `sizeof(void*)`.

This problem existed for a very long time, but became more apparent in 3.8+ by change likely due to garbage collection cleanup timing changes.
2023-01-26 00:28:34 -08:00
Kamil Turek
b4e11a7985
gh-99266: ctypes: Preserve more detailed exception in ArgumentError
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-01-21 19:14:43 +05:30
Cristián Maureira-Fredes
5e9df888dd
Fix ctypes typo in set_exception (#101040) 2023-01-15 20:42:37 +05:30
dgelessus
837ba05267
GH-81061: Fix refcount issue when returning None from a ctypes.py_object callback (#13364) 2023-01-09 15:43:04 +05:30
Zachary Ware
2df82db485
gh-100540: Remove obsolete Modules/_ctypes/darwin/ dlfcn shim (GH-100541)
As far as I can tell, this hasn't been actually used since Mac OS X 10.2.
2022-12-29 16:13:28 -06:00
Zachary Ware
025b5c3780
gh-100540: Remove unused Modules/_ctypes/libffi_osx/ (GH-100543)
It was an ancient, modified copy of libffi that has not been in use
since GH-22855.
2022-12-28 13:38:52 -06:00
Zachary Ware
fd4b55ff17
gh-100540: Remove unnecessary '-DMACOSX' for ctypes on macOS (GH-100542)
The define was only used to protect a `#pragma clang diagnostic`
setting, which is already better guarded by `__clang__` anwyay.
2022-12-28 12:06:58 -06:00
Eric Wieser
84bc6a4f25
gh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (GH-5576)
The itemsize returned in a memoryview of a ctypes array is now computed from the item type, instead of dividing the total size by the length and assuming that the length is not zero.
2022-12-23 08:23:19 +00:00
Hai Shi
d713c54ac8
gh-78878: Fix crash when creating an instance of _ctypes.CField (#14837) 2022-12-21 23:01:17 +05:30
Ronald Oussoren
cdde29dde9
gh-99337: Fix compile errors with gcc 12 on macOS (#99470)
Fix a number of compile errors with GCC-12 on macOS:

1. In pylifecycle.c the compile rejects _Pragma within a declaration
2. posixmodule.c was missing a number of ..._RUNTIME macros for non-clang on macOS
3. _ctypes assumed that __builtin_available is always present on macOS
2022-11-21 10:50:20 +01:00