Commit graph

139 commits

Author SHA1 Message Date
Victor Stinner
d50838d84a gh-148675: Add Zd/Zf formats to array, ctypes, memoryview, struct 2026-04-17 14:01:37 +02:00
Bénédikt Tran
69e0a78e6e
gh-148390: fix undefined behavior of memoryview(...).cast("?") (#148454) 2026-04-15 11:42:20 +00:00
Serhiy Storchaka
473d2a35ce
gh-147944: Increase range of bytes_per_sep (GH-147946)
Accepted range for the bytes_per_sep argument of bytes.hex(),
bytearray.hex(), memoryview.hex(), and binascii.b2a_hex()
is now increased, so passing sys.maxsize and -sys.maxsize is now
valid.
2026-04-01 08:33:30 +00:00
sunmy2019
bbf7fb2c15
gh-146615: Fix format specifiers in Objects/ directory (GH-146620) 2026-03-31 10:59:48 +03:00
Sergey B Kirpichev
c68a19448a
gh-146151: memoryview supports 'F' and 'D' format types (complex) (#146241)
Co-authored-by: Victor Stinner <vstinner@python.org>
2026-03-26 15:26:34 +01:00
Victor Stinner
c9d123482a
gh-144995: Optimize memoryview == memoryview (#144996)
Optimize memoryview comparison: a memoryview is equal to itself, there is no
need to compare values, except if it uses float format.

Benchmark comparing 1 MiB:

    from timeit import timeit
    with open("/dev/random", 'br') as fp:
        data = fp.read(2**20)
    view = memoryview(data)
    LOOPS = 1_000
    b = timeit('x == x', number=LOOPS, globals={'x': data})
    m = timeit('x == x', number=LOOPS, globals={'x': view})
    print("bytes %f seconds" % b)
    print("mview %f seconds" % m)
    print("=> %f time slower" % (m / b))

Result before the change:

    bytes 0.000026 seconds
    mview 1.445791 seconds
    => 55660.873940 time slower

Result after the change:

    bytes 0.000026 seconds
    mview 0.000028 seconds
    => 1.104382 time slower

This missed optimization was discovered by Pierre-Yves David
while working on Mercurial.

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
2026-03-03 12:15:32 +01:00
Bénédikt Tran
84fcdbd86e
gh-142664: fix PyObject_Hash invokation post GH-143217 (#143223) 2025-12-27 14:30:09 +00:00
Bénédikt Tran
00e24b80e0
gh-142664: fix UAF in memoryview.__hash__ via re-entrant data's __hash__ (#143217) 2025-12-27 13:12:03 +00:00
Bénédikt Tran
9976c2b634
gh-143195: fix UAF in {bytearray,memoryview}.hex(sep) via re-entrant sep.__len__ (#143209) 2025-12-27 13:32:52 +01:00
Victor Stinner
703da5e81d
gh-129813, PEP 782: Use PyBytesWriter in memoryview (#138836)
Replace PyBytes_FromStringAndSize(NULL, size) with the new public
PyBytesWriter API.
2025-09-13 18:36:05 +02:00
Adam Turner
918e3ba6c0
GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
Eric Snow
2a28b21a51
gh-132776: Revert Moving memoryview XIData Code to memoryobject.c (gh-132960)
This is a partial revert of gh-132821.  It resolves the refleak introduced by that PR.
2025-04-25 16:43:50 +00:00
Eric Snow
e54e828852
gh-132776: Cleanup for XIBufferViewType (gh-132821)
* add notes
* rename XIBufferViewObject to xibufferview
* move memoryview XIData code to memoryobject.c
2025-04-24 18:25:29 -06:00
Victor Stinner
9d759b63d8
gh-111178: Change Argument Clinic signature for METH_O (#130682)
Use "PyObject*" for METH_O functions to fix an undefined behavior.
2025-03-11 16:33:36 +01:00
Victor Stinner
3bebe46d34
gh-128911: Add PyImport_ImportModuleAttr() function (#128912)
Add PyImport_ImportModuleAttr() and
PyImport_ImportModuleAttrString() functions.

* Add unit tests.
* Replace _PyImport_GetModuleAttr()
  with PyImport_ImportModuleAttr().
* Replace _PyImport_GetModuleAttrString()
  with PyImport_ImportModuleAttrString().
* Remove "pycore_import.h" includes, no longer needed.
2025-01-30 11:17:29 +00:00
Edward Xu
4937ba54c0
gh-127085: fix some data races in memoryview in free-threading (#127412) 2024-12-17 00:42:19 +05:30
Bénédikt Tran
4331832db0
gh-125420: implement Sequence.count API on memoryview objects (#125443) 2024-12-10 10:12:33 +00:00
Bénédikt Tran
58c753827a
gh-125420: implement Sequence.index API on memoryview objects (#125446) 2024-12-09 18:48:38 -08:00
Ritvik Pasham
a12690ef49
gh-126341: add release check to __iter__ method of memoryview (#126759)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: sobolevn <mail@sobolevn.me>
2024-11-13 11:51:01 +00:00
Brian Schubert
dc76a4ad3c
gh-126012: Add __class_getitem__ to memoryview (#126013)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2024-10-27 13:36:03 +03:00
Serhiy Storchaka
a1dbf2ea69
gh-77894: Fix a crash when the GC breaks a loop containing a memoryview (GH-123898)
Now a memoryview object can only be cleared if there are no buffers
that refer it.
2024-09-11 12:05:46 +03:00
Victor Stinner
d8e69b2c1b
gh-122854: Add Py_HashBuffer() function (#122855) 2024-08-30 15:42:27 +00:00
Victor Stinner
90b7540526
gh-120155: Fix copy/paste error in HAVE_SUBOFFSETS_IN_LAST_DIM() (#120228)
Don't hardcode 'dest' in HAVE_SUBOFFSETS_IN_LAST_DIM() macro of
memoryobject.c, but use its 'view' parameter instead.

Fix the Coverity issue:

Error: COPY_PASTE_ERROR (CWE-398):
Python-3.12.2/Objects/memoryobject.c:273:14: original: ""dest->suboffsets + (dest->ndim - 1)"" looks like the original copy.
Python-3.12.2/Objects/memoryobject.c:274:14: copy_paste_error: ""dest"" in ""src->suboffsets + (dest->ndim - 1)"" looks like a copy-paste error.
Python-3.12.2/Objects/memoryobject.c:274:14: remediation: Should it say ""src"" instead?
#  272|       assert(dest->ndim > 0 && src->ndim > 0);
#  273|       return (!HAVE_SUBOFFSETS_IN_LAST_DIM(dest) &&
#  274|->             !HAVE_SUBOFFSETS_IN_LAST_DIM(src) &&
#  275|               dest->strides[dest->ndim-1] == dest->itemsize &&
#  276|               src->strides[src->ndim-1] == src->itemsize);
2024-06-07 17:58:21 +02:00
Serhiy Storchaka
3a8c1ca7e7
gh-117764: Fix and add signatures for many builtins (GH-117769) 2024-04-12 13:56:41 +03:00
Christopher Chavez
f4fe65e2dd
gh-111178: Avoid calling functions from incompatible pointer types in memoryobject.c (GH-112863)
* Make memory_clear() compatible with inquiry
* Make memory_traverse() compatible with traverseproc
* Make memory_dealloc() compatible with destructor
* Make memory_repr() compatible with reprfunc
* Make memory_hash() compatible with hashfunc
* Make memoryiter_next() compatible with iternextfunc
* Make memoryiter_traverse() compatible with traverseproc
* Make memoryiter_dealloc() compatible with destructor
* Make several functions compatible with getter
* Make a few functions compatible with getter
* Make memory_item() compatible with ssizeargfunc
* Make memory_subscript() compatible with binaryfunc
* Make memory_length() compatible with lenfunc
* Make memory_ass_sub() compatible with objobjargproc
* Make memory_releasebuf() compatible with releasebufferproc
* Make memory_getbuf() compatible with getbufferproc
* Make mbuf_clear() compatible with inquiry
* Make mbuf_traverse() compatible with traverseproc
* Make mbuf_dealloc() compatible with destructor
2023-12-11 17:43:07 +01:00
Victor Stinner
773b803c02
gh-106320: Remove private _PyManagedBuffer_Type (#108431)
Remove private _PyManagedBuffer_Type variable. Move it to the
internal C API and no longer export it.
2023-08-24 19:07:54 +02:00
Jelle Zijlstra
405eacc1b8
gh-104223: Fix issues with inheriting from buffer classes (#104227)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-08 09:52:41 -07:00
Jelle Zijlstra
04f6733275
gh-102500: Implement PEP 688 (#102521)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-04 07:59:46 -07:00
Eric Wieser
3d2a46845b
gh-83791: Raise TypeError for len(memoryview_0d) (#18463)
Changes the behaviour of `len` on a zero-dimensional `memoryview` to raise `TypeError`. Previously, `len` would return `1`.
2023-04-22 17:32:47 +01:00
Victor Stinner
3a1dde8f29
gh-99300: Use Py_NewRef() in Objects/ directory (#99354)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 23:58:07 +01:00
Dong-hee Na
8d75a13fde
gh-90751: memoryview now supports half-float (#96738)
Co-authored-by: Antoine Pitrou <antoine@python.org>
2022-09-10 22:44:10 +02:00
Ken Jin
11190c4ad0
gh-92888: Fix memoryview bad __index__ use after free (GH-92946)
Co-authored-by: chilaxan <35645806+chilaxan@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <3659035+serhiy-storchaka@users.noreply.github.com>
2022-06-17 23:14:53 +08:00
Serhiy Storchaka
6fd4c8ec77
gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.

Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
2022-06-14 07:15:26 +03:00
Kumar Aditya
d923fdf54b
GH-92804: Fix memory leak in memoryview iterator (gh-92805) 2022-05-14 23:24:20 +09:00
Victor Stinner
760349198d
bpo-46670: Remove unused macros in the Objects directory (GH-31193) 2022-02-07 16:21:41 +01:00
David Hewitt
31ff96712e
bpo-46140: take more Py_buffer arguments as const * (GH-30217) 2021-12-22 15:07:46 +02:00
Victor Stinner
bbe7497c5a
bpo-45434: Remove pystrhex.h header file (GH-28923)
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.

The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:

* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
2021-10-13 15:22:35 +02:00
Mark Shannon
069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
dxflores
b0ac5d75a5
bpo-41732: add iterator to memoryview (GH-22119) 2020-09-08 16:28:45 +09:00
Serhiy Storchaka
80a50368c0
bpo-41262: Convert memoryview to Argument Clinic. (GH-21421) 2020-07-18 11:12:05 +03:00
Serhiy Storchaka
5f4b229df7
bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443)
Previously, the result could have been an instance of a subclass of int.

Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.

Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
2020-05-28 10:33:45 +03:00
Victor Stinner
d9ea5cae1d
bpo-40268: Remove unused pycore_pymem.h includes (GH-19531) 2020-04-15 02:57:50 +02:00
Victor Stinner
e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner
a15e260b70
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
Add _PyIndex_Check() function to the internal C API: fast inlined
verson of PyIndex_Check().

Add Include/internal/pycore_abstract.h header file.

Replace PyIndex_Check() with _PyIndex_Check() in C files of Objects
and Python subdirectories.
2020-04-08 02:01:56 +02:00
Stefan Krah
ee3bac4cba
Give proper credits for the memoryview implementation. (#18626) 2020-02-24 11:15:26 +01:00
Andy Lester
e6be9b59a9
closes bpo-39605: Fix some casts to not cast away const. (GH-18453)
gcc -Wcast-qual turns up a number of instances of casting away constness of pointers. Some of these can be safely modified, by either:

Adding the const to the type cast, as in:

-    return _PyUnicode_FromUCS1((unsigned char*)s, size);
+    return _PyUnicode_FromUCS1((const unsigned char*)s, size);

or, Removing the cast entirely, because it's not necessary (but probably was at one time), as in:

-    PyDTrace_FUNCTION_ENTRY((char *)filename, (char *)funcname, lineno);
+    PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno);

These changes will not change code, but they will make it much easier to check for errors in consts
2020-02-11 18:28:35 -08:00
Petr Viktorin
ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner
47ee8a6063
bpo-38631: Avoid Py_FatalError() in _memory_release() (GH-18214)
If the export count is negative, _memory_release() now raises a
SystemError and returns -1, rather than calling Py_FatalError()
which aborts the process.
2020-01-27 22:37:44 +01:00
Jeroen Demeyer
196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Jeroen Demeyer
530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00