Commit graph

14955 commits

Author SHA1 Message Date
Irit Katriel
082dbf7788
gh-133395: add option for extension modules to specialize BINARY_OP/SUBSCR, apply to arrays (#133396) 2025-05-05 17:46:56 +01:00
Wulian233
b6c2ef0c7a
gh-131296: Avoid posixmodule.c warning (GH-133142)
Clang-cl detects that on 32-bit builds the variable is always smaller than the value. But since the same code is used for other architectures, we can't just _fix_ it. This cast avoids the tautological-constant-out-of-range-compare warning.
2025-05-05 17:45:15 +01:00
Stan Ulbrych
f5b784741d
gh-110067: Make max heap methods public and add missing ones (GH-130725)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-05-05 17:52:49 +02:00
Sergey B Kirpichev
1bc16504ef
gh-61103: drop unused Py_HAVE_C_COMPLEX define (GH-133435)
Py_HAVE_C_COMPLEX was added in 3.14 so the removal doesn't need a deprecation
period even under a strict reading of PEP 387.

The Py_FFI_SUPPORT_C_COMPLEX check configure check implies support for
complex types in ctypes.
2025-05-05 15:50:57 +02:00
Sergey B Kirpichev
8d0e07eb89
gh-61103: don't use C _Complex types to implement F/D/G in ctypes (GH-133237)
According to the C standard, the memory representation of _Complex types
is equivalent to 2-element arrays. Unlike _Complex, arrays are always available.

- drop _complex.h header
- use appropriate real arrays to replace complex types

Co-authored-by: Lisandro Dalcin <dalcinl@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
2025-05-05 11:23:30 +02:00
Adam Turner
3f80165a26
GH-91048: Minor fixes for `_remotedebugging & rename to _remote_debugging` (#133398) 2025-05-05 02:30:14 +02:00
Pablo Galindo Salgado
3109c47be8
gh-91048: Add filename and line number to external inspection routines (GH-133385)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2025-05-04 23:33:37 +02:00
Serhiy Storchaka
5a57248b22
gh-81793: Always call linkat() from os.link(), if available (GH-132517)
This fixes os.link() on platforms (like Linux and OpenIndiana) where the
system link() function does not follow symlinks.

* On Linux, it now follows symlinks by default and if
  follow_symlinks=True is specified.
* On Windows, it now raises error if follow_symlinks=True is passed.
* On macOS, it now raises error if follow_symlinks=False is passed and
  the system linkat() function is not available at runtime.
* On other platforms, it now raises error if follow_symlinks is passed
  with a value that does not match the system link() function behavior
  if if the behavior is not known.

Co-authored-by: Joachim Henke <37883863+jo-he@users.noreply.github.com>
Co-authored-by: Thomas Kluyver <takowl@gmail.com>
2025-05-04 17:24:10 +03:00
Serhiy Storchaka
e9253ebf74
gh-122559: Synchronize C and Python implementation of the io module about pickling (GH-122628)
In the C implementation, remove __reduce__ and __reduce_ex__ methods
that always raise TypeError and restore __getstate__ methods that always
raise TypeErrori.

This restores fine details of the pre-3.12 behavior and unifies
both implementations.
2025-05-04 14:06:49 +00:00
Michael Forney
0af61fe2f4
bpo-44172: Keep reference to original window in curses subwindow objects (GH-26226)
The X/Open curses specification[0] and ncurses documentation[1]
both state that subwindows must be deleted before the main window.

Deleting the windows in the wrong order causes a double-free with
NetBSD's curses implementation.

To fix this, keep track of the original window object in the subwindow
object, and keep a reference to the original for the lifetime of
the subwindow.

[0] https://pubs.opengroup.org/onlinepubs/7908799/xcurses/delwin.html
[1] https://invisible-island.net/ncurses/man/curs_window.3x.html

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-05-04 13:29:44 +03:00
Emma Smith
3b4333583f
gh-132983: Introduce _zstd bindings module (GH-133027)
* Add _zstd module for https://peps.python.org/pep-0784/

This commit introduces the `_zstd` module, with bindings to libzstd from
the pyzstd project. It also includes the unix build system configuration.
Windows build system support will be integrated independently as it
depends on integration with cpython-source-deps.

* Add _zstd to modules

* Fix path for compression.zstd module

* Ignore _zstd module like _io

* Expand module state macros to improve code quality

Also removes module state references from the classes in the _zstd
module and instead uses PyType_GetModuleState()

* Remove backticks suggested in review

Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

* Use critical sections to lock object state

This should avoid races and deadlocks.

* Remove compress/decompress and mark module as not reliant on the GIL

The `compress`/`decompress` functions will be moved to Python code for simplicity.
C implementations can always be re-added in the future.

Also, mark _zstd as not requiring the GIL.

* Lift critical section to avoid clang warning

* Respond to comments by picnixz

* Call out pyzstd explicitly in license description

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

* Use a much more robust implementation...

... for `get_zstd_state_from_type`

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Use PyList_GetItemRef for thread safety purposes

* Use a macro for the minimum supported version

* remove const from primivite types

* Use PyMem_New in another spot

* Simplify error handling in _get_frame_size

* Another simplification of error handling in get_frame_info

* Rename _module_state to mod_state

* Rewrite comment explaining the context of the code

* Add link to pyzstd

* Add TODO about refactoring dict training code

* Use PyModule_AddObjectRef over PyModule_AddObject

PyModule_AddObject is soft-deprecated, so we should use PyModule_AddObjectRef

* Check result of OutputBufferGrow

* Simplify return logic in `add_constant_to_type`

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Ignore return value of _zstd_clear()

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Remove redundant comments

* Remove __reduce__ from ZstdDict

We should instead document that to pickle a dictionary a user should use
the `.dict_content` attribute.

* Use PyUnicode_FromFormat instead of a buffer

* Don't use C constants/types in error messages

* Make error messages easier to understand for Python users

* Lower minimum required version 1.4.0

* Use casts and make slot function signatures correct

* Be consistent with CPython on const usage

* Make else clauses in line with PEP 7

* Fix over-indented blocks in argument clinic

* Add critical section around ZSTD_DCtx_setParameter

* Add a TODO about refactoring critical sections

* Use Py_UNREACHABLE

* Move bytes operations out of Py_BEGIN_ALLOW_THREADS

* Add TODO about ensuring a lock is held

* Remove asserts that may not be correct

* Add TODO to make ZstdDict and others GC objects

* Make objects GC tracked

* Remove unused include

* Fix some memory issues

* Fix refleaks on module and in ZstdDict

* Update configure to check for ZDICT_finalizeDictionary

* Properly check version in configure

* exit(1) if check fails

* Use AC_RUN_IFELSE

* Use a define() to re-use version check

* Actually properly set _zstd module status based on version

---------

Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-05-04 01:29:55 +00:00
Pablo Galindo Salgado
2bc8365231
GH-91048: Add utils for printing the call stack for asyncio tasks (#133284) 2025-05-04 00:51:57 +00:00
Serhiy Storchaka
7363e8d24d
gh-133139: Add curses.assume_default_colors() (GH-133145)
This is a refinement of the curses.use_default_colors() function which
allows to change the color pair 0.
2025-05-03 23:33:22 +03:00
Sergey Miryanov
bd2ed7c7ce
gh-91048: Chain some exceptions in _testexternalinspection.c (#132970) 2025-05-03 01:35:30 +02:00
Sergey Miryanov
a0bc0c462f
gh-100926: Move ctype's pointers cache from _pointer_type_cache to StgInfo (GH-131282)
Deprecate _pointer_type_cache and calling POINTER on a string.

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
Co-authored-by: Jun Komoda <45822440+junkmd@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-05-02 19:06:37 +02:00
Sergey B Kirpichev
f425509349
gh-121249: unconditionally support complex types in struct (GH-132864)
Co-authored-by: Lisandro Dalcin <dalcinl@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-05-02 18:24:52 +02:00
Sam Gross
f2379535fe
gh-133164: Add PyUnstable_Object_IsUniqueReferencedTemporary C API (gh-133170)
After gh-130704, the interpreter replaces some uses of `LOAD_FAST` with
`LOAD_FAST_BORROW` which avoid incref/decrefs by "borrowing" references
on the interpreter stack when the bytecode compiler can determine that
it's safe.

This change broke some checks in C API extensions that relied on
`Py_REFCNT()` of `1` to determine if it's safe to modify an object
in-place. Objects may have a reference count of one, but still be
referenced further up the interpreter stack due to borrowing of
references.

This provides a replacement function for those checks.
`PyUnstable_Object_IsUniqueReferencedTemporary` is more conservative:
it checks that the object has a reference count of one and that it exists as a
unique strong reference in the interpreter's stack of temporary
variables in the top most frame.

See also:

* https://github.com/numpy/numpy/issues/28681

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: mpage <mpage@cs.stanford.edu>
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-05-02 13:24:57 +00:00
Petr Viktorin
2590774c9b
gh-133290: Use PyObject_SetAttr to set _type_ (GH-133292) 2025-05-02 14:47:07 +02:00
Victor Stinner
e26bafd107
gh-133256: Add _Py_NONSTRING macro (#133257)
Fix GCC 15 compiler warnings such as:

    Modules/fcntlmodule.c:27:36: warning: initializer-string for
    array of 'char' truncates NUL terminator but destination lacks
    'nonstring' attribute (9 chars into 8 available)
    [-Wunterminated-string-initialization]
    static const char guard[GUARDSZ] = "\x00\xfa\x69\xc4\x67\xa3\x6c\x58";
2025-05-01 17:55:49 +02:00
Sergey B Kirpichev
ad2f0884b1
gh-130317: Fix test_pack_unpack_roundtrip() and add docs (#133204)
* Skip sNaN's testing in 32-bit mode.
* Drop float_set_snan() helper.
* Use memcpy() workaround for sNaN's in PyFloat_Unpack4().
* Document, that sNaN's may not be preserved by PyFloat_Pack/Unpack API.
2025-05-01 16:20:36 +02:00
neonene
fa52f289a3
gh-133166: Fix missing error emission of PyType_GetModuleByDef (GH-133240) 2025-05-01 14:32:57 +02:00
Nybblista
b99d970bcd
Add missing whitespace in is_running_main comment (#133174) 2025-05-01 04:36:59 +01:00
Russell Keith-Magee
ecbc16a915
Bump the HACL* revision. (#133226)
Bump the HACL* revision to include iOS platform identification.
2025-05-01 10:57:49 +08:00
Eric Snow
811edcf9cd
Revert "gh-132775: Add _PyCode_GetVarCounts() (gh-133128)" (gh-133232)
The change broke the s390 builds, so I'm reverting it while I investigate.

This reverts commit 94b4fcd806.
2025-05-01 02:35:20 +00:00
Eric Snow
cb35c11d82
gh-132775: Add _PyPickle_GetXIData() (gh-133107)
There's some extra complexity due to making sure we we get things right when handling functions and classes defined in the __main__ module.  This is also reflected in the tests, including the addition of extra functions in test.support.import_helper.
2025-04-30 17:34:05 -06:00
Eric Snow
94b4fcd806
gh-132775: Add _PyCode_GetVarCounts() (gh-133128)
This helper is useful in a variety of ways, including in demonstrating how the different counts relate to one another.

It will be used in a later change to help identify if a function is "stateless", meaning it doesn't have any free vars or globals.

Note that a majority of this change is tests.
2025-04-30 18:19:20 +00:00
Mark Shannon
44e4c479fb
GH-124715: Move trashcan mechanism into Py_Dealloc (GH-132280) 2025-04-30 11:37:53 +01:00
Inada Naoki
4e294f6feb
gh-133036: Deprecate codecs.open (#133038)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-04-30 10:11:09 +09:00
Serhiy Storchaka
4265854d96
gh-132987: Support __index__() in the socket module (GH-133093)
ntohl(), htonl(), if_indextoname(), getaddrinfo() now use __index__() if
available.

Also fix the Argument Clinic names for module-level functions (although
this does not affect the user).
2025-04-29 19:27:07 +03:00
Serhiy Storchaka
c33efa8735
gh-132987: Support __index__() in the stat module (GH-133097)
Use it for the mode arguments in filemode(), S_IMODE(), S_ISDIR(), etc.
2025-04-29 19:25:44 +03:00
Victor Stinner
02cd6d7097
gh-130317: Fix strict aliasing in PyFloat_Pack8() (#133150)
* Fix strict aliasing in PyFloat_Pack8() and PyFloat_Pack4().
* Fix _testcapi.float_set_snan() on x86 (32-bit).
2025-04-29 14:27:50 +00:00
Serhiy Storchaka
a64fdc7513
gh-132987: Support __index__() in the lzma module (GH-133099) 2025-04-29 14:14:33 +00:00
Serhiy Storchaka
d2d490064d
gh-132987: Support __index__() in the select.kqueue_event constructor (GH-133094) 2025-04-29 13:55:14 +00:00
Serhiy Storchaka
0fb4c38c27
gh-132987: Support __index__() in the ssl.SSLContext.options setter (GH-133098) 2025-04-29 16:28:47 +03:00
Serhiy Storchaka
d6da6803a1
gh-132987: Support __index__() in hashlib.scrypt() (GH-133100)
Even if such signature is not supported by PyArg_ParseTupleAndKeywords(),
Argument Clinic supports it with inlined converters.
2025-04-29 16:16:05 +03:00
Serhiy Storchaka
07edc0d2b2
gh-132987: Support __index__() in the posix module (GH-133096)
Support it for the dev_t values in mknod(), major(), minor() and makedev(),
CPU numbers in sched_setaffinity(), group numbers in setgroups(),
configuration name in fpathconf(), pathconf(), confstr(), and sysconf().
2025-04-29 16:14:47 +03:00
Eric Snow
219d8d24b5
gh-87859: Track Code Object Local Kinds For Arguments (gh-132980)
Doing this was always the intention. I was finally motivated to find the time to do it.

See #87859 (comment).
2025-04-29 02:21:47 +00:00
Eric Snow
96a7fb93a8
gh-132775: Add _PyCode_ReturnsOnlyNone() (gh-132981)
The function indicates whether or not the function has a return statement.

This is used by a later change related treating some functions like scripts.
2025-04-28 20:12:52 -06:00
Eric Snow
bdd23c0bb9
gh-132775: Add _PyMarshal_GetXIData() (gh-133108)
Note that the bulk of this change is tests.
2025-04-28 17:23:46 -06:00
Neil Schemenauer
22f0730d40
gh-122320: Limit dict key versions used by test_opcache. (gh-132961)
The `test_load_global_module()` test consumes a lot of dict key versions.
Skip the test if we have consumed half of the available versions that can be
used for the "load global" cache.
2025-04-28 12:54:55 -07:00
Eric Snow
6f04325992
gh-132775: Cleanup Related to crossinterp.c Before Further Changes (gh-132974)
This change consists of adding tests and moving code around, with some renaming thrown in.
2025-04-28 11:55:15 -06:00
Jelle Zijlstra
58567cc18c
gh-132952: Speed up startup by importing _io instead of io (#132957) 2025-04-28 08:38:56 -07:00
Serhiy Storchaka
c2eaeee3dc
gh-132915: Try to detect a buffer overflow in fcntl() and ioctl() (GH-132919)
SystemError is raised when buffer overflow is detected.
The stack and memory can already be corrupted, so treat this error as fatal.
2025-04-28 17:56:10 +03:00
Petr Viktorin
4ebbfcf30e
gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread (gh-130402)
If `Py_IsFinalizing()` is true, non-daemon threads (other than the current one)
are done, and daemon threads are prevented from running, so they
cannot finalize themselves and become done. Joining them (without timeout)
would block forever.

Raise PythonFinalizationError instead of hanging.

Raise even when a timeout is given, for consistency with trying to join your own thread.

See gh-123940 for a use case: calling `join()` from `__del__`. This is
ill-advised, but an exception should at least make it easier to diagnose.
2025-04-28 15:48:48 +02:00
Sergey B Kirpichev
6157135a8d
gh-130317: Fix PyFloat_Pack/Unpack[24] for NaN's with payload (#130452)
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-04-28 15:23:26 +02:00
Sergey B Kirpichev
5bf0f3666e
gh-53032: support IEEE 754 contexts in the decimal module (#122003)
This was in C version from beginning, but available only
on conditional compilation (EXTRA_FUNCTIONALITY).  Current
patch adds function to create IEEE contexts to the
pure-python module as well.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-04-28 15:05:56 +02:00
Bénédikt Tran
1b7470f8cb
gh-133061: do not mention UINT32_MAX in HMAC user-facing messages (#133062) 2025-04-28 01:14:12 +02:00
Bénédikt Tran
3695ba93d5
gh-132993: expose HASHLIB_GIL_MINSIZE to private extension modules (#132999) 2025-04-27 22:20:15 +00:00
Alyssa Ross
314f4b9716
gh-132991: Add socket.IP_FREEBIND constant (GH-132998) 2025-04-26 20:54:12 +03:00
Anthony Sottile
bd2e5f044c
gh-131298: update Modules/Setup after HACL* static linking changes (#133012)
This was overlooked in 5f2ba152a0.
2025-04-26 18:44:19 +02:00