Commit graph

6173 commits

Author SHA1 Message Date
Ron Frederick
377b787618
gh-136306: Add support for getting and setting SSL groups (#136307)
Add support for getting and setting groups used for key agreement.

* `ssl.SSLSocket.group()` returns the name of the group used
  for the key agreement of the current session establishment.
  This feature requires Python to be built with OpenSSL 3.2 or later.

* `ssl.SSLContext.get_groups()` returns the list of names of groups
  that are compatible with the TLS version of the current context.
  This feature requires Python to be built with OpenSSL 3.5 or later.

* `ssl.SSLContext.set_groups()` sets the groups allowed for key agreement
  for sockets created with this context. This feature is always supported.
2025-07-28 19:33:31 +02:00
Kumar Aditya
cbe6ebe15b
gh-134043: use stackrefs for dict lookup in _PyObject_GetMethodStackRef (#136412)
Co-authored-by: Sam Gross <colesbury@gmail.com>
2025-07-28 22:04:57 +05:30
Abdul
cfd6da849a
gh-137058: Update C23 standard version check in pyport.h (#137127)
Use `__STDC_VERSION__ >= 202311L` instead of `__STDC_VERSION__ > 201710L`.
2025-07-27 09:47:21 +02:00
AN Long
ec02db5caa
gh-136759: rename lock.h to pylock.h (#137041)
Rename `lock.h` to `pylock.h` to avoid conflicts with headers of other projects.
2025-07-24 16:16:07 +05:30
Nathan Goldbaum
89c220b93c
gh-133296: Publicly expose critical section API that accepts PyMutex (gh-135899)
This makes the following APIs public:

* `Py_BEGIN_CRITICAL_SECTION_MUTEX(mutex),`
* `Py_BEGIN_CRITICAL_SECTION2_MUTEX(mutex1, mutex2)`
* `void PyCriticalSection_BeginMutex(PyCriticalSection *c, PyMutex *mutex)`
* `void PyCriticalSection2_BeginMutex(PyCriticalSection2 *c, PyMutex *mutex1, PyMutex *mutex2)`

The macros are identical to the corresponding `Py_BEGIN_CRITICAL_SECTION` and
`Py_BEGIN_CRITICAL_SECTION2` macros (e.g., they include braces), but they
accept a `PyMutex` instead of an object.

The new macros are still paired with the existing END macros
(`Py_END_CRITICAL_SECTION`, `Py_END_CRITICAL_SECTION2`).
2025-07-21 17:25:43 -04:00
Peter Bierma
a10960699a
gh-136421: Load _datetime static types during interpreter initialization (GH-136583)
`_datetime` is a special module, because it's the only non-builtin C extension that contains static types. As such, it would initialize static types in the module's execution function, which can run concurrently. Since static type initialization is not thread-safe, this caused crashes. This fixes it by moving the initialization of `_datetime`'s static types to interpreter startup (where all other static types are initialized), which is already properly protected through other locks.
2025-07-21 13:47:26 -04:00
Adam Turner
c89a66feb1
GH-133711: Enable UTF-8 mode by default (PEP 686) (#133712)
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-07-15 10:45:41 +01:00
Lee Dogeon
cd3e7ac89b
gh-67341: fix a typo in Include/fileutils.h (#136049)
`IO_REPARSE_TAG_SYMLINK` is mapped to `S_IFLNK` in `fileutils.c`, not in `posixmodule.c`
2025-07-14 08:02:05 +00:00
Serhiy Storchaka
da699ed7e5
gh-121914: Change the names of the symbol tables for lambda and genexpr (GH-135288)
Change the names of the symbol tables for lambda expressions and generator
expressions to "<lambda>" and "<genexpr>" respectively to avoid conflicts
with user-defined names.
2025-07-13 21:09:42 +03:00
Petr Viktorin
c7d24b81c3
gh-111506: Add _Py_OPAQUE_PYOBJECT to hide PyObject layout & related API (GH-136505)
Allow Py_LIMITED_API for (Py_GIL_DISABLED && _Py_OPAQUE_PYOBJECT)


API that's removed when _Py_OPAQUE_PYOBJECT is defined:

    - PyObject_HEAD
    - _PyObject_EXTRA_INIT
    - PyObject_HEAD_INIT
    - PyObject_VAR_HEAD
    - struct _object (i.e. PyObject) (opaque)
    - struct PyVarObject (opaque)
    - Py_SIZE
    - Py_SET_TYPE
    - Py_SET_SIZE
    - PyModuleDef_Base (opaque)
    - PyModuleDef_HEAD_INIT
    - PyModuleDef (opaque)
    - _Py_IsImmortal
    - _Py_IsStaticImmortal

Note that the `_Py_IsImmortal` removal (and a few other issues)
 means _Py_OPAQUE_PYOBJECT only works with limited
API 3.14+ now.


Co-authored-by: Victor Stinner <vstinner@python.org>
2025-07-12 09:55:12 +02:00
Weilin Du
561212a033
Doc: More duplicate word fixes (GH-136299) 2025-07-11 21:18:47 +03:00
Victor Stinner
2c9a8011c6
gh-135906: Test the internal C API in test_cext (#136247)
Remove duplicated definition: atexit_datacallbackfunc type
is already defined by Include/cpython/pylifecycle.h.
2025-07-11 16:48:43 +02:00
Pablo Galindo Salgado
236f733d8f
gh-136541: Fix several problems of perf trampolines in x86_64 and aarch64 (#136500)
This commit fixes the following problems:

* The x86_64 trampolines are not preserving frame pointers
* The hardcoded offsets to the code segment from the FDE only worked properly for x64_64
* The CIE data was not following conventions of aarch64
* The eh_frame for aarch64 was not fully correct
2025-07-11 14:32:35 +01:00
Arseniy Terekhin
a6566e49e6
gh-92536: Fix comment about number of unicode string types (#136439) 2025-07-08 23:49:55 +02:00
Charlie Lin
fe187fae8d
gh-135906: Use _PyObject_CAST in internal headers (GH-135892)
Fixes build errors encountered in python-greenlet/greenlet#450 when building greenlet on the free-threaded build.

---------

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-07-07 12:56:14 -04:00
Victor Stinner
fa43a1e0f8
gh-127705: Move Py_INCREF_MORTAL() to the internal C API (GH-136178)
Rename Py_INCREF_MORTAL() to _Py_INCREF_MORTAL() and move it to
pycore_object.h internal header.
2025-07-02 15:40:41 +02:00
Sam Gross
f41e9c750e
gh-134009: Expose PyMutex_IsLocked in the public C API (gh-134365)
The `PyMutex_IsLocked()` function is useful in assertions for verifying
that code maintains certain locking invariants.
2025-07-01 13:26:13 -04:00
Ken Jin
e0d6500b2d
gh-136125: Use _PyObject_GetMethodStackRef for LOAD_ATTR (GH-136127) 2025-07-02 01:20:46 +08:00
Victor Stinner
28940e8e48
gh-130396: Move PYOS_LOG2_STACK_MARGIN to internal headers (#135928)
Move PYOS_LOG2_STACK_MARGIN, PYOS_STACK_MARGIN,
PYOS_STACK_MARGIN_BYTES and PYOS_STACK_MARGIN_SHIFT macros to
pycore_pythonrun.h internal header. Add underscore (_) prefix to the
names to make them private. Rename _PYOS to _PyOS.
2025-07-01 15:18:17 +02:00
Petr Viktorin
fe119a0817
gh-87135: threading.Lock: Raise rather than hang on Python finalization (GH-135991)
After Python finalization gets to the point where no other thread
can attach thread state, attempting to acquire a Python lock must hang.
Raise PythonFinalizationError instead of hanging.
2025-07-01 10:57:42 +02:00
Petr Viktorin
845263adc6
gh-133157: remove usage of _Py_NO_SANITIZE_UNDEFINED in pyexpat (#135346)
This was the last usage, so the macro is removed as well.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-07-01 10:54:08 +02:00
Xuanteng Huang
b1056c2a44
gh-135607: remove null checking of weakref list in dealloc of extension modules and objects (#135614)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-06-30 11:14:31 +00:00
Weilin Du
698bab5a40
Doc: fix duplicated words (#136086)
---------

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2025-06-29 18:04:02 -04:00
Pablo Galindo Salgado
5334732f9c
gh-91048: Fix external inspection multi-threaded performance (#136005) 2025-06-28 14:11:31 +01:00
Ken Jin
695ab61351
gh-132732: Automatically constant evaluate pure operations (GH-132733)
This adds a "macro" to the optimizer DSL called "REPLACE_OPCODE_IF_EVALUATES_PURE", which allows automatically constant evaluating a bytecode body if certain inputs have no side effects upon evaluations (such as ints, strings, and floats).


Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
2025-06-27 19:37:44 +08:00
Victor Stinner
8594d2c03d
gh-135927: Check _MSC_VER to define _Py_NULL macro (#135987) 2025-06-26 18:11:49 +02:00
Petr Viktorin
6be17baeb5
gh-135755: Use private names (_Py*) for header file guards new in 3.14 (GH-135921)
These are private API; let's name new ones accordingly.
2025-06-26 13:05:01 +02:00
Peter Bierma
10a3d43188
gh-135755: Move PyFunction_GET_BUILTINS to the private API (GH-135938) 2025-06-26 11:43:08 +02:00
Petr Viktorin
a1da208eec
gh-131591: Add Py_ prefix to MAX_SCRIPT_PATH_SIZE; remove unprefixed struct tag (GH-135924)
Names/macros defined in public headers should have `Py`/`_Py` prefixes.
2025-06-26 09:25:41 +02:00
Ken Jin
a88b49c3f2
gh-135927: Fix MSVC Clatest C builds (GH-135935) 2025-06-25 23:02:53 +08:00
Petr Viktorin
1b1ae82fab
gh-135755: Move SPECIAL_ constants to a private header (GH-135922)
Macros without a `Py`/`_Py` prefix should not be defined in public headers.
2025-06-25 13:03:05 +02:00
Neil Schemenauer
113de8545f
GH-133136: Revise QSBR to reduce excess memory held (gh-135473)
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.

Take into account the size of the memory block when deciding when to
process QSBR requests.

Also track the amount of memory being held by QSBR for mimalloc pages.  Advance the write sequence if this memory exceeds a limit.  Advancing the sequence will allow it to be freed more quickly.

Process the held QSBR items from the "eval breaker", rather than from `_PyMem_FreeDelayed()`.  This gives a higher chance that the global read sequence has advanced enough so that items can be freed.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2025-06-25 00:06:32 -07:00
Ken Jin
569fc6870f
gh-134584: Specialize POP_TOP by reference and type in JIT (GH-135761) 2025-06-24 00:57:14 +08:00
Kumar Aditya
a8ec511900
gh-135380: enhance critical section held assertions (#135381) 2025-06-20 22:43:23 +05:30
Ken Jin
0243260284
gh-135379: Move PyLong_CheckCompact to private header and rename it (GH-135707) 2025-06-19 13:09:09 +00:00
Ken Jin
1b969f6d57
gh-135379: Add back const cast to _PyLong_IsCompact (GH-135706) 2025-06-19 13:54:21 +02:00
Mark Shannon
9731dd2c8d
GH-135379: Specialize int operations for compact ints only (GH-135668) 2025-06-19 11:10:29 +01:00
Donghee Na
52be7f445e
gh-133931: Introduce _PyObject_XSetRefDelayed to replace Py_XSETREF (gh-134377) 2025-06-18 08:36:02 +09:00
Ken Jin
fba5dded6d
gh-134584: Decref elimination for float ops in the JIT (GH-134588)
This PR adds a PyJitRef API to the JIT's optimizer that mimics the _PyStackRef API. This allows it to track references and their stack lifetimes properly. Thus opening up the doorway to refcount elimination in the JIT.
2025-06-17 23:25:53 +08:00
Mark Shannon
8dd8b5c2f0
GH-135379: Support limited scalar replacement for replicated uops in the JIT code generator. (GH-135563)
* Use it to support efficient specializations of COPY and SWAP in the JIT.
2025-06-17 13:43:09 +01:00
Eric Snow
a450a0ddec
gh-135443: Sometimes Fall Back to __main__.__dict__ For Globals (gh-135491)
For several builtin functions, we now fall back to __main__.__dict__ for the globals
when there is no current frame and _PyInterpreterState_IsRunningMain() returns
true.  This allows those functions to be run with Interpreter.call().

The affected builtins:

* exec()
* eval()
* globals()
* locals()
* vars()
* dir()

We take a similar approach with "stateless" functions, which don't use any
global variables.
2025-06-16 17:34:19 -06:00
Ken Jin
7b15873ed0
gh-135474: Specialize arithmetic only on compact ints (GH-135479)
Specialize arithmetic only on compact ints. This also makes int operations non-escaping in the JIT and in tier 1.
2025-06-14 17:13:32 +08:00
Eric Snow
c7f4a80079
gh-132775: Clean Up Cross-Interpreter Error Handling (gh-135369)
In this refactor we:

* move some code around
* make a couple of typedefs opaque
* decouple errors from session state
* improve tracebacks for propagated exceptions

This change helps simplify several upcoming changes.
2025-06-13 16:45:21 -06:00
Sergey B Kirpichev
c646846c1e
gh-126703: Add freelist for PyComplexObject's (gh-135233) 2025-06-13 22:32:26 +09:00
Mark Shannon
c87b5b2cb6
GH-135379: Remove types from stack items in code generator. (GH-135384)
* Make casts explicit in the instruction definitions
2025-06-11 15:52:25 +01:00
Petr Viktorin
49d72365cd
gh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (GH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)

This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.

The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.

* Specify minimum PyGC_Head and PyObject alignment

As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.

Encode this requirement using _Py_ALIGNED_DEF.

This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.

Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
2025-06-11 12:44:58 +02:00
Petr Viktorin
e413e26719
gh-134891: Add PyUnstable_Unicode_GET_CACHED_HASH (GH-134892) 2025-06-06 15:51:00 +02:00
Mark Shannon
b90ecea9e6
GH-132554: Fix tier2 FOR_ITER implementation and optimizations (GH-135137) 2025-06-05 18:53:57 +01:00
Victor Stinner
9258f3da91
gh-134989: Fix Py_RETURN_NONE in the limited C API (GH-135165)
Fix Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE macros in the
limited C API 3.11 and older:
Don't treat Py_None, Py_True and Py_False as immortal.
2025-06-05 14:43:47 +02:00
mpage
6b77af257c
gh-134889: Fix handling of a few opcodes when optimizing LOAD_FAST (#134958)
We were incorrectly handling a few opcodes that leave their operands on the stack. Treat all of these conservatively; assume that they always leave operands on the stack.
2025-06-04 16:07:58 -07:00