Commit graph

31760 commits

Author SHA1 Message Date
Hugo van Kemenade
a62562859d Python 3.15.0a2 2025-11-18 16:51:17 +02:00
Stan Ulbrych
16ea9505ce
gh-141004: Document Py_MEMCPY (GH-141676) 2025-11-17 17:52:13 -05:00
Louis
5d2eb98a91
gh-140578: Delete unnecessary NEWS entry (#141427)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-11-17 19:47:00 +02:00
Krishna Chaitanya
6b1bdf6c7a
gh-141497: Make ipaddress.IP{v4,v6}Network.hosts() always returning an iterator (GH-141547) 2025-11-17 19:29:06 +02:00
R.C.M
3d14805947
gh-130693: Support more options for search in tkinter.Text (GH-130848)
* Add parameters nolinestop and strictlimits in the tkinter.Text.search() method.
* Add the tkinter.Text.search_all() method.
* Add more tests for tkinter.Text.search().
* stopindex is now only ignored if it is None.
2025-11-17 14:42:26 +00:00
Stefano Rivera
f6dd9c12a8
GH-139914: Handle stack growth direction on HPPA (GH-140028)
Adapted from a patch for Python 3.14 submitted to the Debian BTS by John
https://bugs.debian.org/1105111#20

Co-authored-by: John David Anglin <dave.anglin@bell.net>
2025-11-17 14:41:22 +01:00
Brandt Bucher
336366fd7c
GH-140643: Add <native> and <GC> frames to the sampling profiler (#141108)
- Introduce a new field in the GC state to store the frame that initiated garbage collection.
- Update RemoteUnwinder to include options for including "<native>" and "<GC>" frames in the stack trace.
- Modify the sampling profiler to accept parameters for controlling the inclusion of native and GC frames.
- Enhance the stack collector to properly format and append these frames during profiling.
- Add tests to verify the correct behavior of the profiler with respect to native and GC frames, including options to exclude them.

Co-authored-by: Pablo Galindo Salgado <pablogsal@gmail.com>
2025-11-17 13:39:00 +00:00
yihong
994ab5c922
gh-140729: Add __mp_main__ as a duplicate for __main__ for pickle to work (#140735) 2025-11-17 12:43:14 +00:00
Stan Ulbrych
31ea3f3c76
gh-141018: Update .exe, .dll, .rtf and .jpg mime types in mimetypes (#141023) 2025-11-17 13:32:00 +02:00
Tian Gao
8be3b2f479
gh-136057: Allow step and next to step over for loops (#136160) 2025-11-16 13:57:07 -08:00
Tian Gao
5348c200f5
gh-125115 : Refactor the pdb parsing issue so positional arguments can pass through (#140933) 2025-11-16 10:58:28 -08:00
Pablo Galindo Salgado
4ceb077c5c
gh-141579: Fix perf_jit backend in sys.activate_stack_trampoline() (#141580) 2025-11-15 02:23:54 +00:00
Takuya UESHIN
53d65c840e
gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception (#136487) 2025-11-14 16:59:51 -08:00
Guo Ci
453d886f85
GH-90344: replace single-call io.IncrementalNewlineDecoder usage with non-incremental newline decoders (GH-30276)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-11-15 00:13:37 +00:00
Raymond Hettinger
f0a8bc737a
gh-140938: Raise ValueError for infinite inputs to stdev/pstdev (GH-141531)
Raise ValueError for infinite inputs to stdev/pstdev

---

Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-11-14 23:25:45 +00:00
Mikhail Efimov
1281be1caf
gh-141367: Use CALL_LIST_APPEND instruction only for lists, not for list subclasses (GH-141398)
Co-authored-by: Ken Jin <kenjin4096@gmail.com>
2025-11-14 21:38:39 +00:00
Kumar Aditya
f26ed455d5
gh-114203: skip locking if object is already locked by two-mutex critical section (#141476) 2025-11-14 23:17:59 +05:30
Sergey Miryanov
fa245df4a0
GH-141509: Fix warning about remaining subinterpreters (GH-141528)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-11-14 14:55:04 +00:00
Sergey Miryanov
10bec7c1eb
GH-141312: Allow only integers to longrangeiter_setstate state (GH-141317)
This fixes an assertion error when the new computed start is not an integer.
2025-11-14 14:52:01 +00:00
Serhiy Storchaka
8deaa9393e
gh-122255: Synchronize warnings in C and Python implementations of the warnings module (GH-122824)
In the linecache module and in the Python implementation of the
warnings module, a DeprecationWarning is issued when
m.__loader__ differs from m.__spec__.loader (like in the C
implementation of the warnings module).
2025-11-14 16:49:28 +02:00
Itamar Oren
1e4e59bb37
gh-116146: Add C-API to create module from spec and initfunc (GH-139196)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-11-14 10:43:25 +01:00
M Bussonnier
4885ecfbda
gh-140790: pdb: Initialize instance variables in Pdb.__init__ (#140791)
Initialize lineno, stack, curindex, curframe, currentbp, and _user_requested_quit attributes in `Pdb.__init__``.
2025-11-13 19:18:54 -08:00
Osama Abdelkader
a486d452c7
gh-140601: Add ResourceWarning to iterparse when not closed (GH-140603)
When iterparse() opens a file by filename and is not explicitly closed,
emit a ResourceWarning to alert developers of the resource leak.

Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-11-13 21:05:28 +02:00
dr-carlos
209eaff68c
gh-137969: Fix double evaluation of ForwardRefs which rely on globals (#140974) 2025-11-13 18:17:17 +00:00
Ken Jin
4fa80ce74c
gh-139109: A new tracing JIT compiler frontend for CPython (GH-140310)
This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus current https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT. **Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR**. The speedup in the merged version is about 1.1%. https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

Stats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long aborts https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .

This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this work gh-140277

The optimizer stack space check is disabled, as it's no longer valid to deal with underflow.

Pros:
* Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.
* `optimizer.c` is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.
* The new JIT frontend is able to handle a lot more control-flow than the old one.
* Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.
* Better handling of polymorphism. We leverage the specializing interpreter for this.

Cons:
* (For now) requires tail calling interpreter or computed gotos. This means no Windows JIT for now :(. Not to fret, tail calling is coming soon to Windows though https://github.com/python/cpython/pull/139962

Design:
* After each instruction, the `record_previous_inst` function/label is executed. This does as the name suggests.
* The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.
* The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.
* The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.
* The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.
* Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.
* Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction --- `_DYNAMIC_EXIT`.
2025-11-13 18:08:32 +00:00
Serhiy Storchaka
b2b68d40f8
gh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (GH-140884) 2025-11-13 19:48:52 +02:00
Victor Stinner
b99db92dde
gh-139653: Add PyUnstable_ThreadState_SetStackProtection() (#139668)
Add PyUnstable_ThreadState_SetStackProtection() and
PyUnstable_ThreadState_ResetStackProtection() functions
to set the stack base address and stack size of a Python
thread state.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-11-13 17:30:50 +01:00
Cody Maloney
732224e113
gh-139871: Add bytearray.take_bytes([n]) to efficiently extract bytes (GH-140128)
Update `bytearray` to contain a `bytes` and provide a zero-copy path to
"extract" the `bytes`. This allows making several code paths more efficient.

This does not move any codepaths to make use of this new API. The documentation
changes include common code patterns which can be made more efficient with
this API.

---

When just changing `bytearray` to contain `bytes` I ran pyperformance on a
`--with-lto --enable-optimizations --with-static-libpython` build and don't see
any major speedups or slowdowns with this; all seems to be in the noise of
my machine (Generally changes under 5% or benchmarks that don't touch
bytes/bytearray).


Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Maurycy Pawłowski-Wieroński <5383+maurycy@users.noreply.github.com>
2025-11-13 13:19:44 +00:00
Serhiy Storchaka
d8e6bdc0d0
gh-135801: Add the module parameter to compile() etc (GH-139652)
Many functions related to compiling or parsing Python code, such as
compile(), ast.parse(), symtable.symtable(),
and importlib.abc.InspectLoader.source_to_code() now allow to pass
the module name used when filtering syntax warnings.
2025-11-13 13:21:32 +02:00
Shamil
63548b3699
gh-140260: fix data race in _struct module initialization with subinterpreters (#140909) 2025-11-13 16:31:31 +05:30
Gregory P. Smith
781cc68c3c
gh-137109: refactor warning about threads when forking (#141438)
* gh-137109: refactor warning about threads when forking

This splits the OS API specific functionality to get the number of threads out
from the fallback Python method and warning raising code itself.  This way the
OS APIs can be queried before we've run
`os.register_at_fork(after_in_parent=...)` registered functions which
themselves may (re)start threads that would otherwise be detected.

This is best effort.  If the OS APIs are either unavailable or fail, the
warning generating code still falls back to looking at the Python threading
state after the CPython interpreter world has been restarted and the
after_in_parent calls have been made.  The common case for most Linux and macOS
environments should work today.

This also lines up with the existing TODO refactoring, we may choose to expose
this API to get the number of OS threads in the `os` module in the future.

* NEWS entry

* avoid "function-prototype" compiler warning?
2025-11-12 18:15:16 -08:00
Michael Cho
dc0987080e
gh-124111: Fix TCL 9 thread detection (GH-128103) 2025-11-12 22:16:58 +00:00
Russell Keith-Magee
558936bec1
gh-141442: Add escaping to iOS testbed arguments (#141443)
Xcode concatenates the test argument array, losing quoting in the process.
2025-11-13 05:41:26 +08:00
Savannah Ostrowski
d162c42790
GH-140479: Update JIT builds to use LLVM 21 (#140973) 2025-11-12 18:09:25 +00:00
Stan Ulbrych
88aeff8eab
gh-87710: Update mime type for `.ai` (#141239) 2025-11-12 16:22:01 +02:00
Sergey B Kirpichev
e2026731f5
gh-141004: soft-deprecate Py_INFINITY macro (#141033)
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-11-12 13:44:49 +01:00
Rani Pinchuk
c6f3dd6a50
gh-98896: resource_tracker: use json&base64 to allow arbitrary shared memory names (GH-138473) 2025-11-12 13:35:01 +01:00
Jacob Austin Lincoln
70748bdbea
gh-131116: Fix inspect.getdoc() to work with cached_property objects (GH-131165) 2025-11-12 10:07:21 +00:00
Sergey B Kirpichev
23d85a2a3f
gh-141042: fix sNaN's packing for mixed floating-point formats (#141107) 2025-11-12 11:06:29 +01:00
Cody Maloney
7d54374f9c
gh-141311: Avoid assertion in BytesIO.readinto() (GH-141333)
Fix error in assertion which causes failure if pos is equal to PY_SSIZE_T_MAX.
Fix undefined behavior in read() and readinto() if pos is larger that the size
of the underlying buffer.
2025-11-12 11:57:05 +02:00
Cody Maloney
6f988b08d1
gh-85524: Raise "UnsupportedOperation" on FileIO.readall (#141214)
io.UnsupportedOperation is a subclass of OSError and recommended by
io.IOBase for this case; matches other read methods on io.FileIO.
2025-11-12 10:37:48 +01:00
Mark Byrne
35908265b0
gh-75593: Add support of bytes and path-like paths in wave.open() (GH-140951) 2025-11-12 10:20:55 +02:00
Kumar Aditya
ef474cfafb
gh-103847: fix cancellation safety of asyncio.create_subprocess_exec (#140805) 2025-11-12 10:47:38 +05:30
Adam Turner
9ce99c6c19
GH-137618: Require Python 3.10 to Python 3.15 for PYTHON_FOR_REGEN (GH-137619)
* Require Python 3.11 to Python 3.15 for PYTHON_FOR_REGEN

* NEWS

* keep allowing python 3.10

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2025-11-12 00:53:21 +00:00
yihong
f5c2a41f9a
gh-138775: fix handle python -m base64 stdin correct with EOF signal (GH-138776)
* fix: handle  stdin correct with EOF single.
* fix: flollow the comments when pipe stdin use buffer
* Apply suggestions from code review
* fix: apply review comments in Lib/base64.py
* fix: address comments
* Reword comment and NEWS entry.

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2025-11-11 23:47:57 +00:00
Serhiy Storchaka
4359706ac8
gh-120950: Fix overflow in math.log() with large int-like argument (GH-121011)
Handling of arbitrary large int-like argument is now consistent with
handling arbitrary large int arguments.
2025-11-12 00:27:13 +02:00
J Berg
9e7340cd3b
gh-139462: Make the ProcessPoolExecutor BrokenProcessPool exception report which child process terminated (GH-139486)
Report which process terminated as cause of BPE
2025-11-11 22:09:58 +00:00
Serhiy Storchaka
7906f4d96a
gh-132686: Add parameters inherit_class_doc and fallback_to_class_doc for inspect.getdoc() (GH-132691) 2025-11-12 00:01:25 +02:00
Mohsin Mehmood
af80fac425
gh-141314: Fix TextIOWrapper.tell() assertion failure with standalone carriage return (GH-141331)
The assertion was checking wrong variable (skip_back vs skip_bytes).
2025-11-11 23:49:54 +02:00
Alper
298e9074cd
gh-140476: optimize PySet_Add for frozenset in free-threading (#140440)
Avoids critical section in `PySet_Add` when adding items to newly created frozensets.

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-11-12 01:57:21 +05:30