Commit graph

34590 commits

Author SHA1 Message Date
Shamil Abdulaev
ed1bf54efe
gh-45154: Fix imaplib handling of READ-ONLY mailboxes 2025-11-18 14:08:17 +03:00
Cody Maloney
58f3fe0d9b
gh-129005: Remove copies from _pyio using take_bytes (#141539)
Memory usage now matches that of _io for large files.
2025-11-18 10:10:32 +01:00
Pablo Galindo Salgado
4867f717e2
gh-140729: Fix subprocess handling in test_process_pool_executor_pickle (#141688) 2025-11-18 02:26:40 +00:00
Krishna Chaitanya
6b1bdf6c7a
gh-141497: Make ipaddress.IP{v4,v6}Network.hosts() always returning an iterator (GH-141547) 2025-11-17 19:29:06 +02:00
Pablo Galindo Salgado
274a26cca8
gh-135953: Simplify GC markers in the tachyon profiler (#141666) 2025-11-17 16:32:08 +00:00
R.C.M
3d14805947
gh-130693: Support more options for search in tkinter.Text (GH-130848)
* Add parameters nolinestop and strictlimits in the tkinter.Text.search() method.
* Add the tkinter.Text.search_all() method.
* Add more tests for tkinter.Text.search().
* stopindex is now only ignored if it is None.
2025-11-17 14:42:26 +00:00
Stefano Rivera
f6dd9c12a8
GH-139914: Handle stack growth direction on HPPA (GH-140028)
Adapted from a patch for Python 3.14 submitted to the Debian BTS by John
https://bugs.debian.org/1105111#20

Co-authored-by: John David Anglin <dave.anglin@bell.net>
2025-11-17 14:41:22 +01:00
Brandt Bucher
336366fd7c
GH-140643: Add <native> and <GC> frames to the sampling profiler (#141108)
- Introduce a new field in the GC state to store the frame that initiated garbage collection.
- Update RemoteUnwinder to include options for including "<native>" and "<GC>" frames in the stack trace.
- Modify the sampling profiler to accept parameters for controlling the inclusion of native and GC frames.
- Enhance the stack collector to properly format and append these frames during profiling.
- Add tests to verify the correct behavior of the profiler with respect to native and GC frames, including options to exclude them.

Co-authored-by: Pablo Galindo Salgado <pablogsal@gmail.com>
2025-11-17 13:39:00 +00:00
Pablo Galindo Salgado
89a914c58d
gh-135953: Add GIL contention markers to sampling profiler Gecko format (#139485)
This commit enhances the Gecko format reporter in the sampling profiler
to include markers for GIL acquisition events.
2025-11-17 12:46:26 +00:00
yihong
994ab5c922
gh-140729: Add __mp_main__ as a duplicate for __main__ for pickle to work (#140735) 2025-11-17 12:43:14 +00:00
Stan Ulbrych
31ea3f3c76
gh-141018: Update .exe, .dll, .rtf and .jpg mime types in mimetypes (#141023) 2025-11-17 13:32:00 +02:00
Tian Gao
8be3b2f479
gh-136057: Allow step and next to step over for loops (#136160) 2025-11-16 13:57:07 -08:00
Tian Gao
5348c200f5
gh-125115 : Refactor the pdb parsing issue so positional arguments can pass through (#140933) 2025-11-16 10:58:28 -08:00
Ken Jin
ed73c909f2
gh-139109: JIT _EXIT_TRACE to ENTER_EXECUTOR rather than _DEOPT (GH-141573) 2025-11-15 20:19:41 +00:00
Pablo Galindo Salgado
4ceb077c5c
gh-141579: Fix perf_jit backend in sys.activate_stack_trampoline() (#141580) 2025-11-15 02:23:54 +00:00
Takuya UESHIN
53d65c840e
gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception (#136487) 2025-11-14 16:59:51 -08:00
Guo Ci
453d886f85
GH-90344: replace single-call io.IncrementalNewlineDecoder usage with non-incremental newline decoders (GH-30276)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-11-15 00:13:37 +00:00
Raymond Hettinger
f0a8bc737a
gh-140938: Raise ValueError for infinite inputs to stdev/pstdev (GH-141531)
Raise ValueError for infinite inputs to stdev/pstdev

---

Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-11-14 23:25:45 +00:00
Mikhail Efimov
1281be1caf
gh-141367: Use CALL_LIST_APPEND instruction only for lists, not for list subclasses (GH-141398)
Co-authored-by: Ken Jin <kenjin4096@gmail.com>
2025-11-14 21:38:39 +00:00
Sergey Miryanov
fa245df4a0
GH-141509: Fix warning about remaining subinterpreters (GH-141528)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-11-14 14:55:04 +00:00
Sergey Miryanov
10bec7c1eb
GH-141312: Allow only integers to longrangeiter_setstate state (GH-141317)
This fixes an assertion error when the new computed start is not an integer.
2025-11-14 14:52:01 +00:00
Serhiy Storchaka
8deaa9393e
gh-122255: Synchronize warnings in C and Python implementations of the warnings module (GH-122824)
In the linecache module and in the Python implementation of the
warnings module, a DeprecationWarning is issued when
m.__loader__ differs from m.__spec__.loader (like in the C
implementation of the warnings module).
2025-11-14 16:49:28 +02:00
Peter Bierma
c10fa5be61
gh-131229: Temporarily skip test_basic_multiple_interpreters_deleted_no_reset (GH-141552)
This is a temporary band-aid to unblock other PRs.

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-11-14 14:22:36 +00:00
Itamar Oren
1e4e59bb37
gh-116146: Add C-API to create module from spec and initfunc (GH-139196)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-11-14 10:43:25 +01:00
Petr Viktorin
a4dd66275b
gh-140550: Use a bool for the Py_mod_gil value (GH-141519)
This needs a single bit, but was stored as a void* in the module
struct. This didn't matter due to packing, but now that there's
another bool in the struct, we can save a bit of memory by
making md_gil a bool.

Variables that changed type are renamed, to detect conflicts.
2025-11-14 10:38:49 +01:00
Osama Abdelkader
a486d452c7
gh-140601: Add ResourceWarning to iterparse when not closed (GH-140603)
When iterparse() opens a file by filename and is not explicitly closed,
emit a ResourceWarning to alert developers of the resource leak.

Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-11-13 21:05:28 +02:00
dr-carlos
209eaff68c
gh-137969: Fix double evaluation of ForwardRefs which rely on globals (#140974) 2025-11-13 18:17:17 +00:00
Ken Jin
4fa80ce74c
gh-139109: A new tracing JIT compiler frontend for CPython (GH-140310)
This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus current https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT. **Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR**. The speedup in the merged version is about 1.1%. https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

Stats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long aborts https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .

This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this work gh-140277

The optimizer stack space check is disabled, as it's no longer valid to deal with underflow.

Pros:
* Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.
* `optimizer.c` is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.
* The new JIT frontend is able to handle a lot more control-flow than the old one.
* Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.
* Better handling of polymorphism. We leverage the specializing interpreter for this.

Cons:
* (For now) requires tail calling interpreter or computed gotos. This means no Windows JIT for now :(. Not to fret, tail calling is coming soon to Windows though https://github.com/python/cpython/pull/139962

Design:
* After each instruction, the `record_previous_inst` function/label is executed. This does as the name suggests.
* The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.
* The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.
* The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.
* The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.
* Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.
* Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction --- `_DYNAMIC_EXIT`.
2025-11-13 18:08:32 +00:00
Serhiy Storchaka
b2b68d40f8
gh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (GH-140884) 2025-11-13 19:48:52 +02:00
Cody Maloney
732224e113
gh-139871: Add bytearray.take_bytes([n]) to efficiently extract bytes (GH-140128)
Update `bytearray` to contain a `bytes` and provide a zero-copy path to
"extract" the `bytes`. This allows making several code paths more efficient.

This does not move any codepaths to make use of this new API. The documentation
changes include common code patterns which can be made more efficient with
this API.

---

When just changing `bytearray` to contain `bytes` I ran pyperformance on a
`--with-lto --enable-optimizations --with-static-libpython` build and don't see
any major speedups or slowdowns with this; all seems to be in the noise of
my machine (Generally changes under 5% or benchmarks that don't touch
bytes/bytearray).


Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Maurycy Pawłowski-Wieroński <5383+maurycy@users.noreply.github.com>
2025-11-13 13:19:44 +00:00
Serhiy Storchaka
2fbd396666
gh-140601: Refactor ElementTree.iterparse() tests (GH-141499)
Split existing tests on smaller methods and move them to separate class.
Rename variable "content" to "it".
Use BytesIO instead of StringIO.
Add few more tests.
2025-11-13 13:37:01 +02:00
Serhiy Storchaka
d8e6bdc0d0
gh-135801: Add the module parameter to compile() etc (GH-139652)
Many functions related to compiling or parsing Python code, such as
compile(), ast.parse(), symtable.symtable(),
and importlib.abc.InspectLoader.source_to_code() now allow to pass
the module name used when filtering syntax warnings.
2025-11-13 13:21:32 +02:00
Shamil
63548b3699
gh-140260: fix data race in _struct module initialization with subinterpreters (#140909) 2025-11-13 16:31:31 +05:30
Mikhail Efimov
35ed3e4ced
gh-140936: Fix JIT assertion crash at finalization if some generator is alive (GH-140969) 2025-11-12 19:04:02 +00:00
Stan Ulbrych
88aeff8eab
gh-87710: Update mime type for `.ai` (#141239) 2025-11-12 16:22:01 +02:00
Rani Pinchuk
c6f3dd6a50
gh-98896: resource_tracker: use json&base64 to allow arbitrary shared memory names (GH-138473) 2025-11-12 13:35:01 +01:00
Jacob Austin Lincoln
70748bdbea
gh-131116: Fix inspect.getdoc() to work with cached_property objects (GH-131165) 2025-11-12 10:07:21 +00:00
Sergey B Kirpichev
23d85a2a3f
gh-141042: fix sNaN's packing for mixed floating-point formats (#141107) 2025-11-12 11:06:29 +01:00
Cody Maloney
7d54374f9c
gh-141311: Avoid assertion in BytesIO.readinto() (GH-141333)
Fix error in assertion which causes failure if pos is equal to PY_SSIZE_T_MAX.
Fix undefined behavior in read() and readinto() if pos is larger that the size
of the underlying buffer.
2025-11-12 11:57:05 +02:00
Cody Maloney
6f988b08d1
gh-85524: Raise "UnsupportedOperation" on FileIO.readall (#141214)
io.UnsupportedOperation is a subclass of OSError and recommended by
io.IOBase for this case; matches other read methods on io.FileIO.
2025-11-12 10:37:48 +01:00
Mark Byrne
35908265b0
gh-75593: Add support of bytes and path-like paths in wave.open() (GH-140951) 2025-11-12 10:20:55 +02:00
Kumar Aditya
ef474cfafb
gh-103847: fix cancellation safety of asyncio.create_subprocess_exec (#140805) 2025-11-12 10:47:38 +05:30
Jan-Eric Nitschke
0e88be6f55
gh-138621: Increase test coverage for csv.DictReader and csv.Sniffer (GH-138622)
* Increase test coverage for csv.DictReader and csv.Sniffer

Previously there were no tests for the DictReader fieldnames
setter, the case where a StopIteration was encountered when trying
to determine the fieldnames from the content or the case where
Sniffer could not find a delimiter.

* Revert whitespace change to comment

* Add a test that csv.Sniffer.has_header checks up to 20 rows

* Replace name and age with letter and offset

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Address review comment

---------

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
2025-11-12 00:32:26 +00:00
Serhiy Storchaka
4359706ac8
gh-120950: Fix overflow in math.log() with large int-like argument (GH-121011)
Handling of arbitrary large int-like argument is now consistent with
handling arbitrary large int arguments.
2025-11-12 00:27:13 +02:00
J Berg
9e7340cd3b
gh-139462: Make the ProcessPoolExecutor BrokenProcessPool exception report which child process terminated (GH-139486)
Report which process terminated as cause of BPE
2025-11-11 22:09:58 +00:00
Serhiy Storchaka
7906f4d96a
gh-132686: Add parameters inherit_class_doc and fallback_to_class_doc for inspect.getdoc() (GH-132691) 2025-11-12 00:01:25 +02:00
Mohsin Mehmood
af80fac425
gh-141314: Fix TextIOWrapper.tell() assertion failure with standalone carriage return (GH-141331)
The assertion was checking wrong variable (skip_back vs skip_bytes).
2025-11-11 23:49:54 +02:00
Alper
c13b59204a
gh-116738: use PyMutex in lzma module (#140711)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-11-12 02:01:55 +05:30
Mikhail Efimov
2fb2b82161
gh-141367: Use actual SPECIALIZATION_THRESHOLD value in specialization related test (GH-141417) 2025-11-11 20:16:46 +00:00
yihong
0f09bda643
gh-140193: Forward port test_exec_set_nomemory_hang from 3.13 (GH-140187)
* chore: test_exec_set_nomemory_hang from 3.13

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: apply comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* Update Lib/test/test_exceptions.py

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* Update Lib/test/test_exceptions.py

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* fix: windows too long name 60 times is enough

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-11-11 11:27:56 -08:00