Commit graph

31113 commits

Author SHA1 Message Date
Mark Shannon
c87b5b2cb6
GH-135379: Remove types from stack items in code generator. (GH-135384)
* Make casts explicit in the instruction definitions
2025-06-11 15:52:25 +01:00
Petr Viktorin
49d72365cd
gh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (GH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)

This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.

The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.

* Specify minimum PyGC_Head and PyObject alignment

As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.

Encode this requirement using _Py_ALIGNED_DEF.

This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.

Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
2025-06-11 12:44:58 +02:00
Justin Applegate
2b8b4774d2
gh-135321: Always raise a correct exception for BINSTRING argument > 0x7fffffff in pickle (GH-135322)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-06-11 10:15:12 +00:00
Serhiy Storchaka
0f866cbfef
gh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UTF-8' (#135347) 2025-06-10 13:38:32 +00:00
Pablo Galindo Salgado
ff2b5f40c2
gh-130077: Properly match full soft keywords in the parser (#135317) 2025-06-10 14:19:03 +01:00
Ajay Kamdar
598aa7cc98
gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222)
When shutdown is called with wait=False, the executor thread keeps running
even after the ProcessPoolExecutor's state is reset. The executor then tries
to replenish the worker processes pool resulting in an error and a potential hang
when it comes across a worker that has died. Fixed the issue by having
_adjust_process_count() return without doing anything if the ProcessPoolExecutor's
state has been reset.

Added unit tests to validate two scenarios:
max_workers < num_tasks (exception)
max_workers > num_tasks (exception + hang)
2025-06-10 13:28:31 +02:00
Alper
a58026a5e3
gh-116738: Make _heapq module thread-safe (#135036)
Use critical sections to make heapq methods that update the heap thread-safe when the GIL is disabled.

---------

Co-authored-by: mpage <mpage@meta.com>
2025-06-09 10:57:29 -07:00
Noam Cohen
b150b6aca7
gh-131798: Optimize _UNARY_INVERT (GH-135222) 2025-06-09 18:33:18 +08:00
Edward Z. Yang
8441b263af
bpo-45210: Document that error indicator may be set in tp_dealloc (#28358)
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-06-09 08:56:32 +00:00
Bénédikt Tran
3cb109796d
gh-135004: rewrite and cleanup blake2module.c (#135006)
* Cleanup imports and update module docstring.
* Simplify detection of SIMD support.
* Correctly guard `update()` cases.
* Rewrite `py_blake2b_or_s_new` and rename it to `py_blake2_new`.
* Rewrite `blake2_blake2b_copy_locked` and `py_blake2_clear`.
* Refactor computations of `digest` and `hexdigest`.
* Simplify `py_blake2b_get_name` and `py_blake2b_get_block_size`.
* Add `hacl_get_blake2_info` to extract static BLAKE-2 information.
   This new helper is used by `py_blake2b_get_digest_size`, but can
   be later used to expose `key_length` more easily.
2025-06-09 09:17:43 +02:00
Bénédikt Tran
83b94e856e
gh-135234: improve _hashlib exceptions when reporting an OpenSSL function failure (#135250)
- Refactor `get_openssl_evp_md_by_utf8name` error branches.
- Refactor `HASH.{digest,hexdigest}` computations.
- Refactor `_hashlib_HASH_copy_locked` and `locked_HMAC_CTX_copy`.
2025-06-09 09:13:29 +02:00
Jason R. Coombs
8d6eb0c262
gh-135276: Refresh zipfile.Path from zipp 3.23 (#135277)
Apply changes from zipp 3.23
2025-06-08 19:20:20 +00:00
LamentXU
1cb7163872
gh-135244: generate UUID random Node ID with a CSPRNG as per RFC 9562, §6.10.3 (#135226)
This aligns with the recommendations of RFC 9562, Section 6.10, paragraph 3 [1].

[1]: https://www.rfc-editor.org/rfc/rfc9562.html#section-6.10-3.

---------

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-06-08 11:46:16 +00:00
Jiucheng(Oliver)
bcb6b45cb8
gh-134151 Fix TypeError in email.utils.decode_params when sorting RFC 2231 continuations (#134687)
- Fix sorting logic in `email.utils.decode_params` to handle None values.
- Update tests for RFC 2231 continuation sorting.
2025-06-08 09:13:21 +02:00
Amit Lavon
8fdbbf8b18
GH-131798: Type-propagate string/list/tuple slices (GH-134671) 2025-06-07 14:08:44 -07:00
Daniel Golding
ac9c3431cc
gh-134876: Add fallback for when process_vm_readv fails with ENOSYS (#134878) 2025-06-07 19:32:06 +01:00
Petr Viktorin
24069fbca8
Revert "gh-133390: Support SQL keyword completion for sqlite3 CLI (#133393)" temporarily (GH-135232)
This reverts commit 62b3d2d443,
which broke buildbots
2025-06-07 08:56:43 +00:00
Tomas R.
46151648ca
GH-131798: Optimize away type(x) in the JIT when the result is known (GH-135194) 2025-06-06 16:44:43 -07:00
Tan Long
62b3d2d443
gh-133390: Support SQL keyword completion for sqlite3 CLI (#133393) 2025-06-06 16:52:41 +02:00
Petr Viktorin
e413e26719
gh-134891: Add PyUnstable_Unicode_GET_CACHED_HASH (GH-134892) 2025-06-06 15:51:00 +02:00
Serhiy Storchaka
6ef06fad84
gh-135120: Add test.support.subTests() (GH-135121) 2025-06-06 13:52:48 +02:00
sobolevn
0d9ccc87a2
gh-134036: Improve error messages for invalid raise statements (#134077) 2025-06-06 01:51:06 +01:00
Sergey Miryanov
d9cad074d5
gh-134155: fix AttributeError in email._header_value_parser.get_address (#134194)
Append the defect to defects instead of to the parse tree.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-06-05 13:28:11 -04:00
Victor Stinner
9258f3da91
gh-134989: Fix Py_RETURN_NONE in the limited C API (GH-135165)
Fix Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE macros in the
limited C API 3.11 and older:
Don't treat Py_None, Py_True and Py_False as immortal.
2025-06-05 14:43:47 +02:00
mpage
6b77af257c
gh-134889: Fix handling of a few opcodes when optimizing LOAD_FAST (#134958)
We were incorrectly handling a few opcodes that leave their operands on the stack. Treat all of these conservatively; assume that they always leave operands on the stack.
2025-06-04 16:07:58 -07:00
Victor Stinner
c21113072c
gh-134989: Implement PyObject_DelAttr() as a macro in the limited C API (GH-135021) 2025-06-04 15:07:52 +02:00
Sam Gross
cc581f32bf
gh-135099: Only wait on _PyOS_SigintEvent() in main thread (GH-135100)
On Windows, the `_PyOS_SigintEvent()` event handle is used to interrupt
the main thread when Ctrl-C is pressed. Previously, we also waited on
the event from other threads, but ignored the result. However, this can
race with interpreter shutdown because the main thread closes the handle
in `_PySignal_Fini` and threads may still be running and using mutexes
during interpreter shtudown.

Only use `_PyOS_SigintEvent()` in the main thread in parking_lot.c, like
we do in other places in the CPython codebase.
2025-06-04 09:35:56 +02:00
Duane Griffin
1ffe913c20
gh-127081: use getlogin_r if available (gh-132751)
The `getlogin` function is not thread-safe: replace with `getlogin_r` where
available.
2025-06-03 13:28:58 -04:00
Łukasz Langa
3612d8f517
gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (#135037)
Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517.

Signed-off-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-06-03 12:42:11 +02:00
Roei Ben Artzi
ec12559eba
gh-131884: Fix incorrect formatting in json.dumps() when using indent and skipkeys=True (GH-132200) 2025-06-03 10:40:25 +03:00
Serhiy Storchaka
df98a47a61
gh-132813: Improve error messages for incorrect types and values of csv.Dialog attributes (GH-133241)
Make them similar to PyArg_Parse error messages, mention None as
a possible value, show a wrong type and the string length.
2025-06-02 23:35:41 +03:00
Serhiy Storchaka
0cec424af5
gh-66234: Add flag to disable the use of mmap in dbm.gnu (GH-135005)
This may harm performance, but improve crash tolerance.
2025-06-02 21:08:26 +03:00
Duane Griffin
44fb7c361c
gh-134908: Protect textiowrapper_iternext with critical section (gh-134910)
The `textiowrapper_iternext` function called `_textiowrapper_writeflush`, but did not
use a critical section, making it racy in free-threaded builds.
2025-06-02 17:22:41 +00:00
Pieter Eendebak
26a1cd4e8c
gh-123471: make concurrent iteration over itertools.cycle safe under free-threading (#131212)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-06-02 20:13:32 +05:30
Duane Griffin
b6237c3602
gh-117852: fix argument checking of async_generator.athrow (#134868)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-06-02 20:04:26 +05:30
Sergey B Kirpichev
9c72658e49
gh-130662: Accept leading zeros in precision/width for Decimal's formatting (#132549)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-06-02 15:30:52 +02:00
Sergey B Kirpichev
5bc2d99126
gh-130662: Accept leading zeros in precision/width for Fraction's formatting (#130663)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-06-02 15:28:20 +02:00
devdanzin
baccfdb3d4
gh-130999: Avoid exiting the new REPL when there are non-string candidates for suggestions (gh-131001) 2025-06-02 13:04:59 +02:00
Sergey B Kirpichev
5f61cde80a
gh-132908: Add math.isnormal/issubnormal() functions (GH132935) 2025-06-02 13:38:05 +03:00
Bénédikt Tran
ee65ebdb50
gh-134978: deprecate string keyword parameter for hash function constructors (#134979) 2025-06-02 10:25:50 +02:00
Andrea-Oliveri
f806463e16
gh-134004: Added the reorganize() methods to dbm.sqlite, dbm.dumb and shelve (GH-134028)
They are similar to the same named method in dbm.gnu.
2025-06-01 15:30:04 +03:00
Michał Górny
965c480566
gh-134970: Fix exception message in argparse module (GH-134971)
Fix the "unknown action" exception in argparse.ArgumentParser.add_argument_group()
to correctly replace the action class.
2025-06-01 08:56:56 +03:00
Serhiy Storchaka
ad39f01788
gh-108885: Use subtests for doctest examples run by unittest (GH-134890)
Run each example as a subtest in unit tests synthesized by
doctest.DocFileSuite() and doctest.DocTestSuite().

Add the doctest.DocTestRunner.report_skip() method.
2025-05-31 13:01:46 +03:00
Serhiy Storchaka
68784fed78
gh-133489: Remove size restrictions on getrandbits() and randbytes() (GH-133658)
random.getrandbits() can now generate more that 2**31 bits.
random.randbytes() can now generate more that 256 MiB.
2025-05-31 11:23:01 +03:00
Bénédikt Tran
c6e63d9d35
gh-134696: align OpenSSL and HACL*-based hash functions constructors AC signatures (#134713)
OpenSSL and HACL*-based hash functions constructors now support both `data` and `string` parameters.
Previously these constructor functions inconsistently supported sometimes `data` and sometimes `string`,
while the documentation expected `data` to be given in all cases.
2025-05-31 09:37:47 +02:00
Serhiy Storchaka
4d31d19a1d
gh-134718: Omit optional Load() values in ast.dump() (GH-134934) 2025-05-31 10:32:53 +03:00
Steve Dower
8865b4f95b
gh-134923: Use /GENPROFILE and /USEPROFILE for Windows PGO builds (GH-134924) 2025-05-30 19:37:29 +01:00
Serhiy Storchaka
cc344e8dd0
gh-134718: Fix ast.dump() for empty non-default values (GH-134926) 2025-05-30 17:25:07 +03:00
Victor Stinner
ebf6d13567
gh-134745: Change PyThread_allocate_lock() implementation to PyMutex (#134747)
Co-authored-by: Sam Gross <colesbury@gmail.com>
2025-05-30 10:15:47 +00:00
Jelle Zijlstra
45c6c48afc
gh-134885: zstd: Use Py_XSETREF (GH-134886) 2025-05-30 11:30:05 +02:00