gh-146452: Fix pickle segfault on concurrent mutation of dict in pickle (GH-146470)
(cherry picked from commit e62a61177f)
Co-authored-by: Farhan Saif <fsaif@uic.edu>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
This has not been observed in practice, but we cannot be 100% sure that
it will not happen with some weird gzip data.
(cherry picked from commit 28eac9a726)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-149995: Update typing.py docstrings and documentation (GH-149996)
Some of these docstrings read as if they were written when typing.py was
first written, and things have evolved since then.
A few motivations:
- Call protocols protocols instead of ABCs. They are also ABCs, but the fact
they are protocols is more relevant to typing.
- Avoid recommending direct use of .__annotations__ and steer users to
annotationlib instead.
- For TypedDict, mention NotRequired before total=False since it is more
general and probably more frequently useful.
- For overloads, mention runtime use first instead of stub use. I think early on
there was talk of allowing overload only in stubs, but it is now heavily used at
runtime too and that's more likely to be relevant to users.
(cherry picked from commit f159419ae2)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-149819: fix .pth and .start file processing in subprocess when inheriting PYTHONPATH (GH-150177)
* gh-149819: Fix .pth files not loaded in Python subprocesses
After PR gh-149583 (Fix double evaluation of .pth and .site files in
venvs), .pth files are no longer loaded in subprocesses started with
subprocess.run([sys.executable, ...]). The root cause: main() seeds
known_paths from removeduppaths() with all sys.path entries inherited
from the parent process. addsitedir() then skips .pth processing for
every directory already in known_paths.
Fix:
- main(): call removeduppaths() for dedup but start known_paths as a
fresh empty set, so that addsitedir() processes .pth files in every
site-packages directory regardless of inherited sys.path.
- addsitedir(): move known_paths.add() before the sys.path.append and
guard the append with 'sitedir not in sys.path' to avoid creating
duplicate entries when called with a fresh known_paths.
This preserves the gh-75723 dedup guarantee while allowing subprocesses
to load .pth files.
* Fill out the tests for GH#149888
* Extend _make_start() and _make_pth() to take an optional `basedir` which is used instead of
`site.tmpdir` if given.
* Add test_pth_processed_when_sitedir_already_on_path() to test the core GH#149819 bug: .pth files
in subprocesses aren't handled if PYTHONPATH pointing to the .pth directory is inherited.
* Similarly add test_start_processed_when_sitedir_already_on_path() to verify that .start files in
the same circumstances are also now processed.
* Update Lib/site.py
* Oops! Remove redundant code
---------
(cherry picked from commit 3c298e2e38)
Co-authored-by: Barry Warsaw <barry@python.org>
Co-authored-by: BugBounty Mind <bugbounty-mind@deepseek.tui>
Co-authored-by: scoder <stefan_ml@behnel.de>
gh-149219: Test `frozendict` in `Lib/test/test_crossinterp.py` (GH-149220)
(cherry picked from commit c35b0f2b62)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-134261: ZipFile - Don't rely on local time for reproducible builds & tests (GH-134264)
---------
(cherry picked from commit 9dcf94e906)
Co-authored-by: Caleb <23644849+ctrlaltf2@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Emma Smith <emma@emmatyping.dev>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Reduce NUMITEMS from 100000 to 5000. Peak RSS for the full
test_free_threading suite drops from ~850 MB to ~175 MB.
(cherry picked from commit 61f12211fc)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Also disables the UWP build in CI, since it was breaking (and is no longer released).
(cherry picked from commit ec9ce3ee98)
Co-authored-by: Steve Dower <steve.dower@python.org>
Do not read the whole filename and comment to memory for calculating the CRC.
(cherry picked from commit 51a5715df9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
bpo-42367: Restore os.makedirs() and pathlib.mkdir() ability to apply *mode* recursively via a new parent_mode= keyword argument.
(cherry picked from commit 9770e32ce0)
+ Make Path.mkdir parent_mode tests umask-independent
test_mkdir_with_parent_mode, test_mkdir_parent_mode_deep_hierarchy and
test_mkdir_parent_mode_same_as_mode assert exact directory mode bits but
did not pin the process umask. On buildbots running with a restrictive
umask (e.g. 0o077) the 0o755 leaf was masked down to 0o700, failing the
assertions. Wrap them in os_helper.temp_umask(0o022), matching the
other umask-aware mkdir tests in this file.
---------
Co-authored-by: nessita <124304+nessita@users.noreply.github.com>
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-149977: Fix extra output of `-m test test_lazy_import` (GH-149978)
(cherry picked from commit 6d5be4b1d6)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-87451: Apply CVE-2021-4189 PASV fix to ftplib.ftpcp() (GH-149648)
ftpcp() called parse227() directly and passed the source server's
self-reported PASV IPv4 address to the target server's PORT command,
bypassing the CVE-2021-4189 fix that was applied only to FTP.makepasv().
A malicious source FTP server could use this to redirect the target
server's data connection to an arbitrary host:port (SSRF).
ftpcp() now uses the source server's actual peer address, honoring the
existing trust_server_pasv_ipv4_address opt-out, the same as makepasv().
Thanks to Qi Ding at Aurascape AI for the report. (GHSA-w8c5-q2xf-gf7c)
(cherry picked from commit eac4fe3b2c)
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
gh-149144: Use decodeURIComponent() for UTF-8 support in js_output() (GH-149157)
(cherry picked from commit 461b1d9631)
Co-authored-by: Seth Larson <seth@python.org>
* gh-149504: Fix re-entrancy bug when .pth/.start file invokes site.addsitedir() (#149659)
* Add re-entrant tests for gh-149504
* Add end-to-end integration test coverage
This ensures that future whitebox internal test changes do not regress the
public surface semantics.
* Implement a state class to process .pth and .start files
By using this state class and managing implicit and explicit batching, we make it structurally
impossible to get bitten by re-entrant site startup processing.
Fixes#149504
(cherry picked from commit b162307d7f)
* Add myself back to CODEOWNERS
RFC 2047 Section 6.2 requires that "any 'linear-white-space' that
separates a pair of adjacent 'encoded-word's is ignored." The modern
header value parser correctly implements that for unstructured headers,
but had missed a case in structured headers. This could cause a parsed
address header to include extraneous spaces in a display-name.
Switch to @bitdancer's fix from review feedback. Recharacterize space
between ews as fws after parsing in get_phrase.
RDM: This fix is dependent on the fact that "subsequent" atoms will never have
leading whitespace because that's been consumed already. I don't think
it's worth adding extra code for the possibility of leading whitespace
because the parser won't produce it. It's a bit of parser fragility in the
face of code changes, but I think that's a minor concern given the
parser design (which is that it consumes whitespace greedily)
(cherry picked from commit 7a4c6dfb88)
Co-authored-by: Mike Edmunds <medmunds@gmail.com>
Co-authored-by: R David Murray <rdmurray@bitdance.com>
gh-149776: Skip UDP Lite tests if it's not supported (GH-149777)
Fix test_socket on Linux kernel 7.1 and newer: skip UDP Lite tests if
it's not supported.
(cherry picked from commit 3cfc249e11)
Co-authored-by: Victor Stinner <vstinner@python.org>
Exclude encodings like 'utf-8-sig', 'iso2022-jp' and 'hz' from the list of
supported encodings.
(cherry picked from commit fa2afa64d9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Check the header checksum it the HCRC field is present.
(cherry picked from commit dd94457893)
Co-authored-by: Ruben Vorderman <r.h.p.vorderman@lumc.nl>
gh-149718: Aggregate same stack frames in Tachyon in some collectors (GH-149719)
(cherry picked from commit 76f2285341)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-149496: Fix MacOSTest.test_default regression when BROWSER env var is set (GH-149579)
gh-149496: Fix MacOSTest.test_default failing when BROWSER env var is set
MacOSTest.test_default calls webbrowser.get() and asserts it returns a
MacOS instance. When BROWSER is set in the environment (e.g. BROWSER=open,
a common macOS workaround for the old osascript-based implementation),
register_standard_browsers() registers a GenericBrowser as the preferred
browser instead, causing the assertion to fail.
This is a regression introduced in gh-137586, which added MacOSTest and
moved test_default into it from MacOSXOSAScriptTest. MacOSXOSAScriptTest
had an identical setUp() guard added in gh-131254 specifically to fix this
same failure. The guard was not carried over to MacOSTest.
Add setUp() to MacOSTest to unset BROWSER for the duration of each test,
restoring the isolation that was already established as the correct pattern
for macOS webbrowser tests.
(cherry picked from commit 45c47d26c2)
Co-authored-by: Jeff Lyon <146767590+secengjeff@users.noreply.github.com>
In free-threaded builds, concurrent calls to PyDict_AddWatcher, PyDict_ClearWatcher, PyDict_Watch, and PyDict_Unwatch can race on the shared callback array and the per-dict watcher tags. This change adds a mutex to serialize watcher registration and removal, atomic operations for tag updates, and atomic acquire/release synchronization for callback dispatch in _PyDict_SendEvent.
(cherry picked from commit 8a4895985f)
Co-authored-by: Alper <alperyoney@fb.com>
gh-112821: Fix rlcompleter failures on objects with descriptors (GH-149577)
* gh-112821: Fix rlcompleter failures on objects with descriptors
* Confirm no accesses
(cherry picked from commit f23a1837d7)
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
