Defer to the codecs module for all aliases.
Use MIME/IANA names for all IANA registered charsets.
Fix email.contentmanager.set_text_content().
(cherry picked from commit c195a046f8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Support all aliases officially registered in IANA, except
Extended_UNIX_Code_Packed_Format_for_Japanese.
New names:
KSC_5601, KS_C_5601-1989, iso-ir-149, GB_2312-80, windows-936, mac,
CCSID00858, CCSID01140, and a number of "cs"-prefixed names.
Fix csHPRoman8, which was not normalized.
(cherry picked from commit 49f4ecfb08)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
ass_subscript_slice() returned early when the computed slice length
was zero, bypassing validation performed for non-empty slices.
(cherry picked from commit fc9c4db130)
Co-authored-by: Jiseok CHOI <jiseok.dev@gmail.com>
gh-150750: Fix a race condition in `deque.index` with free-threading (GH-150779)
(cherry picked from commit d83d50b5b7)
Co-authored-by: sobolevn <mail@sobolevn.me>
* bpo-46943: fix[imaplib]: call Exception with string instance
Adjust the behavior of 'login' to be similar to `authenticate()`,
where self.error is called with a str() instance.
(cherry picked from commit 29805f00a1)
Co-authored-by: Florian Best <spaceone@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
gh-150228: Improve the PEP 829 batch processing APIs (GH-150542)
* gh-150228: Improve the PEP 829 batch processing APIs
As previously discussed with @ncoghlan and approved for 3.15b2 by @hugovk,
this implements the batch processing APIs for addsitedir() and friends. We
remove the `defer_processing_start_files` flag which required some implicit
module global state, and promote StartupState to the public documented API.
This also moves the bulk of the module global functions into methods of the
`StartupState` class, so it removes the awkward APIs in 3.15b1. Now, instances
of this class are an accumulator for startup state, using `StartupState.process()`
to process them. Callers can now batch up startup state themselves by using
the methods on this class. The module global functions are shims for this
which preserve the legacy APIs and semantics using the new state class.
This PR also fixes the interleaving regression identified by @ncoghlan in the
same issue. Now, .pth file sys.path extensions are added to sys.path after
the sitedir that the .pth file is found in, restoring the legacy behavior.
Along the way, I've made a lot of improvements to function docstrings,
site.rst documentation, and comments in the code explaining what's going on.
* Add a note that if known_paths is provided to StartupState.__init__(), it
will get mutated in place.
* Improve some conditional flows.
* Improve some comments.
* Improve the what's new entry.
* Make test_impl_exec_imports_suppressed_by_matching_start() more robust
Based on PR comment, we need to read both the .pth and .start files, and prove
that the .pth file's import line (which passes a bigger increment) is not
called, but the .start file's entry point (which uses the default increment)
is called.
* As per review, move some methods to the private API
_read_pth_file() and _read_start_file() are not intended to be part of the
public API surface outside of the site module, so even though they are used by
methods outside of the StartupState class, make them privately named.
* Resolve several review feedbacks
* Move a `versionadded`
* Better list comprehension formatting (use the output from
`ruff format --line-length 78`)
* Add docs for site.makepath() and point the case-normalization requirement to
this utility function.
* Note that StartupState.process() is not idempotent.
* Address another feedback comment
This time, we get rid of the legacy implementation `reset` local, which was
always difficult to understand, and just implement a return value based on the
processing mode selected.
* Changes based on gh-150228 review
The comment by @encukou that started this change:
```
I still see two red flags here though: an argument that doesn't combine with
other arguments, and (another instance of) changing the return type based on
an argument.
Did you consider adding a StartupState.addsitedir(sitedir) method, instead of
the startup_state argument?
```
As it turns out, this is an even cleaner design. By moving the bulk of the
previous module global functions into `StartupState` methods, we can get rid
of all the awkward `startup_state` keyword-only arguments which conflict
with `known_path` (Petr's first point). We can also get rid of the
return value dichotomy (Petr's second point) because now we can preserve
exactly the Python 3.14 API in the module global functions, and implement
the better APIs in the class methods. We also generally don't have to
pass around `process_known_sitedirs`.
Now the following module global functions are essentially shims around
class methods:
* site.addsitedir() -> StartupState.addsitedir()
* site.addusersitepackages() -> StartupState.addusersitepackages()
* site.addsitepackages() -> StartupState.addsitepackages()
* Additional minor changes
* Remove a now unused parameter
(cherry picked from commit 27ebd9abce)
Co-authored-by: Barry Warsaw <barry@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
gh-149534: Fix unification of `defaultdict` and `frozendict` with `|` (GH-149539)
(cherry picked from commit cc0269334f)
Co-authored-by: sobolevn <mail@sobolevn.me>
* The content of comments, processing instructions and elements "xmp",
"iframe", "noembed", "noframes", and "plaintext" is no longer escaped.
* The "plaintext" element no longer have the closing tag.
* Add support of empty attributes (with value None).
(cherry picked from commit bcd29e466f)
Preserve non-UTF-8 filenames when appending to a ZipFile.
---------
(cherry picked from commit 24c6bbc92b)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
It no longer emits text for comments and processing instructions.
(cherry picked from commit 7de4fcd445)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-150175: Fix ThreadingMock call_count race condition (GH-150176)
ThreadingMock._increment_mock_call() was not thread-safe.
Multiple threads calling the mock simultaneously could lose
increments due to race conditions on call_count and other
attributes.
Fix by overriding _increment_mock_call in ThreadingMixin
and wrapping it with the existing _mock_calls_events_lock.
(cherry picked from commit 388e023fe1)
Co-authored-by: saisneha196 <156835592+saisneha196@users.noreply.github.com>
gh-146452: Fix pickle segfault on concurrent mutation of dict in pickle (GH-146470)
(cherry picked from commit e62a61177f)
Co-authored-by: Farhan Saif <fsaif@uic.edu>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-149995: Update typing.py docstrings and documentation (GH-149996)
Some of these docstrings read as if they were written when typing.py was
first written, and things have evolved since then.
A few motivations:
- Call protocols protocols instead of ABCs. They are also ABCs, but the fact
they are protocols is more relevant to typing.
- Avoid recommending direct use of .__annotations__ and steer users to
annotationlib instead.
- For TypedDict, mention NotRequired before total=False since it is more
general and probably more frequently useful.
- For overloads, mention runtime use first instead of stub use. I think early on
there was talk of allowing overload only in stubs, but it is now heavily used at
runtime too and that's more likely to be relevant to users.
(cherry picked from commit f159419ae2)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-149819: fix .pth and .start file processing in subprocess when inheriting PYTHONPATH (GH-150177)
* gh-149819: Fix .pth files not loaded in Python subprocesses
After PR gh-149583 (Fix double evaluation of .pth and .site files in
venvs), .pth files are no longer loaded in subprocesses started with
subprocess.run([sys.executable, ...]). The root cause: main() seeds
known_paths from removeduppaths() with all sys.path entries inherited
from the parent process. addsitedir() then skips .pth processing for
every directory already in known_paths.
Fix:
- main(): call removeduppaths() for dedup but start known_paths as a
fresh empty set, so that addsitedir() processes .pth files in every
site-packages directory regardless of inherited sys.path.
- addsitedir(): move known_paths.add() before the sys.path.append and
guard the append with 'sitedir not in sys.path' to avoid creating
duplicate entries when called with a fresh known_paths.
This preserves the gh-75723 dedup guarantee while allowing subprocesses
to load .pth files.
* Fill out the tests for GH#149888
* Extend _make_start() and _make_pth() to take an optional `basedir` which is used instead of
`site.tmpdir` if given.
* Add test_pth_processed_when_sitedir_already_on_path() to test the core GH#149819 bug: .pth files
in subprocesses aren't handled if PYTHONPATH pointing to the .pth directory is inherited.
* Similarly add test_start_processed_when_sitedir_already_on_path() to verify that .start files in
the same circumstances are also now processed.
* Update Lib/site.py
* Oops! Remove redundant code
---------
(cherry picked from commit 3c298e2e38)
Co-authored-by: Barry Warsaw <barry@python.org>
Co-authored-by: BugBounty Mind <bugbounty-mind@deepseek.tui>
Co-authored-by: scoder <stefan_ml@behnel.de>
gh-134261: ZipFile - Don't rely on local time for reproducible builds & tests (GH-134264)
---------
(cherry picked from commit 9dcf94e906)
Co-authored-by: Caleb <23644849+ctrlaltf2@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Emma Smith <emma@emmatyping.dev>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
bpo-42367: Restore os.makedirs() and pathlib.mkdir() ability to apply *mode* recursively via a new parent_mode= keyword argument.
(cherry picked from commit 9770e32ce0)
+ Make Path.mkdir parent_mode tests umask-independent
test_mkdir_with_parent_mode, test_mkdir_parent_mode_deep_hierarchy and
test_mkdir_parent_mode_same_as_mode assert exact directory mode bits but
did not pin the process umask. On buildbots running with a restrictive
umask (e.g. 0o077) the 0o755 leaf was masked down to 0o700, failing the
assertions. Wrap them in os_helper.temp_umask(0o022), matching the
other umask-aware mkdir tests in this file.
---------
Co-authored-by: nessita <124304+nessita@users.noreply.github.com>
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-146452: Improve locking granularity in pickle's batch_dict_exact and fix race condition (GH-150025)
Remove assertion that could fail in rare race condition.
Replace the coarse critical section wrapping the entire function with
fine-grained sections covering only PyDict_Next + Py_INCREF.
Also handle PyDict_Next returning 0 in the single-item fast path.
(cherry picked from commit 57a0e570d3)
Co-authored-by: Saul Cooperman <58375603+scopreon@users.noreply.github.com>
gh-149921: Fix reference leaks in _interpchannels and _interpqueues modules (GH-149922)
(cherry picked from commit acefff95ea)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
* gh-149504: Fix re-entrancy bug when .pth/.start file invokes site.addsitedir() (#149659)
* Add re-entrant tests for gh-149504
* Add end-to-end integration test coverage
This ensures that future whitebox internal test changes do not regress the
public surface semantics.
* Implement a state class to process .pth and .start files
By using this state class and managing implicit and explicit batching, we make it structurally
impossible to get bitten by re-entrant site startup processing.
Fixes#149504
(cherry picked from commit b162307d7f)
* Add myself back to CODEOWNERS
RFC 2047 Section 6.2 requires that "any 'linear-white-space' that
separates a pair of adjacent 'encoded-word's is ignored." The modern
header value parser correctly implements that for unstructured headers,
but had missed a case in structured headers. This could cause a parsed
address header to include extraneous spaces in a display-name.
Switch to @bitdancer's fix from review feedback. Recharacterize space
between ews as fws after parsing in get_phrase.
RDM: This fix is dependent on the fact that "subsequent" atoms will never have
leading whitespace because that's been consumed already. I don't think
it's worth adding extra code for the possibility of leading whitespace
because the parser won't produce it. It's a bit of parser fragility in the
face of code changes, but I think that's a minor concern given the
parser design (which is that it consumes whitespace greedily)
(cherry picked from commit 7a4c6dfb88)
Co-authored-by: Mike Edmunds <medmunds@gmail.com>
Co-authored-by: R David Murray <rdmurray@bitdance.com>
Check the header checksum it the HCRC field is present.
(cherry picked from commit dd94457893)
Co-authored-by: Ruben Vorderman <r.h.p.vorderman@lumc.nl>
gh-149718: Aggregate same stack frames in Tachyon in some collectors (GH-149719)
(cherry picked from commit 76f2285341)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-112821: Fix rlcompleter failures on objects with descriptors (GH-149577)
* gh-112821: Fix rlcompleter failures on objects with descriptors
* Confirm no accesses
(cherry picked from commit f23a1837d7)
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
