gh-123619: Fix PyUnstable_Object_EnableDeferredRefcount() (GH-151260)
Return 0 if the object is not tracked by the GC.
(cherry picked from commit 72e7eddce6)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-151278: Fix test_faulthandler on UBSan (GH-151279)
* Py_FatalError() no longer calls _PyFaulthandler_Fini() if it
doesn't hold the GIL.
* Skip test_faulthandler tests raising signals if run with UBSan.
* Enable test_faulthandler in GitHub Action "Reusable Sanitizer".
(cherry picked from commit e60c42dc3f)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-151253: Dump the Python path configuration on _PyCodec_InitRegistry() failure (#151250)
If "import encodings" fails at Python startup, dump the Python path
configuration to help users debugging their configuration. The
encodings module is the first module imported during Python startup.
(cherry picked from commit 7b6e98911e)
gh-143988: Fix re-entrant mutation crashes in socket sendmsg/recvmsg_into (GH-143987)
Fix crashes in socket.sendmsg() and socket.recvmsg_into() that could
occur if buffer sequences are mutated re-entrantly during argument
parsing via __buffer__ protocol callbacks.
The bug occurs because:
1. PySequence_Fast() returns the original list object when the input
is already a list (not a copy).
2. During iteration, PyObject_GetBuffer() triggers __buffer__
callbacks which may clear the list.
3. Subsequent iterations access invalid memory (heap OOB read).
The fix replaces PySequence_Fast() with PySequence_Tuple() which
always creates a new tuple, ensuring the sequence cannot be mutated
during iteration.
(cherry picked from commit 896f7fdc7d)
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
Co-authored-by: tonghuaroot <23011166+tonghuaroot@users.noreply.github.com>
Add yet one assertion in test_set_text_charset_cp949 (GH-151224)
Check bytes(m), not only str(m).
(cherry picked from commit 3ca93ab198)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-150988: Fix refleak in `OSError` when attrs are set before `super().__init__()` (GH-150990)
(cherry picked from commit f2a0f82282)
Co-authored-by: Lukas Geiger <lukas.geiger94@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-98894: Fix DTrace test_check_probes for shared builds (GH-151122)
When building with --enable-shared, the SystemTap/DTrace notes
live in libpython. Add detection logic to be used by readelf.
Force the C locale on readelf output.
(cherry picked from commit 3a08e93739)
Co-authored-by: stratakis <cstratak@redhat.com>
gh-150700: Fix class-scope inline comprehensions when nested scopes reference `__class__` and friends (GH-150735)
* Fix class-scope inline comprehensions when nested scopes reference `__class__` and friends
In `inline_comprehension()`, when `__class__` / `__classdict__` /
`__conditional_annotations__` appears as `FREE` in a comprehension's
symbol table because a nested scope captured it (e.g. nested lambdas),
this name is still discarded from `comp_free` unconditionally.
This prevents `drop_class_free()` from seeing it, so the appropriate
`ste_needs_(...)` flag is never set on the enclosing class.
That leads to `codegen_make_closure()` throwing `SystemError` when it
couldn't find `__class__` / `__classdict__` /
`__conditional_annotations__` in the class's cellvars.
From now on we just discard from `comp_free` when no child scope
(e.g. a lambda) still needs the name as `FREE`. When a child scope does
need it, keep it in `comp_free` so `drop_class_free()` can set the
appropriate flag and the class creates the implicit cell.
* Fix tests
* Fix typo
* Fix formatting
* Add test checking validity of `__class__` returned
* Prefer 'used' to 'deferred'
(cherry picked from commit ce916dc506)
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Nonatomic move might be used even if the files are
on the same filesystem in some cases.
(cherry picked from commit 6ecd197c03)
Co-authored-by: Fang Li <fangli@users.noreply.github.com>
gh-150633: Minor improvement of a newly added test (GH-151103)
Minor improvement of a newly added test.
(cherry picked from commit fccf67a354)
Co-authored-by: Barry Warsaw <barry@python.org>
gh-151019: Fix test_os on 32-bit FreeBSD (GH-151087)
Remove references to server.handler_instance. This attribute has been
removed in 2022 by commit 3ae975f1ac.
(cherry picked from commit a9002349cb)
Co-authored-by: Victor Stinner <vstinner@python.org>
This was added in 3.15; let's use a real sentinel instead of an ad-hoc list object.
(cherry picked from commit 884ac3e3ec)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-149977: Fix extra output of `-m test test_lazy_import`, again (GH-150965)
(cherry picked from commit 9b4090c48e)
Co-authored-by: sobolevn <mail@sobolevn.me>
Defer to the codecs module for all aliases.
Use MIME/IANA names for all IANA registered charsets.
Fix email.contentmanager.set_text_content().
(cherry picked from commit c195a046f8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Support all aliases officially registered in IANA, except
Extended_UNIX_Code_Packed_Format_for_Japanese.
New names:
KSC_5601, KS_C_5601-1989, iso-ir-149, GB_2312-80, windows-936, mac,
CCSID00858, CCSID01140, and a number of "cs"-prefixed names.
Fix csHPRoman8, which was not normalized.
(cherry picked from commit 49f4ecfb08)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
ass_subscript_slice() returned early when the computed slice length
was zero, bypassing validation performed for non-empty slices.
(cherry picked from commit fc9c4db130)
Co-authored-by: Jiseok CHOI <jiseok.dev@gmail.com>
gh-150750: Fix a race condition in `deque.index` with free-threading (GH-150779)
(cherry picked from commit d83d50b5b7)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-150436: Skip subprocess test on STATUS_DLL_INIT_FAILED (GH-150704)
If a subprocess spawned with CREATE_NEW_CONSOLE creation flag fails
with STATUS_DLL_INIT_FAILED return code, skip the test. It's likely a
memory allocation failure in the desktop heap memory which caused the
DLL init failure.
(cherry picked from commit e8034dd841)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-148613: Fix race in `gc_set_threshold` and `gc_get_threshold` (GH-150356)
(cherry picked from commit 41eb8ee2bb)
Co-authored-by: Edward Xu <xuxiangad@gmail.com>
gh-132467: Document and test that generic aliases are not classes (GH-133504)
(cherry picked from commit 5915a1fb9d)
Co-authored-by: Abduaziz π <mail@ziyodov.uz>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* bpo-46943: fix[imaplib]: call Exception with string instance
Adjust the behavior of 'login' to be similar to `authenticate()`,
where self.error is called with a str() instance.
(cherry picked from commit 29805f00a1)
Co-authored-by: Florian Best <spaceone@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
gh-150723: Fix perf jitdump files on macOS (GH-150728)
The perf jitdump format defines the thread id field of the JR_CODE_LOAD
record as a 32-bit value, but on macOS it was declared as a uint64_t
(since pthread_threadid_np() returns a uint64_t). Those extra 8 bytes
plus alignment padding shifted every following field, so parsers reading
the file by the spec misread code_size as the code address and failed to
resolve any Python frames.
Declare thread_id as uint32_t on all platforms and truncate the macOS
thread id when writing the record. The value is only informational.
Symbols are resolved by address, and not thread ids so truncation is
safe here.
* Use mach_absolute_time for macOS jitdump timestamps
On macOS the jitdump file is consumed by profilers such as samply, which
timestamp their samples using mach_absolute_time(). The jitdump events were
stamped with clock_gettime(CLOCK_MONOTONIC), a different clock domain that
keeps advancing while the system is asleep, so the JIT code mappings could be
off by days relative to the samples and no Python frame would resolve. Stamp
jitdump events with mach_absolute_time() on macOS so they share the sampler's
clock domain. Linux continues to use CLOCK_MONOTONIC to stay aligned with perf.
Exercise the -Xperf_jit (jitdump) backend through samply and assert that
Python frames resolve, exercising the binary jitdump path end to end.
Skipped when samply is not installed.
(cherry picked from commit 494f2e3c92)
Co-authored-by: Nazım Can Altınova <canaltinova@gmail.com>
Fixed incorrect word.
(cherry picked from commit f7e0fb60cf)
Co-authored-by: Gustaf <79180496+gostak-dd@users.noreply.github.com>
Co-authored-by: Gustaf <79180496+GGyll@users.noreply.github.com>
Replace the insertion sort used for canonical ordering of combining
characters with a hybrid approach: insertion sort for short runs (< 20)
and counting sort for longer runs, reducing worst-case complexity from
O(n^2) to O(n). This prevents denial of service via crafted Unicode
strings with many combining characters in alternating CCC order.
(cherry picked from commit 991224b1e8)
Co-authored-by: Seth Larson <seth@python.org>
Co-authored-by: ch4n3-yoon <ch4n3.yoon@gmail.com>
Co-authored-by: Seokchan Yoon <13852925+ch4n3-yoon@users.noreply.github.com>
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-149738: Fix segmentation fault bug in sqllite3 (GH-149754)
Deleting the `row_factory` or `text_factory` attribute is no longer allowed.
(cherry picked from commit 60fdb3192b)
Co-authored-by: Sepehr Rasouli <sepehrrasouli06@gmail.com>
gh-150228: Improve the PEP 829 batch processing APIs (GH-150542)
* gh-150228: Improve the PEP 829 batch processing APIs
As previously discussed with @ncoghlan and approved for 3.15b2 by @hugovk,
this implements the batch processing APIs for addsitedir() and friends. We
remove the `defer_processing_start_files` flag which required some implicit
module global state, and promote StartupState to the public documented API.
This also moves the bulk of the module global functions into methods of the
`StartupState` class, so it removes the awkward APIs in 3.15b1. Now, instances
of this class are an accumulator for startup state, using `StartupState.process()`
to process them. Callers can now batch up startup state themselves by using
the methods on this class. The module global functions are shims for this
which preserve the legacy APIs and semantics using the new state class.
This PR also fixes the interleaving regression identified by @ncoghlan in the
same issue. Now, .pth file sys.path extensions are added to sys.path after
the sitedir that the .pth file is found in, restoring the legacy behavior.
Along the way, I've made a lot of improvements to function docstrings,
site.rst documentation, and comments in the code explaining what's going on.
* Add a note that if known_paths is provided to StartupState.__init__(), it
will get mutated in place.
* Improve some conditional flows.
* Improve some comments.
* Improve the what's new entry.
* Make test_impl_exec_imports_suppressed_by_matching_start() more robust
Based on PR comment, we need to read both the .pth and .start files, and prove
that the .pth file's import line (which passes a bigger increment) is not
called, but the .start file's entry point (which uses the default increment)
is called.
* As per review, move some methods to the private API
_read_pth_file() and _read_start_file() are not intended to be part of the
public API surface outside of the site module, so even though they are used by
methods outside of the StartupState class, make them privately named.
* Resolve several review feedbacks
* Move a `versionadded`
* Better list comprehension formatting (use the output from
`ruff format --line-length 78`)
* Add docs for site.makepath() and point the case-normalization requirement to
this utility function.
* Note that StartupState.process() is not idempotent.
* Address another feedback comment
This time, we get rid of the legacy implementation `reset` local, which was
always difficult to understand, and just implement a return value based on the
processing mode selected.
* Changes based on gh-150228 review
The comment by @encukou that started this change:
```
I still see two red flags here though: an argument that doesn't combine with
other arguments, and (another instance of) changing the return type based on
an argument.
Did you consider adding a StartupState.addsitedir(sitedir) method, instead of
the startup_state argument?
```
As it turns out, this is an even cleaner design. By moving the bulk of the
previous module global functions into `StartupState` methods, we can get rid
of all the awkward `startup_state` keyword-only arguments which conflict
with `known_path` (Petr's first point). We can also get rid of the
return value dichotomy (Petr's second point) because now we can preserve
exactly the Python 3.14 API in the module global functions, and implement
the better APIs in the class methods. We also generally don't have to
pass around `process_known_sitedirs`.
Now the following module global functions are essentially shims around
class methods:
* site.addsitedir() -> StartupState.addsitedir()
* site.addusersitepackages() -> StartupState.addusersitepackages()
* site.addsitepackages() -> StartupState.addsitepackages()
* Additional minor changes
* Remove a now unused parameter
(cherry picked from commit 27ebd9abce)
Co-authored-by: Barry Warsaw <barry@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
