Avoid locking in the PyType_Lookup cache-miss path if the type's
tp_version_tag is already valid.
(cherry picked from commit cd52172831)
Co-authored-by: Sam Gross <colesbury@gmail.com>
We already have a stop-the-world pause elsewhere in this code path
(type_set_bases) and this makes will make it easier to avoid contention
on the TYPE_LOCK when looking up names in the MRO hierarchy.
Also use deferred reference counting for non-immortal MROs.
(cherry picked from commit 0b65c88c2a)
Fix three issues that caused mimalloc pages to be leaked until the
owning thread exited:
1. In _PyMem_mi_page_maybe_free(), move pages out of the full queue
when relying on QSBR to defer freeing the page. Pages in the full
queue are never searched by mi_page_queue_find_free_ex(), so a page
left there is unusable for allocations.
2. Move _PyMem_mi_page_clear_qsbr() from _mi_page_free_collect() to
_mi_page_thread_free_collect() where it only fires when all blocks
on the page are free (used == 0). The previous placement was too
broad: it cleared QSBR state whenever local_free was non-NULL, but
_mi_page_free_collect() is called from non-allocation paths (e.g.,
page visiting in mi_heap_visit_blocks) where the page is not being
reused.
3. In _PyMem_mi_page_maybe_free(), use the page's heap tld to find the
correct thread state for QSBR list insertion instead of
PyThreadState_GET(). During stop-the-world pauses, the function may
process pages belonging to other threads, so the current thread
state is not necessarily the owner of the page.
(cherry picked from commit d76df75f51)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Fix thread-safety issues when accessing frame attributes while another
thread is executing the frame:
- Add critical section to frame_repr() to prevent races when accessing
the frame's code object and line number
- Add _Py_NO_SANITIZE_THREAD to PyUnstable_InterpreterFrame_GetLasti()
to allow intentional racy reads of instr_ptr.
- Fix take_ownership() to not write to the original frame's f_executable
(cherry picked from commit 5bb3bbb9c6)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-144330: Initialize classmethod and staticmethod in new
Initialize cm_callable and sm_callable to None in classmethod and
staticmethod constructor.
Co-authored-by: Aniket Singh Yadav <singhyadavaniket43@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
In `_Py_dict_lookup_threadsafe_stackref`, call `ensure_shared_on_read()` to
prevent a race between the lookup and concurrent dict resizes, which may free
the PyDictKeysObject (i.e., it ensures that the resize uses QSBR).
(cherry picked from commit e666a01ef4)
gh-102809: No longer mention `Misc/gdbinit` in the code (GH-143980)
Fix misleading comment on `PyObject_Dump`.
(cherry picked from commit f84ea11071)
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
This happens when the set contained several elements with the same hash,
and then some of them were removed.
(cherry picked from commit b8e925b4f8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Remove outdated comment about `excess_args` warning in `object.__init__` (GH-143669)
The code emitting a warning was removed in 96384b93aa.
(cherry picked from commit 03e6457096)
Co-authored-by: Manuel Jacob <me@manueljacob.de>
When comparing negative non-integer float and int with the same number
of bits in the integer part, __neg__() in the int subclass returning
not an int caused an assertion error.
Now the integer is no longer negated. Also, reduced the number of
temporary created Python objects.
(cherry picked from commit 66bca383bd)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
When __length_hint__() returns 0 for non-empty iterator, the data can be
written past the shared 0-terminated buffer, corrupting it.
(cherry picked from commit 522563549a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-142557: fix UAF in `bytearray.__mod__` when object is mutated while formatting `%`-style arguments (GH-143213)
(cherry picked from commit 61ee04834b)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-142560: prevent use-after-free in search-like methods by exporting buffer in bytearray (GH-142938)
(cherry picked from commit 220f0b1077)
Co-authored-by: wangxiaolei <fatelei@gmail.com>
gh-142554: avoid `divmod` crashes due to bad `_pylong.int_divmod` (GH-142673)
(cherry picked from commit 4e4163676a)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
There are places we use "relaxed" loads where C11 requires "consume" or
stronger. Unfortunately, compilers don't really implement "consume" so
fake it for our use in a way that avoids upsetting TSan.
(cherry picked from commit 0a62f8277e)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-142433: Move deref to below the error when checking for laststring (GH-142402)
Move deref of laststring to below the error checking so the deref
is applied after the object in strings is replaced.
(cherry picked from commit 785268fdce)
Co-authored-by: AZero13 <gfunni234@gmail.com>
This fixes a regression introduced in gh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
(cherry picked from commit 547d8daf78)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The dataclasses `__init__` function is generated dynamically by a call to `exec()` and so doesn't have deferred reference counting enabled. Enable deferred reference counting on functions when assigned as an attribute to type objects to avoid reference count contention when creating dataclass instances.
(cherry picked from commit ce79154176)
Co-authored-by: Edward Xu <xuxiangad@gmail.com>
This fixes an assertion error when the new computed start is not an integer.
(cherry picked from commit 10bec7c1eb)
Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>
gh-140815: Fix faulthandler for invalid/freed frame (GH-140921)
faulthandler now detects if a frame or a code object is invalid or
freed.
Add helper functions:
* _PyCode_SafeAddr2Line()
* _PyFrame_SafeGetCode()
* _PyFrame_SafeGetLasti()
_PyMem_IsPtrFreed() now detects pointers in [-0xff, 0xff] range
as freed.
(cherry picked from commit a84181c31b)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-140348: Fix using | on unusual objects plus Unions (GH-140383)
(cherry picked from commit 7a9437d986)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Remove dead stores to 'size' in UTF-8 decoder (unicodeobject.c) (GH-140637)
(cherry picked from commit 7d70a147f5)
Co-authored-by: Shamil <ashm.tech@proton.me>
The `make_gen()` function creates and tracks generator/coro objects, but
doesn't fully initialize all the fields. At a minimum, we need to
initialize all the fields that may be accessed by gen_traverse because
the call to `compute_cr_origin()` can trigger a GC.
(cherry picked from commit 574405c19e)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* Count number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
* Do not track most untrackable tuples during creation. This prevents large numbers of small tuples causing execessive GCs.
Currently, there are a few places where tp_mro could theoretically
become NULL, but do not in practice. This commit adds defensive checks for
NULL values to ensure that any changes do not introduce a crash and that
state invariants are upheld.
The assertions added in this commit are all instances where a NULL value would get passed to something not expecting a NULL, so it is better to catch an assertion failure than crash later on.
There are a few cases where it is OK for the return of lookup_tp_mro to be NULL, such as when passed to is_subtype_with_mro, which handles this explicitly.
(cherry picked from commit a8edca62fc)
Co-authored-by: Emma Smith <emma@emmatyping.dev>
gh-140406: Fix memory leak upon `__hash__` returning a non-integer (GH-140411)
(cherry picked from commit 71db05a12d)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
