* gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.
thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.
Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
(cherry picked from commit 517cd82ea7)
* gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808)
(cherry picked from commit 1b8f2366b3)
---------
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
gh-110052: Fix faulthandler for freed tstate (#110069)
faulthandler now detected freed interp and freed tstate, and no
longer dereference them.
(cherry picked from commit 2e37a38bcb)
gh-109098: Fuzz re module instead of internal sre (GH-109911)
* gh-109098: Fuzz re module instead of internal sre
* Fix c-analyzer globals test failure
* Put globals exception in ignored.tsv
(cherry picked from commit a829356f86)
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
gh-109631: Allow interruption of short repeated regex matches (GH-109867)
Counting for signal checking now continues in new match from the point where
it ended in the previous match instead of starting from 0.
(cherry picked from commit 8ac2085b80)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-109613: _pystat_fromstructstat() checks for exceptions (GH-109618)
Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.
_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():
Fatal Python error: _Py_CheckSlotResult:
Slot * of type int succeeded with an exception set
(cherry picked from commit d4cea794a7)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-104690: thread_run() checks for tstate dangling pointer (#109056)
thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.
Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
(cherry picked from commit f63d37877a)
gh-109496: Detect Py_DECREF() after dealloc in debug mode (GH-109539)
On a Python built in debug mode, Py_DECREF() now calls
_Py_NegativeRefcount() if the object is a dangling pointer to
deallocated memory: memory filled with 0xDD "dead byte" by the debug
hook on memory allocators. The fix is to check the reference count
*before* checking for _Py_IsImmortal().
Add test_decref_freed_object() to test_capi.test_misc.
(cherry picked from commit 0bb0d88e2d)
Co-authored-by: Victor Stinner <vstinner@python.org>
Add Modules/_testcapi/util.h header (GH-108774)
It contains common macros used in C API tests.
(cherry picked from commit 0e01fac315)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit 8178a88bd8)
- Add param docstrings
- Link to os.SEEK_* constants
- Mention the return value in the initial paragraph
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Clarify distinction between datetime module and class in deprecation messages (GH-108073)
(cherry picked from commit 09343dba44)
Co-authored-by: Clément Robert <cr52@protonmail.com>
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
(cherry picked from commit 2b15536fa9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit 7f87ebbc3f)
Clearly document the supported seek() operations:
- Rewind to the start of the stream
- Restore a previous stream position (given by tell())
- Fast-forward to the end of the stream
* gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
(cherry picked from commit 633ea217a8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Define PY_SSIZE_T_CLEAN.
---------
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
- Add explanatory comments
- Add return value to connection_close() for propagating errors
- Always check the return value of connection_exec_stmt()
- Assert pre/post state in remove_callbacks()
- Don't log unraisable exceptions in case of interpreter shutdown
- Make sure we're not initialized if reinit fails
- Try to close the database even if ROLLBACK fails
(cherry picked from commit fd19509220)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
- name the last parameter *whence*, like it is for seek() methods on
file objects
- add param docstrings
- structure the valid *whence* params
(cherry picked from commit dd4442c8f5)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Cover all the Mapping Protocol, almost all the Sequence Protocol
(except PySequence_Fast) and a part of the Object Protocol.
Move existing tests to Lib/test/test_capi/test_abstract.py and
Modules/_testcapi/abstract.c.
Add also tests for PyDict C API..
(cherry picked from commit 16c9415fba)
* Strings with length from 2**31-1 to 2**32-2 always caused MemoryError,
it doesn't matter how much memory is available.
* Strings with length exactly 2**32-1 caused OSError.
* Strings longer than 2**32-1 characters were truncated due to integer overflow bug.
* Strings containing the null character were truncated at the first null character.
Now strings longer than 2**31-1 characters caused OverflowError and the null character is allowed..
(cherry picked from commit 04cc01453d)
Restore the global Input Stream pointer after trying to match a sub-pattern.
.
(cherry picked from commit abd9cc52d9)
Co-authored-by: SKO <41810398+uyw4687@users.noreply.github.com>
gh-91054: make code watcher tests resilient to other watchers (GH-107821)
(cherry picked from commit 2ec16fed14)
Co-authored-by: Carl Meyer <carl@oddbird.net>
* Check for linux/limits.h before including it
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
(cherry picked from commit 11c055f5ff)
gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433)
Fix potential unaligned memory access on C APIs involving returned sequences
of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These
were revealed using a ``-fsaniziter=alignment`` build on ARM macOS.
(cherry picked from commit f01e4cedba)
Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
The _xxsubinterpreters module was meant to only use public API. Some internal C-API usage snuck in over the last few years (e.g. gh-28969). This fixes that.
(cherry picked from commit e6373c0d8b)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832)
(cherry picked from commit ebf2c56b33)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-106687: _ssl: use uint64_t for SSL options (GH-106700)
SSL_CTX_get_options() uses uint64_t for options:
https://www.openssl.org/docs/man3.1/man3/SSL_CTX_get_options.html
Fix this compiler warning on Windows with MSC:
conversion from 'uint64_t' to 'long', possible loss of data
(cherry picked from commit ad95c7253a)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252)
* Add a special case for s[-m:] == p in _PyBytes_Find
* Add tests for _PyBytes_Find
* Make sure that start <= end in mmap.find
(cherry picked from commit ab86426a34)
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
gh-105227: Add PyType_GetDict() (GH-105747)
This compensates for static builtin types having `tp_dict` set to `NULL`.
(cherry picked from commit a840806d33)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Some items remained uninitialized if _sre.template() was called with invalid
indices. Then attempt to clear them in the destructor led to dereferencing
of uninitialized pointer.
(cherry picked from commit 2ef1dc37f0)
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
gh-91053: make func watcher tests resilient to other func watchers (GH-106286)
(cherry picked from commit 58906213cc)
Co-authored-by: Carl Meyer <carl@oddbird.net>
[3.12] gh-106033: Get rid of new occurrences of PyDict_GetItem and PyObject_HasAttr (GH-106034)
These functions are broken by design because they discard any exceptions raised
inside, including MemoryError and KeyboardInterrupt. They should not be
used in new code.
(cherry picked from commit 1d33d53780)
