Starting in Python 3.12, we prevented calling fork() and starting new threads
during interpreter finalization (shutdown). This has led to a number of
regressions and flaky tests. We should not prevent starting new threads
(or `fork()`) until all non-daemon threads exit and finalization starts in
earnest.
This changes the checks to use `_PyInterpreterState_GetFinalizing(interp)`,
which is set immediately before terminating non-daemon threads.
(cherry picked from commit 60e105c1c1)
gh-116447: Fix possible UB in `arraymodule` and `getargs` (GH-116459)
(cherry picked from commit fdb2d90a27)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`
Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .
- Please treat as a security fix related to CVE-2023-52425.
(cherry picked from commit 6a95676bb5)
(cherry picked from commit 73807eb634)
(cherry picked from commit eda2963378)
---------
Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573)
* gh-114572: Fix locking in cert_store_stats and get_ca_certs
cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with
X509_STORE_get0_objects, but reading the result requires a lock. See
https://github.com/openssl/openssl/pull/23224 for details.
Instead, use X509_STORE_get1_objects, newly added in that PR.
X509_STORE_get1_objects does not exist in current OpenSSLs, but we can
polyfill it with X509_STORE_lock and X509_STORE_unlock.
* Work around const-correctness problem
* Add missing X509_STORE_get1_objects failure check
* Add blurb
(cherry picked from commit bce693111b)
Co-authored-by: David Benjamin <davidben@google.com>
shm_open() and shm_unlink() now check for embedded null characters in
the name and raise an error instead of silently truncating it.
(cherry picked from commit 79811ededd)
csv.writer() now quotes empty fields if delimiter is a space and
skipinitialspace is true and raises exception if quoting is not possible.
(cherry picked from commit 937d282150)
lseek() always returns 0 for character pseudo-devices like
`/dev/urandom` (for other non-regular files, e.g. `/dev/stdin`, it
always returns -1, to which CPython reacts by raising appropriate
exceptions). They are thus technically seekable despite not having seek
semantics.
When calling read() on e.g. an instance of `io.BufferedReader` that
wraps such a file, `BufferedReader` reads ahead, filling its buffer,
creating a discrepancy between the number of bytes read and the internal
`tell()` always returning 0, which previously resulted in e.g.
`BufferedReader.tell()` or `BufferedReader.seek()` being able to return
positions < 0 even though these are supposed to be always >= 0.
Invariably keep the return value non-negative by returning
max(former_return_value, 0) instead, and add some corresponding tests.
(cherry picked from commit 26800cf25a)
Co-authored-by: 6t8k <58048945+6t8k@users.noreply.github.com>
Immediate merits:
* eliminate complex workarounds for 'z' format support
(NOTE: mpdecimal recently added 'z' support, so this becomes
efficient in the long term.)
* fix 'z' format memory leak
* fix 'z' format applied to 'F'
* fix missing '#' format support
Suggested and prototyped by Stefan Krah.
Fixes gh-114563, gh-91060
(cherry picked from commit 72340d15cd)
Co-authored-by: John Belmonte <john@neggie.net>
Co-authored-by: Stefan Krah <skrah@bytereef.org>
On Alpine Linux it could leave some field non-initialized.
(cherry picked from commit d22c066b80)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-104522: Fix OSError raised when run a subprocess (GH-114195)
Only set filename to cwd if it was caused by failed chdir(cwd).
_fork_exec() now returns "noexec:chdir" for failed chdir(cwd).
(cherry picked from commit e2c097ebde)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Robert O'Shea <PurityLake@users.noreply.github.com>
This avoids impact on later parts of the application which may be able to do things they otherwise shouldn't.
(cherry picked from commit de4ced54eb)
Co-authored-by: Steve Dower <steve.dower@python.org>
(cherry picked from commit c31be58da8)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
io.TextIOWrapper was dropping the internal decoding buffer
during read() and write() calls.
(cherry picked from commit 73c9326563)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
The length field of StgDictObject for Structure class contains now
the total number of items in ffi_type_pointer.elements (excluding
the trailing null).
The old behavior of using the number of elements in the parent class can
cause the array to be truncated when it is copied, especially when there
are multiple layers of subclassing.
(cherry picked from commit 5f3cc90a12)
Co-authored-by: Jeffrey Kintscher <49998481+websurfer5@users.noreply.github.com>
Fix the same issue of PR #112604 on PPC64LE platform
Refactor tests to make easier to add more platfroms if needed.
(cherry picked from commit 6644ca45cd)
Change-Id: I31730a3ebe558570ce1d7a3b26db8392f18d1770
* Fix a crash when pass UINT_MAX.
* Fix an integer overflow on 64-bit non-Windows platforms.
(cherry picked from commit 0daf555c6f)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
First fix resolve situation when pyexpat module (which contains expat_CAPI
capsule) deallocates before _elementtree, so we need to hold a strong
reference to pyexpat module to.
Second fix resolve situation when module state is deallocated before
deallocation of XMLParser instances, which uses module state to clear
some stuff.
(cherry picked from commit 894f0e573d)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
