Previously, the _BlocksOutputBuffer code creates a list of bytes objects to handle the output data from compression libraries. This ends up being slow due to the output buffer code needing to copy each bytes element of the list into the final bytes object buffer at the end of compression.
The new PyBytesWriter API introduced in PEP 782 is an ergonomic and fast method of writing data into a buffer that will later turn into a bytes object. Benchmarks show that using the PyBytesWriter API is 10-30% faster for decompression across a variety of settings. The performance gains are greatest when the decompressor is very performant, such as for Zstandard (and likely zlib-ng). Otherwise the decompressor can bottleneck decompression and the gains are more modest, but still sizable (e.g. 10% faster for zlib)!
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Fix incorrect sharing of current task with the forked child process by clearing thread state's current task and current loop in `PyOS_AfterFork_Child`.
* Modules/pyexpat.c: Disallow collection of in-use parent parsers.
Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.
Fix a compiler warning `-Wunused-function` after f04bea44c3.
The `set_invalid_arg` function in `Modules/pyexpat.c` may be unused if the underlying Expat
version is less than 2.4.0.
Passing a negative or zero size to `cursor.fetchmany()` made it fetch all rows
instead of none.
While this could be considered a security vulnerability, it was decided to treat
this issue as a regular bug as passing a non-sanitized *size* value in the first
place is not recommended.
Expose the XML Expat 2.7.2 APIs to tune protections against
"billion laughs" [1] attacks.
The exposed APIs are available on Expat parsers, that is,
parsers created by `xml.parsers.expat.ParserCreate()`, as:
- `parser.SetBillionLaughsAttackProtectionActivationThreshold(threshold)`, and
- `parser.SetBillionLaughsAttackProtectionMaximumAmplification(max_factor)`.
This completes the work in f04bea44c3,
and improves the existing related documentation.
[1]: https://en.wikipedia.org/wiki/Billion_laughs_attack
* Remove generator type check in raise_SIGINT_then_send_None
In the Cinder JIT we use a different type for generators, which breaks
the test which uses this function.
In general I believe the intent with generators is they have the right
structure rather than type, so a failure to find the 'send()' method is arguably
more correct if the wrong object is used.
* Also stop using PyGenObject type
Expose the XML Expat 2.7.2 mitigation APIs to disallow use of
disproportional amounts of dynamic memory from within an Expat
parser (see CVE-2025-59375 for instance).
The exposed APIs are available on Expat parsers, that is,
parsers created by `xml.parsers.expat.ParserCreate()`, as:
- `parser.SetAllocTrackerActivationThreshold(threshold)`, and
- `parser.SetAllocTrackerMaximumAmplification(max_factor)`.
Functions that take timestamp or timeout arguments now accept any
real numbers (such as Decimal and Fraction), not only integers or floats,
although this does not improve precision.
On some macOS versions there was an off-by-one error in wcsxfrm() which
caused writing past the end of the array if its size was not calculated
by running wcsxfrm() first.
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
During finalization, we need to mark all non-daemon threads as daemon to quickly shut down threads when sending CTRL^C to the process. This was a minor regression from GH-136004.
