Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-12-08 06:10:17 +00:00
Code Issues Projects Releases Packages Wiki Activity

gh-134938: Add set_pledged_input_size() to ZstdCompressor (GH-135010)

Browse source
This commit is contained in:
Emma Smith 2025-06-05 04:31:49 -07:00 • committed by GitHub
parent 3d396ab759
commit 4b44b3409a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 285 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

35
Doc/library/compression.zstd.rst
View file

@ -247,6 +247,27 @@ Compressing and decompressing data in memory
The *mode* argument is a :class:`ZstdCompressor` attribute, either
:attr:`~.FLUSH_BLOCK`, or :attr:`~.FLUSH_FRAME`.
.. method:: set_pledged_input_size(size)
Specify the amount of uncompressed data *size* that will be provided for
the next frame. *size* will be written into the frame header of the next
frame unless :attr:`CompressionParameter.content_size_flag` is ``False``
or ``0``. A size of ``0`` means that the frame is empty. If *size* is
``None``, the frame header will omit the frame size. Frames that include
the uncompressed data size require less memory to decompress, especially
at higher compression levels.
If :attr:`last_mode` is not :attr:`FLUSH_FRAME`, a
:exc:`ValueError` is raised as the compressor is not at the start of
a frame. If the pledged size does not match the actual size of data
provided to :meth:`.compress`, future calls to :meth:`!compress` or
:meth:`flush` may raise :exc:`ZstdError` and the last chunk of data may
be lost.
After :meth:`flush` or :meth:`.compress` are called with mode
:attr:`FLUSH_FRAME`, the next frame will not include the frame size into
the header unless :meth:`!set_pledged_input_size` is called again.
.. attribute:: CONTINUE
Collect more data for compression, which may or may not generate output
@ -266,6 +287,13 @@ Compressing and decompressing data in memory
:meth:`~.compress` will be written into a new frame and
*cannot* reference past data.
.. attribute:: last_mode
The last mode passed to either :meth:`~.compress` or :meth:`~.flush`.
The value can be one of :attr:`~.CONTINUE`, :attr:`~.FLUSH_BLOCK`, or
:attr:`~.FLUSH_FRAME`. The initial value is :attr:`~.FLUSH_FRAME`,
signifying that the compressor is at the start of a new frame.
.. class:: ZstdDecompressor(zstd_dict=None, options=None)
@ -620,12 +648,17 @@ Advanced parameter control
Write the size of the data to be compressed into the Zstandard frame
header when known prior to compressing.
This flag only takes effect under the following two scenarios:
This flag only takes effect under the following scenarios:
* Calling :func:`compress` for one-shot compression
* Providing all of the data to be compressed in the frame in a single
:meth:`ZstdCompressor.compress` call, with the
:attr:`ZstdCompressor.FLUSH_FRAME` mode.
* Calling :meth:`ZstdCompressor.set_pledged_input_size` with the exact
amount of data that will be provided to the compressor prior to any
calls to :meth:`ZstdCompressor.compress` for the current frame.
:meth:`!ZstdCompressor.set_pledged_input_size` must be called for each
new frame.
All other compression calls may not write the size information into the
frame header.