Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2026-01-25 08:42:15 +00:00
Code Issues Projects Releases Packages Wiki Activity
cpython/Modules
History
Emma Smith f262297d52
gh-139877: Use PyBytesWriter in pycore_blocks_output_buffer.h (#139976) 
Previously, the _BlocksOutputBuffer code creates a list of bytes objects to handle the output data from compression libraries. This ends up being slow due to the output buffer code needing to copy each bytes element of the list into the final bytes object buffer at the end of compression.

The new PyBytesWriter API introduced in PEP 782 is an ergonomic and fast method of writing data into a buffer that will later turn into a bytes object. Benchmarks show that using the PyBytesWriter API is 10-30% faster for decompression across a variety of settings. The performance gains are greatest when the decompressor is very performant, such as for Zstandard (and likely zlib-ng). Otherwise the decompressor can bottleneck decompression and the gains are more modest, but still sizable (e.g. 10% faster for zlib)!

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-10-14 10:03:55 -07:00
..
_ctypes gh-138008: Fix segfaults in _ctypes due to invalid argtypes (GH-138285) 2025-09-10 14:01:19 +02:00
_decimal gh-102431: Clarify constraints on operands of Decimal logical operations (GH-102836) 2025-10-14 11:02:02 +00:00
_hacl gh-135532: update Modules/_hacl/python_hacl_namespaces.h (#135741) 2025-06-20 11:32:38 +02:00
_io gh-129813, PEP 782: Use PyBytesWriter in bufferedio.c (#139121) 2025-09-18 17:00:10 +02:00
_multiprocessing gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
_sqlite gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
_sre gh-138264: Fix gcc 14 compiler warnings (GH-138265) 2025-09-03 11:26:56 +03:00
_ssl gh-137583: Only lock the SSL context, not the SSL socket (GH-137588) 2025-08-10 14:47:11 +00:00
_testcapi gh-129813: Fix PyBytesWriter tests (#139892) 2025-10-10 13:01:06 +02:00
_testinternalcapi gh-135641: Fix flaky test_capi.test_lock_two_threads test case (gh-135642) 2025-06-18 14:24:05 -04:00
_testlimitedcapi gh-108512: Add and use new replacements for PySys_GetObject() (GH-111035) 2025-05-28 20:11:09 +03:00
_xxtestfuzz gh-111178: remove redundant casts for functions with correct signatures (#131673) 2025-04-01 17:18:11 +02:00
_zstd gh-139877: Use PyBytesWriter in pycore_blocks_output_buffer.h (#139976) 2025-10-14 10:03:55 -07:00
cjkcodecs gh-101828: Fix jisx0213 codecs removing null characters (gh-139340) 2025-10-14 22:55:00 +09:00
clinic gh-139482: Add posix._clearenv() function (#139965) 2025-10-11 22:58:43 +02:00
expat gh-139312: Update bundled libexpat to 2.7.3 (GH-139319) 2025-09-26 19:44:36 -07:00
_abc.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
_asynciomodule.c Doc: More duplicate word fixes (GH-136299) 2025-07-11 21:18:47 +03:00
_bisectmodule.c GH-135763: AC: Use `Py_ssize_t(allow_negative=False)` (#138394) 2025-09-02 21:29:05 +01:00
_bz2module.c gh-139877: Use PyBytesWriter in pycore_blocks_output_buffer.h (#139976) 2025-10-14 10:03:55 -07:00
_codecsmodule.c gh-139156: Optimize the UTF-7 encoder (#139253) 2025-09-24 17:57:29 +02:00
_collectionsmodule.c gh-135607: remove null checking of weakref list in dealloc of extension modules and objects (#135614) 2025-06-30 11:14:31 +00:00
_csv.c gh-137986: Fix and improve the csv functions docstrings (GH-137987) 2025-08-24 11:13:31 +03:00
_curses_panel.c gh-116946: add Py_TPFLAGS_IMMUTABLETYPE to several internal types (#138582) 2025-09-11 09:56:20 +02:00
_cursesmodule.c gh-129813, PEP 782: Use PyBytesWriter in _curses (#138920) 2025-09-15 16:24:34 +02:00
_datetimemodule.c gh-139774: use relaxed atomics for datetime hashes (#139775) 2025-10-08 19:49:54 +05:30
_dbmmodule.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
_elementtree.c gh-139210: Fix use-after-free in xml.etree.ElementTree.iterparse() (GH-139211) 2025-09-30 20:50:50 +03:00
_functoolsmodule.c gh-138342: Use a common utility for visiting an object's type (GH-138343) 2025-09-01 16:20:33 +00:00
_gdbmmodule.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
_hashopenssl.c gh-129813, PEP 782: Use PyBytesWriter in _hashopenssl (#138922) 2025-09-15 18:41:43 +02:00
_heapqmodule.c Fix comments for heapq.siftup_max (#135359) 2025-07-03 15:21:41 +05:30
_interpchannelsmodule.c gh-123828: Fix data race in _interpchannels._waiting_release (GH-124107) 2025-10-01 11:34:47 +00:00
_interpqueuesmodule.c GH-137630: Convert `_interpqueues` to use Argument Clinic (#137685) 2025-09-17 17:35:00 +01:00
_interpreters_common.h gh-132775: Use _PyObject_GetXIData (With Fallback) (gh-134440) 2025-05-22 06:50:06 -06:00
_interpretersmodule.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
_json.c gh-116738: Make _json module thread-safe in the free-threading (#119438) 2025-08-31 09:42:45 +05:30
_localemodule.c gh-130567: Fix possible crash in locale.strxfrm() (GH-138940) 2025-09-23 15:11:50 +00:00
_lsprof.c gh-116738: make cProfile module thread-safe (#138229) 2025-09-06 13:19:14 +05:30
_lzmamodule.c gh-139877: Use PyBytesWriter in pycore_blocks_output_buffer.h (#139976) 2025-10-14 10:03:55 -07:00
_math.h gh-122681: remove m_atan2()/c_atan2() helpers (#122715) 2024-08-17 13:48:16 +05:30
_opcode.c gh-135755: Move SPECIAL_ constants to a private header (GH-135922) 2025-06-25 13:03:05 +02:00
_operator.c gh-131238: Add explicit includes to pycore headers (#131257) 2025-03-17 12:32:43 +01:00
_pickle.c gh-129813, PEP 782: Use PyBytesWriter in pickle and struct (#138833) 2025-09-13 18:26:49 +02:00
_posixsubprocess.c fix comment reference from man 7 signal to man 7 signal-safety (#138554) 2025-09-06 18:40:15 +00:00
_queuemodule.c gh-135607: remove null checking of weakref list in dealloc of extension modules and objects (#135614) 2025-06-30 11:14:31 +00:00
_randommodule.c gh-116946: add Py_TPFLAGS_IMMUTABLETYPE to _random.Random (#138341) 2025-09-03 09:55:53 +02:00
_remote_debugging_module.c gh-139275: Fix compilation of Modules/_remote_debugging_module.c when the system doesn't have process_vm_readv (#139307) 2025-09-25 00:16:44 +01:00
_scproxy.c gh-116322: Add Py_mod_gil module slot (#116882) 2024-05-03 11:30:55 -04:00
_ssl.c gh-139929: fix incorrect OpenSSL version-based guard in _ssl.c (GH-139945) 2025-10-11 10:34:08 -07:00
_ssl.h GH-103092: isolate _ssl (#104725) 2023-05-22 06:14:48 +05:30
_ssl_data_35.h gh-139573: Update OpenSSL in CI (GH-139577) 2025-10-04 19:43:17 -05:00
_ssl_data_111.h gh-131423: Update OpenSSL data to 3.4.1 on Linux (#131618) 2025-04-25 10:26:58 +02:00
_ssl_data_300.h gh-127330: Update for OpenSSL 3.4 & document+improve the update process (GH-127331) 2024-11-28 13:29:27 +01:00
_ssl_data_340.h gh-131423: Update OpenSSL data to 3.4.1 on Linux (#131618) 2025-04-25 10:26:58 +02:00
_stat.c gh-108765: fix comment about macro definitions in _stat.c post GH-108854 (#136027) 2025-06-27 15:15:11 +00:00
_statisticsmodule.c gh-116322: Add Py_mod_gil module slot (#116882) 2024-05-03 11:30:55 -04:00
_struct.c gh-129813, PEP 782: Use PyBytesWriter in pickle and struct (#138833) 2025-09-13 18:26:49 +02:00
_suggestions.c gh-131936: Strengthen check in _suggestions._generate_suggestions (#131945) 2025-03-31 19:30:29 +02:00
_sysconfig.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
_testbuffer.c gh-136300: Modify C tests to conform to PEP-737 (GH-136301) 2025-07-11 15:18:35 +02:00
_testcapi_feature_macros.inc gh-91325: Skip Stable ABI checks with Py_TRACE_REFS special build (GH-92046) 2024-01-29 16:45:31 +01:00
_testcapimodule.c gh-139276: Remove generator type check in _testcapimodule.c:raise_SIGINT_then_send_None (#139252) 2025-09-26 11:52:10 -07:00
_testclinic.c gh-111489: Remove _PyTuple_FromArray() alias (#139973) 2025-10-11 22:58:14 +02:00
_testclinic_limited.c gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_SetGIL (GH-118645) 2024-05-06 18:59:36 +02:00
_testimportmultiple.c gh-116322: Add Py_mod_gil module slot (#116882) 2024-05-03 11:30:55 -04:00
_testinternalcapi.c gh-139525: Don't specialize functions which have a modified vectorcall (#139524) 2025-10-03 09:58:32 -07:00
_testlimitedcapi.c gh-111495: Add PyFile tests (#129449) 2025-01-30 18:05:32 +01:00
_testmultiphase.c gh-111178: fix UBSan failures for Modules/_testmultiphase.c (#131615) 2025-03-24 10:59:30 +01:00
_testsinglephase.c gh-123880: Allow recursive import of single-phase-init modules (GH-123950) 2024-09-20 10:27:34 +02:00
_threadmodule.c gh-116946: partial revert gh-139073 for thread handle type (#139474) 2025-10-01 17:49:31 +00:00
_tkinter.c gh-140001: Remove obsolete TCL_WIN_SOCKET macro (from Tcl 7.x) from _tkinter.c (GH-139998) 2025-10-13 13:29:42 +03:00
_tracemalloc.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
_typesmodule.c gh-136492: Add FrameLocalsProxyType to types (GH-136546) 2025-07-20 20:49:00 +02:00
_typingmodule.c gh-105499: Merge typing.Union and types.UnionType (#105511) 2025-03-04 11:44:19 -08:00
_uuidmodule.c gh-132710: only use stable _uuid.generate_time_safe() to deduce MAC address (#132901) 2025-05-26 10:56:31 +02:00
_weakref.c gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (#122139) 2024-07-23 17:06:03 +00:00
_winapi.c gh-129813, PEP 782: Use PyBytesWriter in _winapi.PeekNamedPipe() (#138930) 2025-09-15 21:25:47 +01:00
_zoneinfo.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
addrinfo.h gh-114917: add support for AI_NUMERICSERV in getaddrinfo emulation (#114918) 2025-03-18 11:26:51 +01:00
arraymodule.c GH-135763: AC: Use `Py_ssize_t(allow_negative=False)` (#138394) 2025-09-02 21:29:05 +01:00
atexitmodule.c gh-111178: remove redundant casts for functions with correct signatures (#131673) 2025-04-01 17:18:11 +02:00
binascii.c gh-129813, PEP 782: Use PyBytesWriter in binascii (#138825) 2025-09-13 18:25:16 +02:00
blake2module.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
cmathmodule.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
config.c.in gh-128384: Use a context variable for warnings.catch_warnings (gh-130010) 2025-04-09 16:18:54 -07:00
errnomodule.c gh-126585: Add EHWPOISON error code (#126586) 2024-12-03 12:45:50 +00:00
faulthandler.c gh-137725: Convert faulthandler to Argument Clinic (#137726) 2025-08-16 15:16:04 +02:00
fcntlmodule.c gh-129813, PEP 782: Use PyBytesWriter in fcntl (#138921) 2025-09-15 16:21:43 +02:00
gc_weakref.txt GH-135552: Make the GC clear weakrefs later (GH-136189) 2025-08-07 16:32:17 -07:00
gcmodule.c gh-111489: Remove _PyTuple_FromArray() alias (#139973) 2025-10-11 22:58:14 +02:00
getaddrinfo.c gh-114917: Fix typo in getaddrinfo emulation (#131413) 2025-03-18 14:31:13 +01:00
getbuildinfo.c gh-100388: Change undefined __DATE__ to the Unix epoch (#100389) 2025-03-03 14:59:46 +01:00
getnameinfo.c Replace K&R function declarations with ANSI declarations (#114818) 2025-03-11 13:06:36 +01:00
getpath.c GH-131296: Avoid invalid signed char comparison in getpath.c assertion (GH-131594) 2025-04-22 14:06:30 +01:00
getpath.py gh-120037: Disable user site packages when a ._pth file is used (GH-137428) 2025-08-19 21:28:28 +01:00
getpath_noop.c bpo-45582: Port getpath[p].c to Python (GH-29041) 2021-12-03 00:08:42 +00:00
grpmodule.c gh-116738: Make grp module thread-safe (#135434) 2025-07-14 11:18:41 -07:00
hashlib.h gh-131876: Revert "gh-131876: extract _hashlib helpers into a separate directory (#136995) (#137307) 2025-08-01 10:45:40 -07:00
hmacmodule.c gh-116946: remove unnecessary gc from immutable types (#139073) 2025-10-01 13:15:58 +05:30
itertoolsmodule.c gh-111489: Remove _PyTuple_FromArray() alias (#139973) 2025-10-11 22:58:14 +02:00
ld_so_aix.in Issue #10656: Fix out-of-tree building on AIX 2016-11-20 07:56:37 +00:00
main.c gh-108512: Add and use new replacements for PySys_GetObject() (GH-111035) 2025-05-28 20:11:09 +03:00
makesetup gh-116943: Ensure makesetup does not choke on missing EOL in Setup.* files (#139100) 2025-09-18 09:25:01 +00:00
makexp_aix bpo-42087: Remove support for AIX 5.3 and below (GH-22830) 2020-11-16 16:16:10 +01:00
mathmodule.c Revert gh-139075. The sumprod fix needs more work. (gh-139092) 2025-09-18 02:05:27 -05:00
md5module.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
mmapmodule.c gh-116738: make mmap module thread-safe (#139237) 2025-10-09 12:00:47 +05:30
overlapped.c GH-131296: Remove unused variable from overlapped.c (GH-131590) 2025-04-16 22:44:57 +01:00
posixmodule.c gh-139482: Add posix._clearenv() function (#139965) 2025-10-11 22:58:43 +02:00
posixmodule.h gh-85283: Convert grp extension to the limited C API (#116611) 2024-03-12 00:46:53 +00:00
pwdmodule.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
pyexpat.c gh-139400: Make sure that parent parsers outlive their subparsers in pyexpat (#139403) 2025-10-05 17:37:42 +02:00
readline.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
README Issue #18093: Factor out the programs that embed the runtime 2014-07-25 21:52:14 +10:00
resource.c gh-137512: Add new constants in the resource module (GH-137513) 2025-08-19 08:52:45 +02:00
rotatingtree.c gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (#121260) 2024-07-03 13:05:05 +05:30
rotatingtree.h bpo-32150: Expand tabs to spaces in C files. (#4583) 2017-11-28 17:56:10 +02:00
selectmodule.c gh-67795: Accept any real numbers as timestamp and timeout (GH-139224) 2025-09-23 21:31:42 +03:00
Setup GH-132983: Build `_zstd` on Windows (#133366) 2025-05-06 00:58:47 +01:00
Setup.bootstrap.in gh-136421: Load _datetime static types during interpreter initialization (GH-136583) 2025-07-21 13:47:26 -04:00
Setup.stdlib.in gh-137210: Add a struct, slot & function for checking an extension's ABI (GH-137212) 2025-09-05 16:23:18 +02:00
sha1module.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
sha2module.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
sha3module.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
signalmodule.c gh-67795: Accept any real numbers as timestamp and timeout (GH-139224) 2025-09-23 21:31:42 +03:00
socketmodule.c gh-139748: fix leaks in AC error paths when using unicode FS-based converters (#139765) 2025-10-08 20:52:44 +05:30
socketmodule.h gh-86819: Add ISO-TP CAN socket constants (#23794) 2025-09-17 17:18:04 +00:00
symtablemodule.c gh-139748: fix leaks in AC error paths when using unicode FS-based converters (#139765) 2025-10-08 20:52:44 +05:30
syslogmodule.c gh-116738: Make syslog module thread-safe (#136760) 2025-07-21 09:24:42 -07:00
termios.c GH-137623: Use an AC decorator for docstring line length enforcement (#137690) 2025-08-18 18:29:00 +01:00
timemodule.c gh-134771: Fix time_clockid_converter() on Cygwin (#134772) 2025-05-29 19:35:53 +02:00
tkappinit.c gh-103538: Remove unused TK_AQUA code (GH-103539) 2023-05-10 18:53:13 +00:00
tkinter.h gh-103532: Remove TKINTER_PROTECT_LOADTK code (GH-103535) 2023-04-14 09:04:16 -05:00
unicodedata.c gh-138342: Move _PyObject_VisitType() to the internal C API (#139734) 2025-10-08 12:10:58 +02:00
unicodedata_db.h closes gh-138706: update Unicode to 17.0.0 (#138719) 2025-09-11 09:58:39 -07:00
unicodename_db.h closes gh-138706: update Unicode to 17.0.0 (#138719) 2025-09-11 09:58:39 -07:00
winreparse.h bpo-31512: Add non-elevated symlink support for Windows (GH-3652) 2019-04-09 11:19:46 -07:00
xxlimited.c gh-134160: Use PyModuleDef.m_free in the example module xxlimited (GH-135174) 2025-06-06 14:08:58 +00:00
xxlimited_35.c gh-111178: Fix function signatures for multiple tests (#131496) 2025-03-20 12:27:03 +01:00
xxmodule.c gh-122040: reword Modules/xxmodule.c module-level comment (#132201) 2025-04-07 08:03:03 +00:00
xxsubtype.c gh-111178: fix UBSan failures in Modules/xx*.c (GH-129797) 2025-02-25 13:02:32 +01:00
zlibmodule.c gh-139877: Use PyBytesWriter in pycore_blocks_output_buffer.h (#139976) 2025-10-14 10:03:55 -07:00

README 

Source files for standard library extension modules,
and former extension modules that are now builtin modules.