mirror of
https://github.com/python/cpython.git
synced 2025-12-08 06:10:17 +00:00
59f247e43b
Loading a small data which does not even involve arbitrary code execution could consume arbitrary large amount of memory. There were three issues: * PUT and LONG_BINPUT with large argument (the C implementation only). Since the memo is implemented in C as a continuous dynamic array, a single opcode can cause its resizing to arbitrary size. Now the sparsity of memo indices is limited. * BINBYTES, BINBYTES8 and BYTEARRAY8 with large argument. They allocated the bytes or bytearray object of the specified size before reading into it. Now they read very large data by chunks. * BINSTRING, BINUNICODE, LONG4, BINUNICODE8 and FRAME with large argument. They read the whole data by calling the read() method of the underlying file object, which usually allocates the bytes object of the specified size before reading into it. Now they read very large data by chunks. Also add comprehensive benchmark suite to measure performance and memory impact of chunked reading optimization in PR #119204. Features: - Normal mode: benchmarks legitimate pickles (time/memory metrics) - Antagonistic mode: tests malicious pickles (DoS protection) - Baseline comparison: side-by-side comparison of two Python builds - Support for truncated data and sparse memo attack vectors Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: Gregory P. Smith <greg@krypto.org> |
||
|---|---|---|
| .. | ||
| build | ||
| buildbot | ||
| c-analyzer | ||
| cases_generator | ||
| check-c-api-docs | ||
| clinic | ||
| freeze | ||
| ftscalingbench | ||
| gdb | ||
| i18n | ||
| importbench | ||
| inspection | ||
| jit | ||
| lockbench | ||
| msi | ||
| nuget | ||
| patchcheck | ||
| peg_generator | ||
| picklebench | ||
| scripts | ||
| ssl | ||
| tsan | ||
| unicode | ||
| unittestgui | ||
| wasm | ||
| README | ||
| requirements-dev.txt | ||
| requirements-hypothesis.txt | ||
This directory contains a number of Python programs that are useful
while building or extending Python.
build Automatically generated directory by the build system
contain build artifacts and intermediate files.
buildbot Batchfiles for running on Windows buildbot workers.
c-analyzer Tools to check no new global variables have been added.
cases_generator Tooling to generate interpreters.
clinic A preprocessor for CPython C files in order to automate
the boilerplate involved with writing argument parsing
code for "builtins".
freeze Create a stand-alone executable from a Python program.
ftscalingbench Benchmarks for free-threading and finding bottlenecks.
gdb Python code to be run inside gdb, to make it easier to
debug Python itself (by David Malcolm).
i18n Tools for internationalization. pygettext.py
parses Python source code and generates .pot files,
and msgfmt.py generates a binary message catalog
from a catalog in text format.
importbench A set of micro-benchmarks for various import scenarios.
inspection Tooling for PEP-678 "Safe external debugger interface for CPython".
jit Tooling for building the JIT.
lockbench Benchmarks for PyMutex and critical sections.
msi Support for packaging Python as an MSI package on Windows.
nuget Files for the NuGet package manager for .NET.
patchcheck Tools for checking and applying patches to the Python source code
and verifying the integrity of patch files.
peg_generator PEG-based parser generator (pegen) used for new parser.
scripts A number of useful single-file programs, e.g. run_tests.py
which runs the Python test suite.
ssl Scripts to generate ssl_data.h from OpenSSL sources, and run
tests against multiple installations of OpenSSL and LibreSSL.
tsan Utilities for building CPython with thread-sanitizer.
unicode Tools for generating unicodedata and codecs from unicode.org
and other mapping files (by Fredrik Lundh, Marc-Andre Lemburg
and Martin von Loewis).
unittestgui A Tkinter based GUI test runner for unittest, with test
discovery.
wasm Config and helpers to facilitate cross compilation of CPython
to WebAssembly (WASM).
Note: The pynche color editor has moved to https://gitlab.com/warsaw/pynche