mirror of
https://github.com/python/cpython.git
synced 2026-02-16 04:11:55 +00:00
2e279e85fe
`urllib.unquote_to_bytes` and `urllib.unquote` could both potentially generate `O(len(string))` intermediate `bytes` or `str` objects while computing the unquoted final result depending on the input provided. As Python objects are relatively large, this could consume a lot of ram. This switches the implementation to using an expanding `bytearray` and a generator internally instead of precomputed `split()` style operations. Microbenchmarks with some antagonistic inputs like `mess = "\u0141%%%20a%fe"*1000` show this is 10-20% slower for unquote and unquote_to_bytes and no different for typical inputs that are short or lack much unicode or % escaping. But the functions are already quite fast anyways so not a big deal. The slowdown scales consistently linear with input size as expected. Memory usage observed manually using `/usr/bin/time -v` on `python -m timeit` runs of larger inputs. Unittesting memory consumption is difficult and does not seem worthwhile. Observed memory usage is ~1/2 for `unquote()` and <1/3 for `unquote_to_bytes()` using `python -m timeit -s 'from urllib.parse import unquote, unquote_to_bytes; v="\u0141%01\u0161%20"*500_000' 'unquote_to_bytes(v)'` as a test. |
||
|---|---|---|
| .. | ||
| NEWS.d | ||
| rhel7 | ||
| ACKS | ||
| coverity_model.c | ||
| gdbinit | ||
| HISTORY | ||
| indent.pro | ||
| Porting | ||
| python-config.in | ||
| python-config.sh.in | ||
| python-embed.pc.in | ||
| python.man | ||
| python.pc.in | ||
| README | ||
| README.AIX | ||
| README.coverity | ||
| README.valgrind | ||
| requirements-test.txt | ||
| SpecialBuilds.txt | ||
| stable_abi.toml | ||
| svnmap.txt | ||
| valgrind-python.supp | ||
| vgrindefs | ||
Python Misc subdirectory
========================
This directory contains files that wouldn't fit in elsewhere. Some
documents are only of historic importance.
Files found here
----------------
ACKS Acknowledgements
gdbinit Handy stuff to put in your .gdbinit file, if you use gdb
HISTORY News from previous releases -- oldest last
indent.pro GNU indent profile approximating my C style
NEWS News for this release (for some meaning of "this")
Porting Mini-FAQ on porting to new platforms
python-config.in Python script template for python-config
python.man UNIX man page for the python interpreter
python.pc.in Package configuration info template for pkg-config
README The file you're reading now
README.AIX Information about using Python on AIX
README.coverity Information about running Coverity's Prevent on Python
README.valgrind Information for Valgrind users, see valgrind-python.supp
SpecialBuilds.txt Describes extra symbols you can set for debug builds
svnmap.txt Map of old SVN revs and branches to hg changeset ids,
help history-digging
valgrind-python.supp Valgrind suppression file, see README.valgrind
vgrindefs Python configuration for vgrind (a generic pretty printer)