Hold strong references to borrowed items unconditionally (not only in
free-threading builds) in _encoder_iterate_mapping_lock_held and
_encoder_iterate_fast_seq_lock_held. User callbacks invoked during
encoding can mutate or clear the underlying container, invalidating
borrowed references.
The dict iteration path was already fixed by gh-145244.
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
In encoder_encode_key_value(), key is a borrowed reference from
PyDict_Next(). If the default callback mutates or clears the dict,
key becomes a dangling pointer. The error path then calls
_PyErr_FormatNote("%R", key) on freed memory.
Fix by holding strong references to key and value unconditionally
during encoding, not just in the free-threading build.
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Fix json serialization: no longer call str(obj) on str subclasses.
Replace PyUnicodeWriter_WriteStr() with PyUnicodeWriter_WriteASCII()
and private _PyUnicodeWriter_WriteStr().
The internal encoder object returned by undocumented function
json.encoder.c_make_encoder() (aka _json.make_encoder()) crashed
when it was called with non-zero second argument.
Avoid tests being killed due to OOM on Linux if a system is configured with
'ulimit -s unlimited' by skipping tests relying on infinite recursion.
While unclear if Python should support 'ulimit -s unlimited', we should at
least try to avoid failing a PGO build running tests due to an unlimited
stack size being set.
Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>
* Document that ensure_ascii=True forces escaping not only non-ASCII, but also
non-printable characters (the only affected ASCII character is U+007F).
* Ensure that the help output for the json module does not exceed 80
columns (except one long line in an example and generated lines).
* Add more tests.
Removed special-casing for WASI when setting C stack depth limits. Since WASI has its own C stack checking this isn't a security risk.
Also disabled some tests that stopped passing. They all happened to have already been disabled under Emscripten.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Implement C recursion protection with limit pointers for Linux, MacOS and Windows
* Remove calls to PyOS_CheckStack
* Add stack protection to parser
* Make tests more robust to low stacks
* Improve error messages for stack overflow
Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now
Unfortunatlely, the change broke some buildbots.
This reverts commit 2498c22fa0.
* Implement C recursion protection with limit pointers
* Remove calls to PyOS_CheckStack
* Add stack protection to parser
* Make tests more robust to low stacks
* Improve error messages for stack overflow
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.
This PR comes fresh from a pile of work done in our private PSRT security response team repo.
Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
- Add requires_fork and requires_subprocess to more tests
- Skip extension import tests if dlopen is not available
- Don't assume that _testcapi is a shared extension
- Skip a lot of socket tests that don't work on Emscripten
- Skip mmap tests, mmap emulation is incomplete
- venv does not work yet
- Cannot get libc from executable
The "entire" test suite is now passing on Emscripten with EMSDK from git head (91 suites are skipped).
