gh-119451: Fix a potential denial of service in http.client (GH-119454)
Reading the whole body of the HTTP response could cause OOM if
the Content-Length value is too large even if the server does not send
a large amount of data. Now the HTTP client reads large data by chunks,
therefore the amount of consumed memory is proportional to the amount
of sent data.
(cherry picked from commit 5a4c4a033a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-142145: Remove quadratic behavior in node ID cache clearing (GH-142146)
* Remove quadratic behavior in node ID cache clearing
Co-authored-by: Jacob Walls <38668450+jacobtylerwalls@users.noreply.github.com>
* Add news fragment
---------
(cherry picked from commit 08d8e18ad8)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Jacob Walls <38668450+jacobtylerwalls@users.noreply.github.com>
* [3.14] gh-142754: Ensure that Element & Attr instances have the ownerDocument attribute (GH-142794) (#142818)
gh-142754: Ensure that Element & Attr instances have the ownerDocument attribute (GH-142794)
(cherry picked from commit 1cc7551b3f)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* gh-142145: relax the no-longer-quadratic test timing (GH-143030)
* gh-142145: relax the no-longer-quadratic test timing
* require cpu resource
(cherry picked from commit 8d2d7bb2e7)
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
* merge NEWS entries into one
---------
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Jacob Walls <38668450+jacobtylerwalls@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
[3.14] gh-119452: Fix a potential virtual memory allocation denial of service in http.server (GH-142216)
The CGI server on Windows could consume the amount of memory specified
in the Content-Length header of the request even if the client does not
send such much data. Now it reads the POST request body by chunks,
therefore the memory consumption is proportional to the amount of sent
data.
(cherry picked from commit 0e4f4f1a46)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-143935: Email preserve parens when folding comments (GH-143936)
Fix a bug in the folding of comments when flattening an email message
using a modern email policy. Comments consisting of a very long sequence of
non-foldable characters could trigger a forced line wrap that omitted the
required leading space on the continuation line, causing the remainder of
the comment to be interpreted as a new header field. This enabled header
injection with carefully crafted inputs.
(cherry picked from commit 17d1490aa9)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Denis Ledoux <dle@odoo.com>
gh-143919: Reject control characters in http cookies
(cherry picked from commit 95746b3a13)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-143916: Reject control characters in wsgiref.headers.Headers (GH-143917)
* Add 'test.support' fixture for C0 control characters
* gh-143916: Reject control characters in wsgiref.headers.Headers
(cherry picked from commit f7fceed79c)
(cherry picked from commit 22e4d55285)
Co-authored-by: Seth Michael Larson <seth@python.org>
Currently, Fedora 42 uses a custom Linux Kernel 6.16.9 that backported an upstream change
from 6.17-rc7 [1,3] but not its subsequent fix [2]. Until the issue is resolved upstream,
we skip the failing test `test_socket.test_aead_aes_gcm` for kernel versions between 6.16
and 6.17.x.
[1] 1b34cbbf4f
[2] d0ca0df179.
[3] 45bcf60fe4
(cherry picked from commit 41712c4e09)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
(cherry picked from commit 333d4a6f49)
Support records with "zip64 extensible data" if there are no bytes
prepended to the ZIP file.
(cherry picked from commit 162997bb70)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
"] ]>" and "]] >" no longer end the CDATA section.
Make CDATA section parsing context depending.
Add private method HTMLParser._set_support_cdata() to change the context.
If called with True, "<[CDATA[" starts a CDATA section which ends with "]]>".
If called with False, "<[CDATA[" starts a bogus comments which ends with ">".
(cherry picked from commit 0cbbfc4621)
(cherry picked from commit dcf24768c9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-139400: Make sure that parent parsers outlive their subparsers in `pyexpat` (#139403)
* Modules/pyexpat.c: Disallow collection of in-use parent parsers.
Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.
(cherry picked from commit 6edb2ddb5f)
* Move news item from section "Core and Builtins" to section "Security"
ensurepip installs a bundled copy of distutils, which overrides
the stdlib module. This affects several tests. This commit:
- skips distutils in test___all__, as we're unlikely to break
`__all__` in a security-fix-only branch (and if we do it's not
much of a a big deal)
- skips importability tests of distutils submodules if the
setuptools hack is detected
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Emma Smith <emma@emmatyping.dev>
gh-130577: tarfile now validates archives to ensure member offsets are non-negative (GH-137027)
(cherry picked from commit 7040aa54f1)
Co-authored-by: Alexander Urieles <aeurielesn@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Whitespaces no longer accepted between `</` and the tag name.
E.g. `</ script>` does not end the script section.
* Vertical tabulation (`\v`) and non-ASCII whitespaces no longer recognized
as whitespaces. The only whitespaces are `\t\n\r\f `.
* Null character (U+0000) no longer ends the tag name.
* Attributes and slashes after the tag name in end tags are now ignored,
instead of terminating after the first `>` in quoted attribute value.
E.g. `</script/foo=">"/>`.
* Multiple slashes and whitespaces between the last attribute and closing `>`
are now ignored in both start and end tags. E.g. `<a foo=bar/ //>`.
* Multiple `=` between attribute name and value are no longer collapsed.
E.g. `<a foo==bar>` produces attribute "foo" with value "=bar".
* Whitespaces between the `=` separator and attribute name or value are no
longer ignored. E.g. `<a foo =bar>` produces two attributes "foo" and
"=bar", both with value None; `<a foo= bar>` produces two attributes:
"foo" with value "" and "bar" with value None.
* Fix data loss after unclosed script or style tag (gh-86155).
Also backport test.support.subTests() (gh-135120).
---------
(cherry picked from commit 0243f97cba)
(cherry picked from commit c555f889c3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Waylan Limberg <waylan.limberg@icloud.com>
End-of-file errors are now handled according to the HTML5 specs --
comments and declarations are automatically closed, tags are ignored.
(cherry picked from commit 6eb6c5dbfb)
Fix functionality that was broken with better textual representation for IPv4-mapped addresses (gh-87799)
(cherry picked from commit 77a2fb4bf1)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Limit length of IP address string to 39
(cherry picked from commit 47f1161d3a)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Represent IPv4-mapped IPv6 address as x❌x❌x❌d.d.d.d,
where the 'x's are the hexadecimal values
of the six high-order 16-bit pieces of the address,
and the 'd's are the decimal values
of the four low-order 8-bit pieces of the address
(standard IPv4 representation).
(cherry picked from commit f22bf8e3cf)
Co-authored-by: opavliuk <40970635+opavlyuk@users.noreply.github.com>
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
If the error handler is used, a new bytes object is created to set as
the object attribute of UnicodeDecodeError, and that bytes object then
replaces the original data. A pointer to the decoded data will became invalid
after destroying that temporary bytes object. So we need other way to return
the first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal().
_PyBytes_DecodeEscape() does not have such issue, because it does not
use the error handlers registry, but it should be changed for compatibility
with _PyUnicode_DecodeUnicodeEscapeInternal().
(cherry picked from commit 9f69a58623)
(cherry picked from commit 6279eb8c07)
(cherry picked from commit a75953b347)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503)
If `read()` in the ConnectionHandler thread raises `OSError` (except `ConnectionError`),
the ConnectionHandler shuts down the entire ThreadedEchoServer,
preventing further connections.
It also does that for `EPROTOTYPE` in `wrap_conn`.
As far as I can see, this is done to avoid the server thread getting stuck,
forgotten, in its accept loop. However, since 2011 (5b95eb90a7)
the server is used as a context manager, and its `__exit__` does `stop()` and `join()`.
(I'm not sure if we *always* used `with` since that commit, but currently we do.)
Make sure that the context manager *is* used, and remove the `server.stop()`
calls from ConnectionHandler.
(cherry picked from commit c9cda1608e)
(cherry picked from commit aee80cd5e7)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Email generators using email.policy.default may convert an RFC 2047
encoded-word to unencoded form during header refolding. In a structured
header, this could allow 'specials' chars outside a quoted-string,
leading to invalid address headers and enabling spoofing. This change
ensures a parsed encoded-word that contains specials is kept as an
encoded-word while the header is refolded.
[Better fix from @bitdancer.]
(cherry picked from commit 295b53df2a)
Co-authored-by: Mike Edmunds <medmunds@gmail.com>
Co-authored-by: R David Murray <rdmurray@bitdance.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
When using threaded applications, there is a high risk of a deadlock in
the interpreter. It's a lock ordering deadlock with HEAD_LOCK(&_PyRuntime); and the GIL.
By disabling the GC during the _PyThread_CurrentFrames() and
_PyThread_CurrentExceptions() calls fixes the issue.
The IMAP4 client could consume an arbitrary amount of memory when trying
to connect to a malicious server, because it read a "literal" data with a
single read(size) call, and BufferedReader.read() allocates the bytes
object of the specified size before reading. Now the IMAP4 client reads data
by chunks, therefore the amount of used memory is limited by the
amount of the data actually been sent by the server.
(cherry picked from commit 735f25c5e3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Email generators using email.policy.default could incorrectly omit the
quote ('"') characters from a quoted-string during header refolding,
leading to invalid address headers and enabling header spoofing. This
change restores the quote characters on a bare-quoted-string as the
header is refolded, and escapes backslash and quote chars in the string.
(cherry picked from commit 5aaf416858)
Co-authored-by: Mike Edmunds <medmunds@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-119461: Restore the testSocket VSOCK skipUnless removed by PR GH-119465 (GH-129561)
Restore the skipUnless removed by GH-119465.
This test can only pass on virtual machines, not actual machines.
actual machines see:
```
self.cli.connect((cid, VSOCKPORT))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
OSError: [Errno 19] No such device
```
Reproduced on (Linux) Ubuntu 24.04.1 running 6.8.0-52-generic.
(cherry picked from commit e1006ce1de)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Fix ThreadedVSOCKSocketStreamTest: if get_cid() returns the host
address or the "any" address, use the local communication address
(loopback): VMADDR_CID_LOCAL.
On Linux 6.9, apparently, the /dev/vsock device is now available but
get_cid() returns VMADDR_CID_ANY (-1).
(cherry picked from commit e94dbe4ed8)
(cherry picked from commit c750061047)
(cherry picked from commit cbfe3023e4)
