Commit graph

28987 commits

Author SHA1 Message Date
Paulo Neves
4abca7e1e7
gh-98966: Handle stdout=subprocess.STDOUT (GH-98967)
Explicitly handle the case where stdout=STDOUT
as otherwise the existing error handling gets
confused and reports hard to understand errors.

Signed-off-by: Paulo Neves <ptsneves@gmail.com>
2024-03-26 13:37:50 +01:00
Serhiy Storchaka
9654daf793
gh-66543: Fix mimetype.guess_type() (GH-117217)
Fix parsing of the following corner cases:

* URLs with only a host name
* URLs containing a fragment
* URLs containing a query
* filenames with only a UNC sharepoint on Windows

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2024-03-26 13:26:45 +02:00
Mark Shannon
8bef34f625
GH-117108: Set the "old space bit" to "visited" for all young objects (#117213)
Change old space bit of young objects from 0 to gcstate->visited_space.
This ensures that any object created *and* collected during cycle GC has the bit set correctly.
2024-03-26 11:11:42 +00:00
Pablo Galindo Salgado
61599a48f5
bpo-24612: Improve syntax error for 'not' after an operator (GH-28170)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2024-03-26 10:30:46 +01:00
yevgeny hong
ea9a296fce
gh-115627: Fix PySSL_SetError handling SSL_ERROR_SYSCALL (GH-115628)
Python 3.10 changed from using SSL_write() and SSL_read() to SSL_write_ex() and
SSL_read_ex(), but did not update handling of the return value.

Change error handling so that the return value is not examined.
OSError (not EOF) is now returned when retval is 0.

According to *recent* man pages of all functions for which we call
PySSL_SetError, (in OpenSSL 3.0 and 1.1.1), their return value should
be used to determine whether an error happened (i.e. if PySSL_SetError
should be called), but not what kind of error happened (so,
PySSL_SetError shouldn't need retval). To get the error,
we need to use SSL_get_error.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-03-26 08:45:43 +01:00
Victor Stinner
d52bdfb19f
gh-83434: Disable XML in regrtest when -R option is used (#117232) 2024-03-26 08:35:59 +01:00
Sebastian Pipping
9f74e86c78
gh-117187: Fix XML tests for vanilla Expat <2.6.0 (GH-117203)
This fixes XML unittest fallout from the https://github.com/python/cpython/issues/115398 security fix.  When configured using `--with-system-expat` on systems with older pre 2.6.0 versions of libexpat, our unittests were failing.

* sax|etree: Simplify Expat version guard where simplifiable

Idea by Matěj Cepl

* sax|etree: Fix reparse deferral tests for vanilla Expat <2.6.0

This *does not fix* the case of distros with an older version of libexpat with the 2.6.0 feature backported as a security fix.  (Ubuntu is a known example of this with its libexpat1 2.5.0-2ubunutu0.1 package)
2024-03-25 18:48:27 -07:00
Jonathan Protzenko
872e212378
gh-99108: Refresh HACL*; update modules accordingly; fix namespacing (GH-117237)
Pulls in a new update from https://github.com/hacl-star/hacl-star and fixes our C "namespacing" done by `Modules/_hacl/refresh.sh`.
2024-03-26 00:35:26 +00:00
Eric V. Smith
8945b7ff55
gh-109870: Dataclasses: batch up exec calls (gh-110851)
Instead of calling `exec()` once for each function added to a dataclass, only call `exec()` once per dataclass. This can lead to speed improvements of up to 20%.
2024-03-25 19:59:14 -04:00
Nice Zombies
0821923aa9
gh-117114: Make os.path.isdevdrive available on all platforms (GH-117115) 2024-03-25 22:55:11 +00:00
Victor Stinner
507896d97d
gh-116936: Add PyType_GetModuleByDef() to the limited C API (#116937) 2024-03-25 16:32:20 +00:00
Serhiy Storchaka
0c1a42cf9c
gh-87193: Support bytes objects with refcount > 1 in _PyBytes_Resize() (GH-117160)
Create a new bytes object and destroy the old one if it has refcount > 1.
2024-03-25 16:32:11 +01:00
Tian Gao
01e7405da4
gh-112948: Make pdb completion similar to repl completion (#112950) 2024-03-25 15:18:09 +00:00
Irit Katriel
d610d821fd
gh-112383: teach dis how to interpret ENTER_EXECUTOR (#117171) 2024-03-23 22:32:33 +00:00
Erik Soma
f11d0d8be8
gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (#32011) 2024-03-23 08:39:35 -07:00
Mark Shannon
e28477f214
GH-117108: Change the size of the GC increment to about 1% of the total heap size. (GH-117120) 2024-03-22 18:43:25 +00:00
Serhiy Storchaka
e2e0b4b4b9
gh-113024: C API: Add PyObject_GenericHash() function (GH-113025) 2024-03-22 20:19:10 +02:00
Serhiy Storchaka
567ab3bd15
gh-117084: Fix ZIP file extraction for directory entry names with backslashes on Windows (GH-117129) 2024-03-22 20:08:00 +02:00
Jakub Stasiak
40d75c2b7f
GH-113171: Fix "private" (non-global) IP address ranges (GH-113179)
* GH-113171: Fix "private" (really non-global) IP address ranges

The _private_networks variables, used by various is_private
implementations, were missing some ranges and at the same time had
overly strict ranges (where there are more specific ranges considered
globally reachable by the IANA registries).

This patch updates the ranges with what was missing or otherwise
incorrect.

I left 100.64.0.0/10 alone, for now, as it's been made special in [1]
and I'm not sure if we want to undo that as I don't quite understand the
motivation behind it.

The _address_exclude_many() call returns 8 networks for IPv4, 121
networks for IPv6.

[1] https://github.com/python/cpython/issues/61602
2024-03-22 17:49:56 +01:00
Thomas A Caswell
c4bf58a14f
gh-116745: Remove all internal usage of @LIBPYTHON@ (#116746)
Replace with MODULE_LDFLAGS.
2024-03-22 00:54:50 +01:00
Malcolm Smith
3ec57307e7
gh-71052: Add Android build script and instructions (#116426) 2024-03-22 00:52:29 +01:00
Will Childs-Klein
c85d84166a
gh-116333: Relax error string text expectations in SSL-related tests (GH-116334)
* Relax error string text expectations in SSL-related tests

As suggested [here][1], this change relaxes the OpenSSL error string
text expectations in a number of tests. This was specifically done in
support of more easily building CPython [AWS-LC][2], but because AWS-LC
is a fork of [BoringSSL][3], it should increase compatibility with that
library as well.

In addition to the error string relaxations, we also add some guards
around the `tls-unique` channel binding being used with TLSv1.3, as that
feature (described in [RFC 6929][4]) is [not defined][5] for TLSv1.3.

[1]: https://discuss.python.org/t/support-building-ssl-and-hashlib-modules-against-aws-lc/44505/4
[2]: https://github.com/aws/aws-lc
[3]: https://github.com/google/boringssl
[4]: https://datatracker.ietf.org/doc/html/rfc5929#section-3
[5]: https://datatracker.ietf.org/doc/html/rfc8446#appendix-C.5
2024-03-21 19:16:36 +00:00
Victor Stinner
abdd1f938f
gh-85283: Build _testconsole extension with limited C API (#117125) 2024-03-21 17:45:43 +01:00
Victor Stinner
8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Malcolm Smith
1f8b24ef69
gh-71052: Implement ctypes.util.find_library on Android (GH-116379) 2024-03-21 14:20:57 +01:00
Tian Gao
d16c9d1278
gh-116987: Support class code objects in inspect.findsource() (GH-117025) 2024-03-21 10:30:10 +00:00
Adam Turner
6547330f4e
GH-109653: Defer import of `importlib.metadata._adapters` (#109829)
* adapters

* Add comments for deferred imports with links to rationale.

* Add blurb

---------

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2024-03-21 03:49:10 +00:00
Jason R. Coombs
8ad8898420
gh-117089: Move importlib.metadata tests to their own package (#117092)
* Ensure importlib.metadata tests do not leak references in sys.modules.

* Move importlib.metadata tests to their own package for easier syncing with importlib_metadata.

* Update owners and makefile for new directories.

* Add blurb
2024-03-20 17:11:00 -04:00
Petr Viktorin
dcaf33a41d
gh-114314: ctypes: remove stgdict and switch to heap types (GH-116458)
Before this change, ctypes classes used a custom dict subclass, `StgDict`,
as their `tp_dict`. This acts like a regular dict but also includes extra information
about the type.

This replaces stgdict by `StgInfo`, a C struct on the type, accessed by
`PyObject_GetTypeData()` (PEP-697).
All usage of `StgDict` (mainly variables named `stgdict`, `dict`, `edict` etc.) is
converted to `StgInfo` (named `stginfo`, `info`, `einfo`, etc.).
Where the dict is actually used for class attributes (as a regular PyDict), it's now
called `attrdict`.

This change -- not overriding `tp_dict` -- is made to make me comfortable with
the next part of this PR: moving the initialization logic from `tp_new` to `tp_init`.

The `StgInfo` is set up in `__init__` of each class, with a guard that prevents
calling `__init__` more than once. Note that abstract classes (like `Array` or
`Structure`) are created using `PyType_FromMetaclass` and do not have
`__init__` called.
Previously, this was done in `__new__`, which also wasn't called for abstract
classes.
Since `__init__` can be called from Python code or skipped, there is a tested
guard to ensure `StgInfo` is initialized exactly once before it's used.

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-20 17:33:08 +01:00
jkriegshauser
fc45998007
gh-116773: Ensure overlapped objects on Windows are not deallocated too early by asyncio (GH-116774) 2024-03-20 14:33:28 +00:00
Serhiy Storchaka
519b2ae22b
gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms (GH-117064) 2024-03-20 15:39:53 +02:00
Mark Shannon
15309329b6
GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Sam Gross
60e105c1c1
gh-113964: Don't prevent new threads until all non-daemon threads exit (#116677)
Starting in Python 3.12, we prevented calling fork() and starting new threads
during interpreter finalization (shutdown). This has led to a number of
regressions and flaky tests. We should not prevent starting new threads
(or `fork()`) until all non-daemon threads exit and finalization starts in
earnest.

This changes the checks to use `_PyInterpreterState_GetFinalizing(interp)`,
which is set immediately before terminating non-daemon threads.
2024-03-19 14:40:20 -04:00
et-repositories
75935746be
gh-116647: Fix recursive child in dataclasses (#116790) 2024-03-19 08:58:40 -06:00
Victor Stinner
27cf3ed00c
gh-90872: Fix subprocess.Popen.wait() for negative timeout (#116989)
On Windows, subprocess.Popen.wait() no longer calls
WaitForSingleObject() with a negative timeout: pass 0 ms if the
timeout is negative.
2024-03-19 14:42:44 +01:00
Russell Keith-Magee
408e127159
gh-114099 - Add iOS framework loading machinery. (GH-116454)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-19 08:36:19 -04:00
David Röthlisberger
b1bc37597f
gh-116957: configparser: Do post-process values after DuplicateOptionError (GH-116958)
If you catch DuplicateOptionError / DuplicateSectionError when reading a
config file (the intention is to skip invalid config files) and then
attempt to use the ConfigParser instance, any values it *had* read
successfully so far, were stored as a list instead of string! Later
`get` calls would raise "AttributeError: 'list' object has no attribute
'find'" from somewhere deep in the interpolation code.
2024-03-19 11:59:08 +02:00
Bogdan Romanyuk
a8e93d3dca
gh-115756: make PyCode_GetFirstFree an unstable API (GH-115781) 2024-03-19 09:20:38 +00:00
Aviel Boag
a22d05f04c
gh-105866: fix dataclass with slots=True, weakref_slot=True (#105870)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-03-18 18:53:14 -06:00
Pierre Ossman (ThinLinc team)
4159644177
gh-113538: Add asycio.Server.{close,abort}_clients (redo) (#116784)
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.

This is a do-over with a test fix for gh-114432, which was reverted.
2024-03-18 13:15:53 -07:00
Victor Stinner
a9c304cf02
gh-116869: Make C API compatible with ISO C90 (#116950)
Make the C API compatible with -Werror=declaration-after-statement
compiler flag again.
2024-03-18 20:16:58 +01:00
Victor Stinner
590a26010d
gh-116869: Add test_cext test: build a C extension (#116954) 2024-03-18 20:15:20 +01:00
Zachary Ware
849e0716d3
gh-115119: Switch Windows build to mpdecimal external (GH-115182)
This includes adding what should be a relatively temporary
`Modules/_decimal/windows/mpdecimal.h` shim to choose between `mpdecimal32vc.h`
or `mpdecimal64vc.h` based on which of `CONFIG_64` or `CONFIG_32` is defined.
2024-03-18 12:07:25 -05:00
Victor Stinner
1d95451be1
gh-63207: Use GetSystemTimePreciseAsFileTime() in time.time() (#116822) 2024-03-18 17:13:01 +01:00
kernc
52ef4430a9
gh-71765: Fix inspect.getsource() on empty file (GH-20809)
* bpo-27578: Fix inspect.getsource() on empty file

For modules from empty files, `inspect.getsource()` now
returns an empty string, and `inspect.getsourcelines()` returns
a list of one empty string, fixing the expected invariant.

As indicated by `exec('')`, empty strings are valid Python
source code.

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2024-03-18 16:13:02 +01:00
AN Long
cd2ed91780
gh-115538: Emit warning when use bool as fd in _io.WindowsConsoleIO (GH-116925) 2024-03-18 11:48:50 +00:00
Victor Stinner
2982bdb936
gh-85283: Build _statistics extension with the limited C API (#116927)
Argument Clinic now inlines _PyArg_CheckPositional() for the limited
C API. The generated code should be as fast or even a little bit
faster.
2024-03-17 18:59:02 +01:00
Victor Stinner
1cf0301086
gh-85283: Build termios extension with the limited C API (#116928) 2024-03-17 15:12:29 +00:00
Victor Stinner
8e3c953b3a
gh-73468: Add math.fma() function (#116667)
Added new math.fma() function, wrapping C99's ``fma()`` operation:
fused multiply-add function.

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
2024-03-17 13:58:26 +00:00
John Sloboda
649857a157
gh-85287: Change codecs to raise precise UnicodeEncodeError and UnicodeDecodeError (#113674)
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2024-03-17 04:58:42 +00:00