Commit graph

13932 commits

Author SHA1 Message Date
Dino Viehland
92abb01240
gh-112075: Add critical sections for most dict APIs (#114508)
Starts adding thread safety to dict objects.


Use @critical_section for APIs which are exposed via argument clinic and don't directly correlate with a public C API which needs to acquire the lock
Use a _lock_held suffix for keeping changes to complicated functions simple and just wrapping them with a critical section
Acquire and release the lock in an existing function where it won't be overly disruptive to the existing logic
2024-02-06 14:03:43 -08:00
Sam Gross
b6228b521b
gh-115035: Mark ThreadHandles as non-joinable earlier after forking (#115042)
This marks dead ThreadHandles as non-joinable earlier in
`PyOS_AfterFork_Child()` before we execute any Python code. The handles
are stored in a global linked list in `_PyRuntimeState` because `fork()`
affects the entire process.
2024-02-06 14:45:04 -05:00
Sam Gross
7fdd4235d7
gh-112529: Stop the world around gc.get_referents (#114823)
We do not want to add locking in `tp_traverse` slot implementations.
Instead, stop the world when calling `gc.get_referents`. Note that the the
stop the world call is a no-op in the default build.

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2024-02-06 11:45:42 -05:00
Sam Gross
de61d4bd4d
gh-112066: Add PyDict_SetDefaultRef function. (#112123)
The `PyDict_SetDefaultRef` function is similar to `PyDict_SetDefault`,
but returns a strong reference through the optional `**result` pointer
instead of a borrowed reference.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-06 11:36:23 -05:00
Mariusz Felisiak
1a10437a14
gh-91602: Add iterdump() support for filtering database objects (#114501)
Add optional 'filter' parameter to iterdump() that allows a "LIKE"
pattern for filtering database objects to dump.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-02-06 12:34:56 +01:00
Serhiy Storchaka
652fbf88c4
gh-82626: Emit a warning when bool is used as a file descriptor (GH-111275) 2024-02-05 22:51:11 +02:00
Erlend E. Aasland
09096a1647
gh-115015: Argument Clinic: fix generated code for METH_METHOD methods without params (#115016) 2024-02-05 21:49:17 +01:00
Mark Shannon
36518e69d7
GH-108362: Incremental GC implementation (GH-108038) 2024-02-05 18:28:51 +00:00
Nikita Sobolev
87cd20a567
gh-115026: Argument Clinic: handle PyBuffer_FillInfo errors in generated code (#115027) 2024-02-05 11:45:09 +01:00
Nikita Sobolev
929d44e15a
gh-114685: PyBuffer_FillInfo() now raises on PyBUF_{READ,WRITE} (GH-114802) 2024-02-04 19:16:43 +00:00
Jason Zhang
efc489021c
gh-111417: Remove unused code block in math.trunc() and round() (GH-111454)
_PyObject_LookupSpecial() now ensures that the type is ready.
2024-02-03 17:11:10 +02:00
Sam Gross
d0f1307580
gh-114329: Add PyList_GetItemRef function (GH-114504)
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
2024-02-02 14:03:15 +01:00
Mark Shannon
0e71a295e9
GH-113710: Add a "globals to constants" pass (GH-114592)
Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
2024-02-02 12:14:34 +00:00
Sam Gross
587d480203
gh-112529: Remove PyGC_Head from object pre-header in free-threaded build (#114564)
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build

This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.

 * The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
   in the free-threaded build.
 * The GDB libpython.py file now determines the offset of the managed
   dict field based on whether the running process is a free-threaded
   build. Those are identified by the `ob_ref_local` field in PyObject.
 * Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
   size of `PyGC_Head` in the size of static `PyTypeObject`.
2024-02-01 12:29:19 -08:00
Serhiy Storchaka
b7688ef71e
gh-114685: Check flags in PyObject_GetBuffer() (GH-114707)
PyObject_GetBuffer() now raises a SystemError if called with
PyBUF_READ or PyBUF_WRITE as flags. These flags should
only be used with the PyMemoryView_* C API.
2024-01-31 13:11:35 +02:00
Diego Russo
a06b606462
gh-110190: Fix ctypes structs with array on Windows ARM64 (GH-114753) 2024-01-30 23:53:04 +00:00
Eugene Toder
1f515e8a10
gh-112919: Speed-up datetime, date and time.replace() (GH-112921)
Use argument clinic and call new_* functions directly. This speeds up
these functions 6x to 7.5x when calling with keyword arguments.
2024-01-30 15:19:46 +00:00
Serhiy Storchaka
ea30a28c3e
gh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in csv.reader (GH-113738) 2024-01-30 14:21:12 +02:00
Serhiy Storchaka
aa3402ad45
gh-114678: Fix incorrect deprecation warning for 'N' specifier in Decimal format (GH-114683)
Co-authored-by: Stefan Krah <skrah@bytereef.org>
2024-01-29 19:58:31 +02:00
Soumendra Ganguly
e351ca3c20
gh-85984: Add POSIX pseudo-terminal functions. (GH-102413)
Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-01-29 16:10:28 +00:00
Petr Viktorin
15fe8cea17
gh-91325: Skip Stable ABI checks with Py_TRACE_REFS special build (GH-92046)
Skip Stable ABI checks with Py_TRACE_REFS special build

This build is not compatible with Py_LIMITED_API nor with
the stable ABI.
2024-01-29 16:45:31 +01:00
mpage
c87233fd3f
gh-112050: Adapt collections.deque to Argument Clinic (#113963) 2024-01-29 15:08:23 +00:00
Steve Dower
102569d150
Use Unicode unconditionally for _winapi.CreateFile (GH-114611)
Currently it switches based on build settings, but argument clinic does not handle it correctly.
2024-01-26 17:27:29 +00:00
Erlend E. Aasland
dcd28b5c35
gh-114569: Use PyMem_* APIs for most non-PyObject uses (#114574)
Fix usage in Modules, Objects, and Parser subdirectories.
2024-01-26 10:11:35 +00:00
Michael Droettboom
d0f7f5c41d
gh-114312: Fix rare event counter tests on aarch64 (GH-114554) 2024-01-26 10:10:03 +00:00
Nikita Sobolev
d96358ff9d
gh-114315: Make threading.Lock a real class, not a factory function (#114479)
`threading.Lock` is now the underlying class and is constructable rather than the old
factory function. This allows for type annotations to refer to it which had no non-ugly
way to be expressed prior to this.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2024-01-25 19:46:32 +00:00
Michael Droettboom
ea3cd0498c
gh-114312: Collect stats for unlikely events (GH-114493) 2024-01-25 11:10:51 +00:00
Serhiy Storchaka
d22c066b80
gh-114492: Initialize struct termios before calling tcgetattr() (GH-114495)
On Alpine Linux it could leave some field non-initialized.
2024-01-23 23:27:04 +02:00
mpage
925907ea36
gh-113884: Make queue.SimpleQueue thread-safe when the GIL is disabled (#114161)
* use the ParkingLot API to manage waiting threads
* use Argument Clinic's critical section directive to protect queue methods
* remove unnecessary overflow check

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-01-23 20:25:41 +01:00
Serhiy Storchaka
1b719b39b9
gh-114321: Expose more constants in the fcntl module (GH-114322) 2024-01-22 18:09:22 +02:00
neonene
9f7176d360
gh-103092: Ensure _ctypes.c static types are accessed via global state (#113857) 2024-01-22 14:40:36 +01:00
Serhiy Storchaka
c8351a617b
gh-113796: Add more validation checks in the csv.Dialect constructor (GH-113797)
ValueError is now raised if the same character is used in different roles.
2024-01-22 15:34:16 +02:00
Nikita Sobolev
d1b031cc58
gh-114414: Assert PyType_GetModuleByDef result in _threadmodule (#114415) 2024-01-22 10:19:25 +01:00
mpage
28eacf27ef
gh-113884: Refactor queue.SimpleQueue to use a ring buffer to store items (#114259)
Use a ring buffer instead of a Python list in order to simplify the
process of making queue.SimpleQueue thread-safe in free-threaded
builds. The ring buffer implementation has no places where critical
sections may be released.
2024-01-19 12:17:51 +00:00
Nikita Sobolev
05e47202a3
gh-114286: Fix maybe-uninitialized warning in Modules/_io/fileio.c (GH-114287) 2024-01-19 10:25:05 +00:00
Skip Montanaro
72abb8c5d4
gh-114123: Migrate docstring from _csv to csv (#114124)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Éric <merwok@netwok.org>
2024-01-18 22:18:42 +00:00
AN Long
8e31cdc945
gh-103092: Convert some _ctypes metatypes to heap types (GH-113620)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-01-18 16:30:27 +01:00
Serhiy Storchaka
e2c097ebde
gh-104522: Fix OSError raised when run a subprocess (#114195)
Only set filename to cwd if it was caused by failed chdir(cwd).

_fork_exec() now returns "noexec:chdir" for failed chdir(cwd).

Co-authored-by: Robert O'Shea <PurityLake@users.noreply.github.com>
2024-01-17 16:52:42 -08:00
Radislav Chugunov
0154405350
gh-104282: Fix null pointer dereference in lzma._decode_filter_properties (GH-104283) 2024-01-17 13:15:44 +00:00
Steve Dower
de4ced54eb
gh-114096: Restore privileges in _winapi.CreateJunction after creating the junction (GH-114089)
This avoids impact on later parts of the application which may be able to do things they otherwise shouldn't.
2024-01-16 16:40:02 +00:00
Jonathon Reinhart
e454f9383c
Fix an incorrect comment in iobase_is_closed (GH-102952)
This comment appears to have been mistakenly copied from what is now
called iobase_check_closed() in commit 4d9aec0220.

Also unite the iobase_check_closed() code with the relevant comment.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-01-16 18:27:17 +02:00
AN Long
6c502ba809
gh-114101: Correct PyErr_Format arguments in _testcapi module (#114102)
- use PyErr_SetString() iso. PyErr_Format() in parse_tuple_and_keywords()
- fix misspelled format specifier in CHECK_SIGNNESS() macro
2024-01-16 09:32:39 +01:00
Zackery Spytz
8fd287b18f
gh-78502: Add a trackfd parameter to mmap.mmap() (GH-25425)
If *trackfd* is False, the file descriptor specified by *fileno*
will not be duplicated.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-01-16 08:51:46 +01:00
Ronald Oussoren
2010d45327
gh-113666: Adding missing UF_ and SF_ flags to module 'stat' (#113667)
Add some constants to module 'stat' that are used on macOS.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-01-15 12:22:43 +01:00
Ronald Oussoren
79970792fd
gh-113868: Add a number of MAP_* flags from macOS to module mmap (#113869)
The new flags were extracted from the macOS 14.2 SDK.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-01-12 16:56:18 +01:00
Steve Dower
ed066481c7
gh-111877: Fixes stat() handling for inaccessible files on Windows (GH-113716) 2024-01-12 15:27:56 +00:00
Zackery Spytz
b4d4aa9e8d
gh-81489: Use Unicode APIs for mmap tagname on Windows (GH-14133)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-01-11 22:39:47 +00:00
Peter Lazorchak
f653caa5a8
gh-89811: Check for valid tp_version_tag in specializer (GH-113558) 2024-01-11 13:33:05 +08:00
Victor Stinner
1d75fa43a2
gh-77046: os.pipe() sets _O_NOINHERIT flag on fds (#113817)
On Windows, set _O_NOINHERIT flag on file descriptors
created by os.pipe() and io.WindowsConsoleIO.

Add test_pipe_spawnl() to test_os.

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2024-01-10 23:02:17 +01:00
Victor Stinner
93930eaf0a
gh-111139: Optimize math.gcd(int, int) (#113887)
Add a fast-path for the common case.

Benchmark:

    python -m pyperf timeit \
        -s 'import math; gcd=math.gcd; x=2*3; y=3*5' \
        'gcd(x,y)'

Result: 1.07x faster (-3.4 ns)

    Mean +- std dev: 52.6 ns +- 4.0 ns -> 49.2 ns +- 0.4 ns: 1.07x faster
2024-01-10 16:38:56 +01:00