Commit graph

797 commits

Author SHA1 Message Date
Steve Dower
2894aa14f2
gh-121115: Skip __index__ in PyLong_AsNativeBytes by default (GH-121118) 2024-06-28 16:26:21 +01:00
Victor Stinner
e51e880e75
gh-120593: Fix const qualifier in _PyLong_CompactValue() (#121053)
Remove the const qualifier of the argument of functions:

* _PyLong_IsCompact()
* _PyLong_CompactValue()

Py_TYPE() argument is not const.

Fix the compiler warning:

  Include/cpython/longintrepr.h: In function ‘_PyLong_CompactValue’:
  Include/pyport.h:19:31: error: cast discards ‘const’ qualifier from
  pointer target type [-Werror=cast-qual]
    (...)
  Include/cpython/longintrepr.h:133:30: note: in expansion of macro
  ‘Py_TYPE’
    assert(PyType_HasFeature(Py_TYPE(op), Py_TPFLAGS_LONG_SUBCLASS));
2024-06-26 20:11:21 +02:00
Victor Stinner
9cd2dcbb87
gh-120593: Fix const qualifier in pyatomic.h (#121055) 2024-06-26 20:10:47 +02:00
Victor Stinner
9e4a81f00f
gh-120642: Move private PyCode APIs to the internal C API (#120643)
* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
2024-06-26 13:54:03 +02:00
Eric Snow
a905721b9c
gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121010)
We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence().

This change only affects internal API.
2024-06-25 14:35:12 -06:00
Victor Stinner
2e157851e3
gh-119182: Add PyUnicodeWriter_WriteUCS4() function (#120849) 2024-06-24 17:40:39 +02:00
Irit Katriel
65a12c559c
gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835) 2024-06-24 10:23:38 +01:00
Sam Gross
8f17d69b7b
gh-119344: Make critical section API public (#119353)
This makes the following macros public as part of the non-limited C-API for
locking a single object or two objects at once.

* `Py_BEGIN_CRITICAL_SECTION(op)` / `Py_END_CRITICAL_SECTION()`
* `Py_BEGIN_CRITICAL_SECTION2(a, b)` / `Py_END_CRITICAL_SECTION2()`

The supporting functions and structs used by the macros are also exposed for
cases where C macros are not available.
2024-06-21 15:50:18 -04:00
Victor Stinner
4123226bbd
gh-119182: Add PyUnicodeWriter_DecodeUTF8Stateful() (#120639)
Add PyUnicodeWriter_WriteWideChar() and
PyUnicodeWriter_DecodeUTF8Stateful() functions.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-06-21 19:33:15 +02:00
Sam Gross
3af7263037
gh-117511: Make PyMutex public in the non-limited API (#117731) 2024-06-20 11:29:08 -04:00
Sam Gross
e8752d7b80
gh-118789: Add PyUnstable_Object_ClearWeakRefsNoCallbacks (#118807)
This exposes `PyUnstable_Object_ClearWeakRefsNoCallbacks` as an unstable
C-API function to provide a thread-safe mechanism for clearing weakrefs
without executing callbacks.

Some C-API extensions need to clear weakrefs without calling callbacks,
such as after running finalizers like we do in subtype_dealloc.
Previously they could use `_PyWeakref_ClearRef` on each weakref, but
that's not thread-safe in the free-threaded build.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-06-18 09:57:23 -04:00
Victor Stinner
5c4235cd8c
gh-119182: Add PyUnicodeWriter C API (#119184) 2024-06-17 17:10:52 +02:00
Victor Stinner
c2d5df5787
gh-83754: Use the Py_TYPE() macro (#120599)
Don't access directly PyObject.ob_type, but use the Py_TYPE() macro
instead.
2024-06-17 10:34:29 +02:00
Xie Yanbo
6a97929a5a
Fix typos in comments (#120188) 2024-06-07 10:19:41 +02:00
Sergey B Kirpichev
61d3ab32da
gh-116560: Add PyLong_GetSign() public function (#116561)
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-06-03 14:06:31 +02:00
Victor Stinner
70934fb469
gh-112026: Deprecate _PyDict_GetItemStringWithError() function (#119855) 2024-06-03 10:26:13 +02:00
Ethan Smith
e50fac96e8
gh-119336: Restore removed _PyLong_NumBits() function (#119418)
It is used by the pywin32 project.
2024-05-30 14:48:18 +00:00
Brett Simmers
be1dfccdf2
gh-118727: Don't drop the GIL in drop_gil() unless the current thread holds it (#118745)
`drop_gil()` assumes that its caller is attached, which means that the current
thread holds the GIL if and only if the GIL is enabled, and the enabled-state
of the GIL won't change. This isn't true, though, because `detach_thread()`
calls `_PyEval_ReleaseLock()` after detaching and
`_PyThreadState_DeleteCurrent()` calls it after removing the current thread
from consideration for stop-the-world requests (effectively detaching it).

Fix this by remembering whether or not a thread acquired the GIL when it last
attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()`
instead of `gil->enabled`.

This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've
reenabled it.
2024-05-23 16:59:35 -04:00
Jelle Zijlstra
e9875ecb5d
gh-119180: PEP 649: Add __annotate__ attributes (#119209) 2024-05-22 04:38:12 +02:00
Irit Katriel
6e9863d7a3
gh-118692: Avoid creating unnecessary StopIteration instances for monitoring (#119216) 2024-05-21 20:42:51 +00:00
Victor Stinner
f6da790122
gh-111389: Add PyHASH_MULTIPLIER constant (#119214) 2024-05-21 19:51:51 +02:00
Petr Viktorin
941eea0a27
gh-118771: Ensure names defined in optimizer.h start with Py/_Py (GH-118825) 2024-05-10 18:20:12 +02:00
Sam Gross
db5af7da09
gh-118789: Restore hidden _PyWeakref_ClearRef (#118797)
_PyWeakref_ClearRef was previously exposed in the public C-API, although
it begins with an underscore and is not documented. It's used by a few
C-API extensions. There is currently no alternative public API that can
replace its use.

_PyWeakref_ClearWeakRefsExceptCallbacks is the only thread-safe way to
use _PyWeakref_ClearRef in the free-threaded build. This exposes the C
symbol, but does not make the API public.
2024-05-10 15:54:18 +02:00
Dino Viehland
ff6cbb2503
gh-112075: use per-thread dict version pool (#118676)
use thread state set of dict versions
2024-05-07 00:22:26 +00:00
Mark Shannon
616b745b89
GH-115709: Invalidate executors when a local variable is changed via frame.f_locals (#118639)
Also fix unrelated assert in debug Tier2/JIT builds.
2024-05-06 21:21:06 +00:00
Dino Viehland
5a1618a2c8
gh-118362: Fix thread safety around lookups from the type cache in the face of concurrent mutators (#118454)
Add _PyType_LookupRef and use incref before setting attribute on type
Makes setting an attribute on a class and signaling type modified atomic
Avoid adding re-entrancy exposing the type cache in an inconsistent state by decrefing after type is updated
2024-05-06 10:50:35 -07:00
Tian Gao
b034f14a4b
gh-74929: Implement PEP 667 (GH-115153) 2024-05-04 12:12:10 +01:00
Irit Katriel
85af789961
gh-111997: C-API for signalling monitoring events (#116413) 2024-05-04 08:23:50 +00:00
Pablo Galindo Salgado
6bcbee09df
gh-93502: Add new C-API functions to trace object creation and destruction (#115945) 2024-05-02 19:30:00 +02:00
Mark Shannon
67bba9dd0f
GH-117442: Check eval-breaker at start (rather than end) of tier 2 loops (GH-118482) 2024-05-02 13:10:31 +01:00
Victor Stinner
b52c753e0f
gh-110850: Add PyTime_TimeRaw() function (#118394)
Add "Raw" variant of PyTime functions:

* PyTime_MonotonicRaw()
* PyTime_PerfCounterRaw()
* PyTime_TimeRaw()

Changes:

* Add documentation and tests. Tests release the GIL while calling
  raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
  the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-01 18:05:01 +00:00
Brandt Bucher
49baa656cb
GH-115802: Use the GHC calling convention in JIT code (GH-118287) 2024-05-01 08:05:53 -07:00
Irit Katriel
c1bf4874c1
gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
2024-05-01 13:01:16 +02:00
Mark Shannon
f6fab21721
GH-118095: Make invalidating and clearing executors memory safe (GH-118459) 2024-05-01 11:34:50 +01:00
Sam Gross
79688b5b0e
gh-118331: Handle errors in _PyObject_SetManagedDict (#118334)
When detaching a dict, the `copy_values` call may fail due to
out-of-memory errors. This can be triggered by test_no_memory in
test_repl.
2024-04-29 15:49:01 -04:00
Dino Viehland
5da0280648
gh-117657: Fixes a few small TSAN issues in dictobject (#118200)
Fixup TSAN errors for dict
2024-04-25 08:53:29 -07:00
Itamar Oren
796b3fb280
gh-118207: Rename the COMMON_FIELDS macro in funcobject.h and undef it after use (GH-118208) 2024-04-25 08:49:15 +02:00
Dino Viehland
8b541c017e
gh-112075: Make instance attributes stored in inline "dict" thread safe (#114742)
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
Dino Viehland
07525c9a85
gh-116818: Make sys.settrace, sys.setprofile, and monitoring thread-safe (#116775)
Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe.

Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version.  There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
2024-04-19 14:47:42 -07:00
Donghee Na
710c01be94
gh-112069: Make PySet_GET_SIZE to be atomic safe. (gh-118053)
gh-112069: Make PySet_GET_SIZE to be atomic operation
2024-04-19 06:40:28 +09:00
Mark Shannon
147cd0581e
GH-117760: Streamline the trashcan mechanism (GH-117763) 2024-04-17 11:08:05 +01:00
Victor Stinner
919784737c
gh-117645: Increase WASI stack size from 512 KiB to 8 MiB (#117674)
Increase also the initial memory from 10 MiB to 20 MiB.

Reenable test_dynamic on WASI build.
2024-04-16 23:26:54 +02:00
Jeff Glass
acf69e09c6
gh-115178: Add Counts of UOp Pairs to pystats (GH-115181) 2024-04-16 14:27:18 +01:00
Sam Gross
25f6ff5d3e
gh-117649: Raise ImportError for unsupported modules in free-threaded build (#117651)
The free-threaded build does not currently support the combination of
single-phase init modules and non-isolated subinterpreters. Ensure that
`check_multi_interp_extensions` is always `True` for subinterpreters in
the free-threaded build so that importing these modules raises an
`ImportError`.
2024-04-11 15:00:54 -04:00
mpage
df73179048
gh-111926: Make weakrefs thread-safe in free-threaded builds (#117168)
Most mutable data is protected by a striped lock that is keyed on the
referenced object's address. The weakref's hash is protected using the
weakref's per-object lock.
 
Note that this only affects free-threaded builds. Apart from some minor
refactoring, the added code is all either gated by `ifdef`s or is a no-op
(e.g. `Py_BEGIN_CRITICAL_SECTION`).
2024-04-08 10:58:38 -04:00
Steve Dower
687616877b
gh-111140: PyLong_From/AsNativeBytes: Take *flags* rather than just *endianness* (GH-116053) 2024-04-05 16:21:16 +02:00
Guido van Rossum
060a96f1a9
gh-116968: Reimplement Tier 2 counters (#117144)
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.

The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
Mark Shannon
c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
Irit Katriel
1d5479b236
gh-117411: move PyFutureFeatures to pycore_symtable.h and make it private (#117412) 2024-04-02 10:34:49 +00:00
Mark Shannon
bf82f77957
GH-116422: Tier2 hot/cold splitting (GH-116813)
Splits the "cold" path, deopts and exits, from the "hot" path, reducing the size of most jitted instructions, at the cost of slower exits.
2024-03-26 09:35:11 +00:00