* [3.13] gh-130077: Properly match full soft keywords in the parser (GH-135317)
(cherry picked from commit ff2b5f40c2)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* Remove line the main-branch commit removed
---------
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-137314: Fix incorrect treatment of format specs in raw fstrings (GH-137328)
(cherry picked from commit 0153d82a5a)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
If the error handler is used, a new bytes object is created to set as
the object attribute of UnicodeDecodeError, and that bytes object then
replaces the original data. A pointer to the decoded data will became invalid
after destroying that temporary bytes object. So we need other way to return
the first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal().
_PyBytes_DecodeEscape() does not have such issue, because it does not
use the error handlers registry, but it should be changed for compatibility
with _PyUnicode_DecodeUnicodeEscapeInternal().
(cherry picked from commit 9f69a58623)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-132769: Refactor possible read-out-of-bounds in `lexer.c` (GH-132770)
(cherry picked from commit ea8ec95cfa)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-131762: Fixed dereferencing the pointer 'parser_token->metadata' with a NULL value (GH-131764)
(cherry picked from commit 2c686a9ac2)
Co-authored-by: rialbat <47256826+rialbat@users.noreply.github.com>
gh-125331: Allow the parser to activate future imports on the fly (GH-125482)
(cherry picked from commit 3bd3e09588)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-130740: Move some `stdbool.h` includes after `Python.h` (#130738)
Move some `#include <stdbool.h>` after `#include "Python.h"` when `pyconfig.h` is not
included first and when we are in a platform-agnostic context. This is to avoid having
features defined by `stdbool.h` before those decided by `Python.h` (this caused some
build failures when compiling CPython with `zig cc`).
(cherry-picked from commit 214562ed4d)
---------
Co-authored-by: Hugo Beauzée-Luyssen <hugo@beauzee.fr>
gh-124363: Treat debug expressions in f-string as raw strings (GH-128399)
(cherry picked from commit 60a3a0dd6f)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
gh-129093: Fix f-string debug text sometimes getting cut off when expression contains `!` (GH-129159)
(cherry picked from commit 767cf70844)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (GH-126241)
check return value of `_Py_asdl_expr_seq_new`
(cherry picked from commit 94639f6b71)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-126105: Fix crash in `ast` module, when `._fields` is deleted (GH-126115)
Previously, if the `ast.AST._fields` attribute was deleted, attempts to create a new `as`t node would crash due to the assumption that `_fields` always had a non-NULL value. Now it has been fixed by adding an extra check to ensure that `_fields` does not have a NULL value (this can happen when you manually remove `_fields` attribute).
(cherry picked from commit b2eaa75b17)
Co-authored-by: sobolevn <mail@sobolevn.me>
When formatting the AST as a string, infinite values are replaced by
1e309, which evaluates to infinity. The initialization of this string
replacement was not thread-safe in the free threading build.
(cherry picked from commit 427dcf24de)
gh-124064: Fix -Wconversion warnings in Parser/string_parser.c (GH-124204)
Fix integer overflow check in decode_unicode_with_escapes(): use
PY_SSIZE_T_MAX instead of SIZE_MAX.
(cherry picked from commit f9fa6ba4f8)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-123321: Make Parser/myreadline.c locking safe in free-threaded build (GH-123690)
Use a `PyMutex` to avoid the race in mutex initialization. Use relaxed
atomics to avoid the data race on reading `_PyOS_ReadlineTState` when
checking for re-entrant calls.
(cherry picked from commit 0c080d7c77)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-123321: Fix Parser/myreadline.c to prevent a segfault during a multi-threaded race (GH-123323)
(cherry picked from commit a4562fedad)
Co-authored-by: Bar Harel <bharel@barharel.com>
gh-123229: Fix valgrind warning by initializing the f-string buffers to 0 in the tokenizer (GH-123263)
(cherry picked from commit adc5190014)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
- gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)
(cherry picked from commit ce1064e4c9)
- gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)
(cherry picked from commit ac61d58db0)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
- Cache line object to avoid creating a Unicode object
for all of the tokens in the same line.
- Speed up byte offset to column offset conversion by using the
smallest buffer possible to measure the difference.
(cherry picked from commit d87b015106)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Now, such classes will no longer require changes in Python 3.13 in the normal case.
The test suite for robotframework passes with no DeprecationWarnings under this PR.
I also added a new DeprecationWarning for the case where `_field_types` exists
but is incomplete, since that seems likely to indicate a user mistake.
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.
PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.
A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
