Commit graph

1325 commits

Author SHA1 Message Date
Miss Islington (bot)
9ff705c6c5
[3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898)
* Support non-UTF-8 shebang and comments if non-UTF-8 encoding is specified.
* Detect decoding error in comments for UTF-8 encoding.
* Include the decoding error position for default encoding in SyntaxError.
(cherry picked from commit 5c942f11cd)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-10-19 19:16:33 +00:00
Miss Islington (bot)
de84d09050
[3.14] gh-139516: Fix lambda colon start format spec in f-string in tokenizer (GH-139657) (#139701)
gh-139516: Fix lambda colon start format spec in f-string in tokenizer (GH-139657)
(cherry picked from commit 539461d9ec)

Co-authored-by: Tomasz Pytel <tompytel@gmail.com>
2025-10-07 18:35:17 +01:00
Miss Islington (bot)
7b56d08219
[3.14] gh-137078: Fix keyword typo recognition when executed over files (GH-137079) (#137826) 2025-08-15 18:22:17 +01:00
Miss Islington (bot)
cf57f4fac3
[3.14] gh-137314: Fix incorrect treatment of format specs in raw fstrings (GH-137328) (#137344)
gh-137314: Fix incorrect treatment of format specs in raw fstrings (GH-137328)
(cherry picked from commit 0153d82a5a)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2025-08-03 16:36:58 +00:00
Lysandros Nikolaou
af8d1f56f5
[3.14] gh-132661: Disallow Template/str concatenation after PEP 750 spec update (#135996) (#136901)
Co-authored-by: Dave Peck <davepeck@gmail.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
2025-07-21 17:34:54 +02:00
Miss Islington (bot)
9bf3464852
[3.14] gh-135148: Correctly handle f/t strings with comments and debug expressions (GH-135198) (#136720) 2025-07-17 00:10:46 +02:00
Miss Islington (bot)
2045453171
[3.14] gh-129958: New syntax error in format spec applies to both f-strings and t-strings (GH-135570) (#135982)
(cherry picked from commit fb9e292919)

Co-authored-by: Dylan <dylwil3@gmail.com>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2025-06-26 18:40:09 +02:00
Miss Islington (bot)
7499bcd530
[3.14] gh-135496: Fix f string exclamation mark error typo (GH-135495) (#135499)
gh-135496: Fix f string exclamation mark error typo (GH-135495)
(cherry picked from commit c2bb3f9843)

Co-authored-by: GiGaGon <107241144+MeGaGiGaGon@users.noreply.github.com>
2025-06-14 05:05:55 +00:00
Miss Islington (bot)
77c4b5d413
[3.14] gh-130077: Properly match full soft keywords in the parser (GH-135317) (#135348)
gh-130077: Properly match full soft keywords in the parser (GH-135317)
(cherry picked from commit ff2b5f40c2)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2025-06-11 02:01:37 +01:00
Miss Islington (bot)
904685f90f
[3.14] gh-133157: remove usage of _Py_NO_SANITIZE_UNDEFINED in Parser/pegen.c (GH-134048) (#135320)
gh-133157: remove usage of `_Py_NO_SANITIZE_UNDEFINED` in `Parser/pegen.c` (GH-134048)
(cherry picked from commit 754e7c9b51)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-06-10 00:34:49 +00:00
Victor Stinner
3d69d18322
[3.14] gh-133968: Add PyUnicodeWriter_WriteASCII() function (#133973) (#134974)
gh-133968: Add PyUnicodeWriter_WriteASCII() function (#133973)

Replace most PyUnicodeWriter_WriteUTF8() calls with
PyUnicodeWriter_WriteASCII().

(cherry picked from commit f49a07b531)

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-06-09 14:37:26 +02:00
Miss Islington (bot)
89df01bd27
[3.14] gh-135028: Increase parser MAXSTACK for nested parenthesis (GH-135031) (#135059)
gh-135028: Increase parser MAXSTACK for nested parenthesis (GH-135031)
(cherry picked from commit 6e80f11eb5)

Co-authored-by: Victor Stinner <vstinner@python.org>
2025-06-03 12:23:06 +00:00
Miss Islington (bot)
d9c08c57e9
[3.14] gh-133999: Fix except parsing regression in 3.14 (GH-134035) (#134206)
gh-133999: Fix `except` parsing regression in 3.14 (GH-134035)
(cherry picked from commit 84914ad0e5)

Co-authored-by: sobolevn <mail@sobolevn.me>
2025-05-19 14:26:27 +03:00
Miss Islington (bot)
69b4387f78
[3.14] gh-133767: Fix use-after-free in the unicode-escape decoder with an error handler (GH-129648) (GH-133942)
If the error handler is used, a new bytes object is created to set as
the object attribute of UnicodeDecodeError, and that bytes object then
replaces the original data. A pointer to the decoded data will became invalid
after destroying that temporary bytes object. So we need other way to return
the first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal().

_PyBytes_DecodeEscape() does not have such issue, because it does not
use the error handlers registry, but it should be changed for compatibility
with _PyUnicode_DecodeUnicodeEscapeInternal().
(cherry picked from commit 9f69a58623)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-05-13 16:25:08 +03:00
Miss Islington (bot)
856e5903ba
[3.14] gh-133783: Fix __replace__ on AST nodes for optional attributes (GH-133797) (#133842)
gh-133783: Fix __replace__ on AST nodes for optional attributes (GH-133797)
(cherry picked from commit 7dddb4e667)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2025-05-10 16:44:07 +00:00
Miss Islington (bot)
54c3aa1597
[3.14] gh-133516: Raise ValueError when constants True, False or None are used as an identifier after NFKC normalization (GH-133523) (#133596) 2025-05-07 22:26:06 +00:00
Jelle Zijlstra
483d130e50
gh-131421: Fix ASDL tests (#133408)
PR #131419 broke this, but we failed to run tests on the PR due to a bug
in our script.
2025-05-04 23:46:21 +00:00
Samuel
30840706b0
gh-131421: fix ASDL grammar for Dict to have an expr?* keys field (#131419)
In the `ast` documentation for Python:

* https://docs.python.org/3/library/ast.html#ast.Dict
it is made clear that:

> When doing dictionary unpacking using dictionary literals the expression to be expanded goes in the values list, with a `None` at the corresponding position in `keys`.

Hence, `keys` is really a `expr?*` and *not* a `expr*`.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2025-05-04 16:03:38 -07:00
sobolevn
ca0a96dfaa
gh-133194: Fix regression with PEP 758 parsing on older feature_version (#133289)
gh-133192: Fix regression with PEP 758 parsing on older `feature_version`
2025-05-03 10:33:14 +03:00
sobolevn
1e9cc3d502
gh-133197: Improve error message for incompatible string / bytes prefixes (#133242) 2025-05-02 14:28:17 +03:00
sobolevn
a6ddd078d0
gh-123539: Improve SyntaxError msg for import as with not a name (#123629) 2025-05-02 08:34:13 +00:00
sobolevn
5cdd49b3f4
gh-133196: Guard PEP 750 grammar with CHECK_VERSION (#133225) 2025-05-01 09:17:07 +02:00
sobolevn
b451516aa6
gh-133197: Improve error message for ft"" and bt"" cases (#133202) 2025-04-30 19:20:44 +03:00
sobolevn
b1f893875b
gh-133194: Add CHECK_VERSION to new PEP758 grammar (#133195) 2025-04-30 13:39:26 +03:00
Lysandros Nikolaou
60202609a2
gh-132661: Implement PEP 750 (#132662)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Wingy <git@wingysam.xyz>
Co-authored-by: Koudai Aono <koxudaxi@gmail.com>
Co-authored-by: Dave Peck <davepeck@gmail.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Paul Everitt <pauleveritt@me.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
2025-04-30 11:46:41 +02:00
Steele Farnsworth
99b71efe8e
gh-129858: Special syntax error for elif block after else (#129902) 2025-04-25 01:25:48 +00:00
Pablo Galindo Salgado
bf3a0a1c0f
gh-132449: Improve syntax error messages for keywords with typos (#132450)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2025-04-22 11:01:55 +02:00
sobolevn
ea8ec95cfa
gh-132769: Refactor possible read-out-of-bounds in lexer.c (#132770) 2025-04-21 18:48:48 +01:00
Pablo Galindo Salgado
2f8b08da47
gh-129958: Properly disallow newlines in format specs in single-quoted f-strings (GH-130063) 2025-04-18 14:30:04 +02:00
Chris Eibl
e94d168473
GH-131296: fix clang-cl warning on Windows in pegen.h (#131584) 2025-04-15 17:01:42 +01:00
Pablo Galindo Salgado
c2ac662f28
gh-131831: Implement PEP 758 – Allow except and except* expressions without parentheses (#131833) 2025-04-01 19:04:56 +00:00
Victor Stinner
3796884528
gh-111178: Skip undefined behavior checks in _PyPegen_lookahead() (#131714)
For example, expression_rule() return type is 'expr_ty', whereas
_PyPegen_lookahead() uses 'void*'.
2025-03-27 10:03:58 +01:00
rialbat
2c686a9ac2
gh-131762: Fixed dereferencing the pointer 'parser_token->metadata' with a NULL value (#131764) 2025-03-26 18:44:56 +00:00
Victor Stinner
b69da006a4
gh-131238: Remove includes from pycore_interp.h (#131495)
Remove also now unused includes in C files.
2025-03-20 11:35:23 +00:00
Victor Stinner
344f3c3fd4
gh-131238: Remove pycore_lock.h includes (#131483)
PyMutex type is now part of <Python.h>, it's no longer needed to
include <pycore_lock.h> to get it.
2025-03-19 23:46:25 +00:00
Victor Stinner
4b54031323
gh-131238: Remove pycore_runtime.h from pycore_pystate.h (#131356)
* Remove includes from pycore_pystate.h:

  * pycore_runtime_structs.h
  * pycore_runtime.h
  * pycore_tstate.h
  * pycore_interp.h

* Reorganize internal headers. Move _gc_thread_state from
  pycore_interp_structs.h to pycore_tstate.h.
* Add 3 new header files to PCbuild/pythoncore.vcxproj.
2025-03-19 17:33:24 +01:00
Mark Shannon
a45f25361d
GH-131238: More refactoring of core header files (GH-131351)
Adds new pycore_stats.h header file to help break dependencies involving the pycore_code.h header.
2025-03-17 14:41:05 +00:00
Victor Stinner
978e37bb5f
gh-131238: Add explicit includes to pycore headers (#131257) 2025-03-17 12:32:43 +01:00
Mark Shannon
a1aeec61c4
GH-131238: Core header refactor (GH-131250)
* Moves most structs in pycore_ header files into pycore_structs.h and pycore_runtime_structs.h

* Removes many cross-header dependencies
2025-03-17 09:19:04 +00:00
Sergey Miryanov
3a7f17c7e2
gh-130790: Remove references about unicode's readiness from comments (#130801) 2025-03-03 19:18:09 +00:00
Hugo Beauzée-Luyssen
214562ed4d
gh-130740: Move some stdbool.h includes after Python.h (#130738)
Move some `#include <stdbool.h>` after `#include "Python.h"` when `pyconfig.h` is not
included first and when we are in a platform-agnostic context. This is to avoid having
features defined by `stdbool.h` before those decided by `Python.h`.
2025-03-02 09:56:49 +00:00
Pablo Galindo Salgado
e06bebb87e
gh-130618: Fix parser error when using lambdas inside f-strings (#130638) 2025-02-27 15:51:17 +00:00
Mark Shannon
014223649c
GH-130396: Use computed stack limits on linux (GH-130398)
* Implement C recursion protection with limit pointers for Linux, MacOS and Windows

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-25 09:24:48 +00:00
Petr Viktorin
ef29104f7d
GH-91079: Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now (GH130413)
Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now

Unfortunatlely, the change broke some buildbots.

This reverts commit 2498c22fa0.
2025-02-24 11:16:08 +01:00
Mark Shannon
2498c22fa0
GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)
* Implement C recursion protection with limit pointers

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-19 11:44:57 +00:00
Sergey Miryanov
bcc9a5dddb
gh-129515: Clarify syntax error messages for conditional expressions (#129880)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-02-18 21:43:19 +00:00
Irit Katriel
c9b1bf302c
gh-130139: always check ast node type in ast.parse() with ast input (#130140) 2025-02-16 13:32:39 +00:00
Pablo Galindo Salgado
3bd3e09588
gh-125331: Allow the parser to activate future imports on the fly (#125482) 2025-02-14 04:54:56 +00:00
Pablo Galindo Salgado
6fb5138776
gh-88535: Improve syntax error for wrongly closed strings (#26633) 2025-02-13 01:30:20 +00:00
Pablo Galindo Salgado
56eda25633
gh-116042: Fix location for SyntaxErrors of invalid escapes in the tokenizer (#116049) 2025-02-13 01:07:37 +00:00