Pablo Galindo Salgado
47c5a0f307
gh-125588: Allow to regenerate the parser with Python < 3.12 ( #127969 )
...
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-12-15 23:17:01 +00:00
Pablo Galindo Salgado
9dfef4e5f4
gh-125588: Teach the python PEG generator the new f-string tokens ( #125589 )
...
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-10-29 23:40:12 +00:00
Mikhail Efimov
c1bdbe84c8
gh-124889: Rework Python generator cache ( #125816 )
2024-10-22 09:42:56 +01:00
efimov-mikhail
1f9025a4e7
gh-124889: Remove redundant artificial rules in PEG parser ( #124893 )
...
Cache in C PEG-generator reworked:
we save artificial rules in cache by Node string representation as a key instead of Node object itself.
As a result total count of artificial rules in parsers.c is lowered from 283 to 170.
More natural number ordering is used for the names of artificial rules.
Auxiliary method CCallMakerVisitor._generate_artificial_rule_call is added.
Its purpose is abstracting work with artificial rules cache.
Explicit using of "is_repeat1" kwarg is added to visit_Repeat0 and visit_Repeat1 methods.
Its slightly improve code readabitily.
2024-10-03 13:58:56 +01:00
Serhiy Storchaka
6c09b8de5c
gh-122270: Fix typos in the Py_DEBUG macro name (GH-122271)
2024-07-25 14:04:22 +03:00
Lysandros Nikolaou
348184845a
gh-120956: Avoid comparison of int to Py_ssize_t in parser ( #120959 )
2024-06-24 18:13:02 +02:00
yf-yang
ace2045ea6
Fix types in pegen parser generator (GH-120720)
2024-06-19 16:12:40 +02:00
Nikita Sobolev
1e5f615086
gh-116991: Improve pygen --help for python subparser ( #116992 )
2024-06-03 10:52:35 +03:00
Petr Viktorin
48f21b3631
gh-118235: Move RAISE_SYNTAX_ERROR actions to invalid rules and make sure they stay there (GH-119731)
...
The Full Grammar specification in the docs omits rule actions, so grammar rules that raise a syntax error looked like valid syntax.
This was solved in ef940de
2024-05-30 09:27:32 +02:00
Xie Yanbo
bf08f0a5fe
Fix typos in comments ( #119645 )
2024-05-28 09:53:32 +02:00
trag1c
c4f9823be2
gh-118671: Updated dead ActiveState links ( #118730 )
...
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-05-08 01:06:38 -06:00
David Rubin
9b280ab0ab
gh-116988: Remove duplicates of annotated_rhs in the Grammar ( #117004 )
2024-04-24 18:16:06 +01:00
Sam Gross
5f1997896d
gh-112984: Fix link error on free-threaded Windows build (GH-114455)
...
The test_peg_generator test tried to link the python313_d.lib library,
which failed because the library is now named python313t_d.lib. The
underlying problem is that the "compiler" attribute was not set when
we call get_libraries() from distutils.
2024-01-23 18:05:15 +00:00
Serhiy Storchaka
b3d2427f22
gh-58032: Do not use argparse.FileType in module CLIs and scripts (GH-113649)
...
Open and close files manually. It prevents from leaking files,
preliminary creation of output files, and accidental closing of stdin
and stdout.
2024-01-10 15:07:19 +02:00
Donghee Na
e5cce70df7
gh-112532: Fix peg generator build.py for mimalloc build (gh-113492)
...
gh-112532: Fix peg generator for mimalloc build
2023-12-27 02:54:16 +09:00
Steve Dower
79dad03747
gh-111650: Ensure pyconfig.h includes Py_GIL_DISABLED on Windows (GH-112778)
2023-12-13 15:38:45 +00:00
Brett Cannon
56e59a49ae
GH-111807: Lower the parser stack depth under WASI debug builds ( #112225 )
2023-11-20 13:27:33 +00:00
Pablo Galindo Salgado
8316769d83
gh-110805: Fix test_peg_generator after the change in the parser for REPL tracebacks ( #110814 )
...
Fix test_peg_generator after the change in the parser for REPL tracebacks
2023-10-13 13:04:23 +02:00
Lysandros Nikolaou
17d65547df
gh-104169: Fix test_peg_generator after tokenizer refactoring ( #110727 )
...
* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
2023-10-12 09:34:35 +02:00
Pablo Galindo Salgado
b28ffaa193
gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords ( #109606 )
2023-09-22 19:03:23 +01:00
Alex Waygood
697c9dcf8f
gh-108455: peg_generator: enable mypy's --warn-unreachable setting and redundant-expr error code ( #109160 )
2023-09-08 22:05:40 +01:00
Alex Waygood
b89b838ebc
gh-108455: peg_generator: use types-setuptools==68.1.0.1 in CI ( #108697 )
2023-08-31 08:41:25 +01:00
Alex Waygood
77e8f233ac
gh-108455: peg_generator: install two stubs packages before running mypy ( #108637 )
2023-08-29 20:14:08 +01:00
Nikita Sobolev
f8be2e262c
gh-108455: peg generator: Use strict_optional=True for grammar_parser ( #108629 )
2023-08-29 13:02:52 +01:00
Alex Waygood
0d140b8c5e
gh-108455: peg_generator: make the mypy config slightly stricter ( #108627 )
...
* Enable `--no-implicit-reexport`
* Enable the `truthy-bool` error code
* Enable the `ignore-without-code` error code
* Explicitly note that `--warn-unreachable` cannot yet be enabled
2023-08-29 11:23:22 +00:00
Alex Waygood
c879b9fddf
gh-108455: Simplify the peg_generator mypy config ( #108620 )
...
make it easier to see exactly which options from mypy's `--strict` mode can't currently be enabled
2023-08-29 10:24:06 +00:00
Nikita Sobolev
cf7ba83eb2
gh-108455: Run mypy on Tools/peg_generator ( #108456 )
...
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-08-28 21:04:12 +01:00
Dennis Sweeney
86617518c4
gh-108179: Add error message for parser stack overflows ( #108256 )
2023-08-22 08:41:50 +01:00
Pablo Galindo Salgado
da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer ( #107018 )
2023-07-26 16:34:15 +01:00
Lysandros Nikolaou
a241003d04
gh-105191: Cleanup peg generator; keep only necessary files ( #105197 )
2023-06-01 17:24:15 +02:00
Lysandros Nikolaou
afa759fb80
gh-100238: Use setuptools in peg-generator and reenable tests ( #104798 )
2023-05-26 15:32:03 +02:00
Victor Stinner
ae00b810d1
gh-104780: Remove 2to3 program and lib2to3 module ( #104781 )
...
* Remove the Tools/scripts/2to3 script.
* Remove the Lib/test/test_lib2to3/ directory.
* Doc/tools/extensions/pyspecific.py: remove the "2to3fixer" object
type.
* Makefile and PC/layout/main.py no longer compile lib2to3 grammar
files.
* Update Makefile for 2to3 removal.
2023-05-23 19:40:02 +02:00
Pablo Galindo Salgado
1ef61cf71a
gh-102856: Initial implementation of PEP 701 ( #102855 )
...
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez MacÃas <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-04-19 11:18:16 -05:00
Pablo Galindo Salgado
f533f216e6
gh-102416: Do not memoize incorrectly loop rules in the parser ( #102467 )
2023-03-06 14:41:53 +01:00
Pablo Galindo Salgado
1de4395f62
gh-101046: Fix a potential memory leak in the parser when raising MemoryError ( #101051 )
2023-01-16 18:45:37 +00:00
Victor Stinner
4ce2a202c7
gh-99300: Use Py_NewRef() in Parser/ directory ( #99330 )
...
Replace Py_INCREF() with Py_NewRef() in C files of the Parser/
directory and in the PEG generator.
2022-11-10 15:30:05 +01:00
Victor Stinner
47e35625ff
gh-84623: Remove unused imports ( #94132 )
2022-06-22 19:14:27 +02:00
Victor Stinner
616fa3465d
gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ ( #94049 )
...
* Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/.
* Remove Lib/test/test_lib2to3.py.
* Update imports.
* all_project_files(): use different paths and sort files
to make the tests more reproducible.
* Update references to tests.
2022-06-21 15:21:22 +02:00
Victor Stinner
5115a16831
gh-93103: Parser uses PyConfig.parser_debug instead of Py_DebugFlag ( #93106 )
...
* Replace deprecated Py_DebugFlag with PyConfig.parser_debug in the
parser.
* Add Parser.debug member.
* Add tok_state.debug member.
* Py_FrozenMain(): Replace Py_VerboseFlag with PyConfig.verbose.
2022-05-24 22:35:08 +02:00
Christian Heimes
137fd3d88a
gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803)
2022-05-19 12:43:16 +02:00
Victor Stinner
804f2529d8
gh-91320: Use _PyCFunction_CAST() ( #92251 )
...
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
Change generated by the command:
sed -i -e \
's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \
$(find -name "*.c")
2022-05-03 21:42:14 +02:00
Jeremy Kloth
612e422c6e
bpo-46576: Speed up test_peg_generator by using a static library for shared sources (GH-32338)
...
Speed up test_peg_generator by using a static library for shared sources to avoid recompiling as much code.
2022-04-06 14:55:58 -07:00
Luca Chiodini
7f07b5ee9c
Update grammar_grapher with the new forced (&&) directive ( #31704 )
2022-03-06 15:26:28 +01:00
Pablo Galindo Salgado
390459de6d
Allow the parser to avoid nested processing of invalid rules (GH-31252)
2022-02-10 13:12:14 +00:00
Gregory P. Smith
164a017e13
bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. ( #31015 )
...
Disable compiler optimization within test_peg_generator.
This speed up test_peg_generator by always disabling compiler
optimizations by using -O0 or equivalent when the test is building its
own C extensions.
A build not using --with-pydebug in order to speed up test execution
winds up with this test taking a very long time as it would do
repeated compilation of parser C code using the same optimization
flags as CPython was built with.
This speeds the test up 6-8x on gps-raspbian.
Also incorporate's #31017 's win32 conditional and flags.
Co-authored-by: Kumar Aditya kumaraditya303
2022-02-02 12:15:16 -08:00
Pablo Galindo Salgado
dd6c35761a
bpo-46110: Restore commit e9898bf153
...
This restores commit e9898bf153
2022-01-03 19:54:06 +00:00
Pablo Galindo Salgado
9d35dedc5e
Revert "bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177)" (GH-30363)
...
This reverts commit e9898bf153
2022-01-03 18:29:18 +00:00
Pablo Galindo Salgado
e9898bf153
bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177)
...
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2021-12-20 15:43:26 +00:00
Victor Stinner
253b7a0a9f
bpo-45866: pegen strips directory of "generated from" header (GH-29777)
...
"make regen-all" now produces the same output when run from a
directory other than the source tree: when building Python out of the
source tree.
2021-11-26 11:50:34 +01:00
Pablo Galindo Salgado
c9c4444d9f
Refactor parser compilation units into specific components (GH-29676)
2021-11-21 01:08:50 +00:00
