Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-11-01 06:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity
cpython/Tools/cases_generator
History
Guido van Rossum 2028a4f6d9
gh-106290: Fix edge cases around uops (#106319) 
- Tweak uops debugging output
- Fix the bug from gh-106290
- Rename `SET_IP` to `SAVE_IP` (per https://github.com/faster-cpython/ideas/issues/558)
- Add a `SAVE_IP` uop at the start of the trace (ditto)
- Allow `unbound_local_error`; this gives us uops for `LOAD_FAST_CHECK`, `LOAD_CLOSURE`, and `DELETE_FAST`
- Longer traces
- Support `STORE_FAST_LOAD_FAST`, `STORE_FAST_STORE_FAST`
- Add deps on pycore_uops.h to Makefile(.pre.in)
2023-07-03 20:05:11 +00:00
..
generate_cases.py gh-106290: Fix edge cases around uops (#106319) 2023-07-03 20:05:11 +00:00
interpreter_definition.md Update DSL docs for cases generator (#105753) 2023-06-14 08:19:24 -07:00
lexer.py gh-106200: Remove unused imports (#106201) 2023-06-28 11:55:41 +00:00
parser.py Remove support for legacy bytecode instructions (#105705) 2023-06-12 18:19:04 +00:00
plexer.py GH-98831: Refactor and fix cases generator (#99526) 2022-11-17 17:06:07 -08:00
README.md gh-104584: Baby steps towards generating and executing traces (#105924) 2023-06-26 19:02:57 -07:00
test_generator.py gh-104584: Baby steps towards generating and executing traces (#105924) 2023-06-26 19:02:57 -07:00

README.md

Tooling to generate interpreters

Documentation for the instruction definitions in Python/bytecodes.c ("the DSL") is here.

What's currently here:

  • lexer.py: lexer for C, originally written by Mark Shannon
  • plexer.py: OO interface on top of lexer.py; main class: PLexer
  • parser.py: Parser for instruction definition DSL; main class Parser
  • generate_cases.py: driver script to read Python/bytecodes.c and write Python/generated_cases.c.h (and several other files)
  • test_generator.py: tests, require manual running using pytest

Note that there is some dummy C code at the top and bottom of Python/bytecodes.c to fool text editors like VS Code into believing this is valid C code.

A bit about the parser

The parser class uses a pretty standard recursive descent scheme, but with unlimited backtracking. The PLexer class tokenizes the entire input before parsing starts. We do not run the C preprocessor. Each parsing method returns either an AST node (a Node instance) or None, or raises SyntaxError (showing the error in the C source).

Most parsing methods are decorated with @contextual, which automatically resets the tokenizer input position when None is returned. Parsing methods may also raise SyntaxError, which is irrecoverable. When a parsing method returns None, it is possible that after backtracking a different parsing method returns a valid AST.

Neither the lexer nor the parsers are complete or fully correct. Most known issues are tersely indicated by # TODO: comments. We plan to fix issues as they become relevant.