cpython/Parser/tokenizer
Serhiy Storchaka 5c942f11cd
gh-63161: Fix PEP 263 support (GH-139481)
* Support non-UTF-8 shebang and comments if non-UTF-8 encoding is specified.
* Detect decoding error in comments for UTF-8 encoding.
* Include the decoding error position for default encoding in SyntaxError.
2025-10-10 12:51:19 +00:00
..
file_tokenizer.c gh-63161: Fix PEP 263 support (GH-139481) 2025-10-10 12:51:19 +00:00
helpers.c gh-63161: Fix PEP 263 support (GH-139481) 2025-10-10 12:51:19 +00:00
helpers.h gh-63161: Fix PEP 263 support (GH-139481) 2025-10-10 12:51:19 +00:00
readline_tokenizer.c gh-63161: Fix PEP 263 support (GH-139481) 2025-10-10 12:51:19 +00:00
string_tokenizer.c gh-63161: Fix PEP 263 support (GH-139481) 2025-10-10 12:51:19 +00:00
tokenizer.h gh-104169: Refactor tokenizer into lexer and wrappers (#110684) 2023-10-11 15:14:44 +00:00
utf8_tokenizer.c gh-104169: Refactor tokenizer into lexer and wrappers (#110684) 2023-10-11 15:14:44 +00:00