cpython/Parser/tokenizer
Miss Islington (bot) 9ff705c6c5
[3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898)
* Support non-UTF-8 shebang and comments if non-UTF-8 encoding is specified.
* Detect decoding error in comments for UTF-8 encoding.
* Include the decoding error position for default encoding in SyntaxError.
(cherry picked from commit 5c942f11cd)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-10-19 19:16:33 +00:00
..
file_tokenizer.c [3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898) 2025-10-19 19:16:33 +00:00
helpers.c [3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898) 2025-10-19 19:16:33 +00:00
helpers.h [3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898) 2025-10-19 19:16:33 +00:00
readline_tokenizer.c [3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898) 2025-10-19 19:16:33 +00:00
string_tokenizer.c [3.14] gh-63161: Fix PEP 263 support (GH-139481) (GH-139898) 2025-10-19 19:16:33 +00:00
tokenizer.h gh-104169: Refactor tokenizer into lexer and wrappers (#110684) 2023-10-11 15:14:44 +00:00
utf8_tokenizer.c gh-104169: Refactor tokenizer into lexer and wrappers (#110684) 2023-10-11 15:14:44 +00:00