Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-12-08 06:10:17 +00:00
Code Issues Projects Releases Packages Wiki Activity

[3.14] gh-140576: Fixed crash produced by lexer in case of dedented zero byte (GH-140583) (#140757)

Browse source 
gh-140576: Fixed crash produced by lexer in case of dedented zero byte (GH-140583)
(cherry picked from commit 8706167474)

Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com>
This commit is contained in:
Miss Islington (bot) 2025-10-29 14:54:37 +01:00 • committed by GitHub
parent e0f54a0bda
commit c45ab05a3a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
Lib/test/test_tokenize.py
View file

@ -3183,6 +3183,7 @@ def get_tokens(string):
f'__{ f'__{
x:d x:d
}__'""", }__'""",
" a\n\x00",
]: ]:
with self.subTest(case=case): with self.subTest(case=case):
self.assertRaises(tokenize.TokenError, get_tokens, case) self.assertRaises(tokenize.TokenError, get_tokens, case)

2
Misc/NEWS.d/next/Core_and_Builtins/2025-10-25-17-36-46.gh-issue-140576.kj0SCY.rst Normal file
View file

@ -0,0 +1,2 @@
Fixed crash in :func:`tokenize.generate_tokens` in case of
specific incorrect input. Patch by Mikhail Efimov.

3
Parser/lexer/lexer.c
View file

@ -539,6 +539,9 @@ tok_get_normal_mode(struct tok_state *tok, tokenizer_mode* current_tok, struct t
return MAKE_TOKEN(ERRORTOKEN); return MAKE_TOKEN(ERRORTOKEN);
} }
} }
else if (c == EOF && PyErr_Occurred()) {
return MAKE_TOKEN(ERRORTOKEN);
}
else { else {
break; break;
} }