Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2026-01-06 15:32:22 +00:00
Code Issues Projects Releases Packages Wiki Activity

[3.13] gh-127303: Add docs for token.EXACT_TOKEN_TYPES (GH-127304) (#127390)

Browse source 
gh-127303: Add docs for token.EXACT_TOKEN_TYPES (GH-127304)

---------

(cherry picked from commit dd3a87d2a8)

Co-authored-by: Илья Любавский <100635212+lubaskinc0de@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
This commit is contained in:
Miss Islington (bot) 2024-11-30 09:13:13 +01:00 • committed by GitHub
parent 2f56c68dec
commit 4cba0e66c2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 13 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

7
Doc/library/token.rst
View file

@ -79,6 +79,13 @@ the :mod:`tokenize` module.
``type_comments=True``.
.. data:: EXACT_TOKEN_TYPES
A dictionary mapping the string representation of a token to its numeric code.
.. versionadded:: 3.8
.. versionchanged:: 3.5
Added :data:`!AWAIT` and :data:`!ASYNC` tokens.

3
Lib/token.py generated
View file

@ -1,7 +1,8 @@
"""Token constants."""
# Auto-generated by Tools/build/generate_token.py
__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF']
__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF',
'EXACT_TOKEN_TYPES']
ENDMARKER = 0
NAME = 1

1
Misc/ACKS
View file

@ -1149,6 +1149,7 @@ Mark Lutz
Taras Lyapun
Jim Lynch
Mikael Lyngvig
Ilya Lyubavski
Jeff MacDonald
John Machin
Andrew I MacIntyre

1
Misc/NEWS.d/next/Library/2024-11-27-16-06-10.gh-issue-127303.asqkgh.rst Normal file
View file

@ -0,0 +1 @@
Publicly expose :data:`~token.EXACT_TOKEN_TYPES` in :attr:`!token.__all__`.

3
Tools/build/generate_token.py
View file

@ -226,7 +226,8 @@ def make_rst(infile, outfile='Doc/library/token-list.inc'):
# {AUTO_GENERATED_BY_SCRIPT}
'''
token_py_template += '''
__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF']
__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF',
'EXACT_TOKEN_TYPES']
%s
N_TOKENS = %d