| 
									
										
										
										
											2008-03-19 05:04:44 +00:00
										 |  |  | # Copyright 2006 Google, Inc. All Rights Reserved. | 
					
						
							|  |  |  | # Licensed to PSF under a Contributor Agreement. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | """Export the Python grammar and symbols.""" | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # Python imports | 
					
						
							|  |  |  | import os | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # Local imports | 
					
						
							|  |  |  | from .pgen2 import token | 
					
						
							|  |  |  | from .pgen2 import driver | 
					
						
							|  |  |  | from . import pytree | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # The grammar file | 
					
						
							|  |  |  | _GRAMMAR_FILE = os.path.join(os.path.dirname(__file__), "Grammar.txt") | 
					
						
							| 
									
										
											  
											
												Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
  r83852 | benjamin.peterson | 2010-08-08 15:45:44 -0500 (Sun, 08 Aug 2010) | 1 line
  wrap with parens
........
  r83853 | benjamin.peterson | 2010-08-08 15:46:31 -0500 (Sun, 08 Aug 2010) | 1 line
  use parens
........
  r83857 | benjamin.peterson | 2010-08-08 15:59:49 -0500 (Sun, 08 Aug 2010) | 1 line
  things which use touch_import should be pre order
........
  r84042 | george.boutsioukis | 2010-08-14 16:10:19 -0500 (Sat, 14 Aug 2010) | 2 lines
  This revision incorporates into the 2to3 tool the new, faster, tree matching algorithm developed during a GSOC project. The algorithm resides in the two added modules, btm_matcher and btm_utils. New code has been added to drive the new matching process in refactor.py and a few minor changes were made in other modules. A BM_compatible flag(False by default) has been added in fixer_base and it is set to True in most of the current fixers.
........
  r84216 | benjamin.peterson | 2010-08-19 16:44:05 -0500 (Thu, 19 Aug 2010) | 1 line
  allow star_expr in testlist_gexp
........
  r84274 | benjamin.peterson | 2010-08-22 18:40:46 -0500 (Sun, 22 Aug 2010) | 1 line
  wrap long line
........
  r84275 | benjamin.peterson | 2010-08-22 18:42:22 -0500 (Sun, 22 Aug 2010) | 1 line
  cleanup
........
  r84276 | benjamin.peterson | 2010-08-22 18:51:01 -0500 (Sun, 22 Aug 2010) | 1 line
  when there's a None value and a traceback, don't call type with it #9661
........
  r84375 | george.boutsioukis | 2010-08-31 08:38:53 -0500 (Tue, 31 Aug 2010) | 3 lines
  Idiomatic code changes & stylistic issues fixed in the BottomMatcher module. Thanks to Benjamin Peterson for taking the time to review the code.
........
  r85388 | benjamin.peterson | 2010-10-12 17:27:44 -0500 (Tue, 12 Oct 2010) | 1 line
  fix urllib fixer with multiple as imports on a line #10069
........
  r85478 | benjamin.peterson | 2010-10-14 08:09:56 -0500 (Thu, 14 Oct 2010) | 1 line
  stop abusing docstrings
........
  r85506 | benjamin.peterson | 2010-10-14 17:45:19 -0500 (Thu, 14 Oct 2010) | 1 line
  kill sibling import
........
  r85507 | benjamin.peterson | 2010-10-14 17:54:15 -0500 (Thu, 14 Oct 2010) | 1 line
  remove trailing whitespace
........
  r85508 | benjamin.peterson | 2010-10-14 17:55:28 -0500 (Thu, 14 Oct 2010) | 1 line
  typo
........
											
										 
											2010-10-14 23:00:04 +00:00
										 |  |  | _PATTERN_GRAMMAR_FILE = os.path.join(os.path.dirname(__file__), | 
					
						
							|  |  |  |                                      "PatternGrammar.txt") | 
					
						
							| 
									
										
										
										
											2008-03-19 05:04:44 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | class Symbols(object): | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |     def __init__(self, grammar): | 
					
						
							|  |  |  |         """Initializer.
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |         Creates an attribute for each grammar symbol (nonterminal), | 
					
						
							|  |  |  |         whose value is the symbol's type (an int >= 256). | 
					
						
							|  |  |  |         """
 | 
					
						
							| 
									
										
										
										
											2008-03-19 05:33:36 +00:00
										 |  |  |         for name, symbol in grammar.symbol2number.items(): | 
					
						
							| 
									
										
										
										
											2008-03-19 05:04:44 +00:00
										 |  |  |             setattr(self, name, symbol) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | python_grammar = driver.load_grammar(_GRAMMAR_FILE) | 
					
						
							| 
									
										
											  
											
												Merged revisions 74114 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
................
  r74114 | benjamin.peterson | 2009-07-20 10:33:09 -0500 (Mon, 20 Jul 2009) | 110 lines
  Merged revisions 73771,73811,73840,73842,73848-73849,73861,73957-73960,73964-73969,73972-73974,73977,73981,73984,74065,74113 via svnmerge from
  svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
  ........
    r73771 | benjamin.peterson | 2009-07-02 10:56:55 -0500 (Thu, 02 Jul 2009) | 1 line
    force the imports fixer to be run after the import one #6400
  ........
    r73811 | benjamin.peterson | 2009-07-03 09:03:14 -0500 (Fri, 03 Jul 2009) | 1 line
    check for sep, not pathsep when looking for a subpackage #6408
  ........
    r73840 | benjamin.peterson | 2009-07-04 09:52:28 -0500 (Sat, 04 Jul 2009) | 1 line
    don't print diffs by default; it's annoying
  ........
    r73842 | benjamin.peterson | 2009-07-04 09:58:46 -0500 (Sat, 04 Jul 2009) | 1 line
    complain when not showing diffs or writing
  ........
    r73848 | alexandre.vassalotti | 2009-07-04 23:38:19 -0500 (Sat, 04 Jul 2009) | 2 lines
    Fix test_refactor_stdin to handle print_output() method with 4 arguments.
  ........
    r73849 | alexandre.vassalotti | 2009-07-04 23:43:18 -0500 (Sat, 04 Jul 2009) | 5 lines
    Issue 2370: Add fixer for the removal of operator.isCallable() and
    operator.sequenceIncludes().
    Patch contributed by Jeff Balogh (and updated by me).
  ........
    r73861 | benjamin.peterson | 2009-07-05 09:15:53 -0500 (Sun, 05 Jul 2009) | 1 line
    cleanup and use unicode where appropiate
  ........
    r73957 | benjamin.peterson | 2009-07-11 15:49:56 -0500 (Sat, 11 Jul 2009) | 1 line
    fix calls to str() with unicode()
  ........
    r73958 | benjamin.peterson | 2009-07-11 15:51:51 -0500 (Sat, 11 Jul 2009) | 1 line
    more str() -> unicode()
  ........
    r73959 | benjamin.peterson | 2009-07-11 16:40:08 -0500 (Sat, 11 Jul 2009) | 1 line
    add tests for refactor_dir()
  ........
    r73960 | benjamin.peterson | 2009-07-11 16:44:32 -0500 (Sat, 11 Jul 2009) | 1 line
    don't parse files just because they end with 'py' (no dot)
  ........
    r73964 | benjamin.peterson | 2009-07-11 17:30:15 -0500 (Sat, 11 Jul 2009) | 1 line
    simplify
  ........
    r73965 | benjamin.peterson | 2009-07-11 17:31:30 -0500 (Sat, 11 Jul 2009) | 1 line
    remove usage of get_prefix()
  ........
    r73966 | benjamin.peterson | 2009-07-11 17:33:35 -0500 (Sat, 11 Jul 2009) | 1 line
    revert unintended change in 73965
  ........
    r73967 | benjamin.peterson | 2009-07-11 17:34:44 -0500 (Sat, 11 Jul 2009) | 1 line
    avoid expensive checks and assume the node did change
  ........
    r73968 | benjamin.peterson | 2009-07-11 20:46:46 -0500 (Sat, 11 Jul 2009) | 1 line
    use a regular dict for the heads to avoid adding lists in the loop
  ........
    r73969 | benjamin.peterson | 2009-07-11 20:50:43 -0500 (Sat, 11 Jul 2009) | 1 line
    prefix headnode functions with '_'
  ........
    r73972 | benjamin.peterson | 2009-07-11 21:25:45 -0500 (Sat, 11 Jul 2009) | 1 line
    try to make the head node dict as sparse as possible
  ........
    r73973 | benjamin.peterson | 2009-07-11 21:59:49 -0500 (Sat, 11 Jul 2009) | 1 line
    a better idea; add an option to *not* print diffs
  ........
    r73974 | benjamin.peterson | 2009-07-11 22:00:29 -0500 (Sat, 11 Jul 2009) | 1 line
    add space
  ........
    r73977 | benjamin.peterson | 2009-07-12 10:16:07 -0500 (Sun, 12 Jul 2009) | 1 line
    update get_headnode_dict tests for recent changes
  ........
    r73981 | benjamin.peterson | 2009-07-12 12:06:39 -0500 (Sun, 12 Jul 2009) | 4 lines
    detect when "from __future__ import print_function" is given
    Deprecate the 'print_function' option and the -p flag
  ........
    r73984 | benjamin.peterson | 2009-07-12 16:16:37 -0500 (Sun, 12 Jul 2009) | 1 line
    add tests for Call; thanks Joe Amenta
  ........
    r74065 | benjamin.peterson | 2009-07-17 12:52:49 -0500 (Fri, 17 Jul 2009) | 1 line
    pathname2url and url2pathname are in urllib.request not urllib.parse #6496
  ........
    r74113 | benjamin.peterson | 2009-07-20 08:56:57 -0500 (Mon, 20 Jul 2009) | 1 line
    fix deprecation warnings in tests
  ........
................
											
										 
											2009-07-20 16:42:03 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-03-19 05:04:44 +00:00
										 |  |  | python_symbols = Symbols(python_grammar) | 
					
						
							| 
									
										
											  
											
												Merged revisions 74114 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
................
  r74114 | benjamin.peterson | 2009-07-20 10:33:09 -0500 (Mon, 20 Jul 2009) | 110 lines
  Merged revisions 73771,73811,73840,73842,73848-73849,73861,73957-73960,73964-73969,73972-73974,73977,73981,73984,74065,74113 via svnmerge from
  svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
  ........
    r73771 | benjamin.peterson | 2009-07-02 10:56:55 -0500 (Thu, 02 Jul 2009) | 1 line
    force the imports fixer to be run after the import one #6400
  ........
    r73811 | benjamin.peterson | 2009-07-03 09:03:14 -0500 (Fri, 03 Jul 2009) | 1 line
    check for sep, not pathsep when looking for a subpackage #6408
  ........
    r73840 | benjamin.peterson | 2009-07-04 09:52:28 -0500 (Sat, 04 Jul 2009) | 1 line
    don't print diffs by default; it's annoying
  ........
    r73842 | benjamin.peterson | 2009-07-04 09:58:46 -0500 (Sat, 04 Jul 2009) | 1 line
    complain when not showing diffs or writing
  ........
    r73848 | alexandre.vassalotti | 2009-07-04 23:38:19 -0500 (Sat, 04 Jul 2009) | 2 lines
    Fix test_refactor_stdin to handle print_output() method with 4 arguments.
  ........
    r73849 | alexandre.vassalotti | 2009-07-04 23:43:18 -0500 (Sat, 04 Jul 2009) | 5 lines
    Issue 2370: Add fixer for the removal of operator.isCallable() and
    operator.sequenceIncludes().
    Patch contributed by Jeff Balogh (and updated by me).
  ........
    r73861 | benjamin.peterson | 2009-07-05 09:15:53 -0500 (Sun, 05 Jul 2009) | 1 line
    cleanup and use unicode where appropiate
  ........
    r73957 | benjamin.peterson | 2009-07-11 15:49:56 -0500 (Sat, 11 Jul 2009) | 1 line
    fix calls to str() with unicode()
  ........
    r73958 | benjamin.peterson | 2009-07-11 15:51:51 -0500 (Sat, 11 Jul 2009) | 1 line
    more str() -> unicode()
  ........
    r73959 | benjamin.peterson | 2009-07-11 16:40:08 -0500 (Sat, 11 Jul 2009) | 1 line
    add tests for refactor_dir()
  ........
    r73960 | benjamin.peterson | 2009-07-11 16:44:32 -0500 (Sat, 11 Jul 2009) | 1 line
    don't parse files just because they end with 'py' (no dot)
  ........
    r73964 | benjamin.peterson | 2009-07-11 17:30:15 -0500 (Sat, 11 Jul 2009) | 1 line
    simplify
  ........
    r73965 | benjamin.peterson | 2009-07-11 17:31:30 -0500 (Sat, 11 Jul 2009) | 1 line
    remove usage of get_prefix()
  ........
    r73966 | benjamin.peterson | 2009-07-11 17:33:35 -0500 (Sat, 11 Jul 2009) | 1 line
    revert unintended change in 73965
  ........
    r73967 | benjamin.peterson | 2009-07-11 17:34:44 -0500 (Sat, 11 Jul 2009) | 1 line
    avoid expensive checks and assume the node did change
  ........
    r73968 | benjamin.peterson | 2009-07-11 20:46:46 -0500 (Sat, 11 Jul 2009) | 1 line
    use a regular dict for the heads to avoid adding lists in the loop
  ........
    r73969 | benjamin.peterson | 2009-07-11 20:50:43 -0500 (Sat, 11 Jul 2009) | 1 line
    prefix headnode functions with '_'
  ........
    r73972 | benjamin.peterson | 2009-07-11 21:25:45 -0500 (Sat, 11 Jul 2009) | 1 line
    try to make the head node dict as sparse as possible
  ........
    r73973 | benjamin.peterson | 2009-07-11 21:59:49 -0500 (Sat, 11 Jul 2009) | 1 line
    a better idea; add an option to *not* print diffs
  ........
    r73974 | benjamin.peterson | 2009-07-11 22:00:29 -0500 (Sat, 11 Jul 2009) | 1 line
    add space
  ........
    r73977 | benjamin.peterson | 2009-07-12 10:16:07 -0500 (Sun, 12 Jul 2009) | 1 line
    update get_headnode_dict tests for recent changes
  ........
    r73981 | benjamin.peterson | 2009-07-12 12:06:39 -0500 (Sun, 12 Jul 2009) | 4 lines
    detect when "from __future__ import print_function" is given
    Deprecate the 'print_function' option and the -p flag
  ........
    r73984 | benjamin.peterson | 2009-07-12 16:16:37 -0500 (Sun, 12 Jul 2009) | 1 line
    add tests for Call; thanks Joe Amenta
  ........
    r74065 | benjamin.peterson | 2009-07-17 12:52:49 -0500 (Fri, 17 Jul 2009) | 1 line
    pathname2url and url2pathname are in urllib.request not urllib.parse #6496
  ........
    r74113 | benjamin.peterson | 2009-07-20 08:56:57 -0500 (Mon, 20 Jul 2009) | 1 line
    fix deprecation warnings in tests
  ........
................
											
										 
											2009-07-20 16:42:03 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | python_grammar_no_print_statement = python_grammar.copy() | 
					
						
							|  |  |  | del python_grammar_no_print_statement.keywords["print"] | 
					
						
							| 
									
										
											  
											
												Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
  r83852 | benjamin.peterson | 2010-08-08 15:45:44 -0500 (Sun, 08 Aug 2010) | 1 line
  wrap with parens
........
  r83853 | benjamin.peterson | 2010-08-08 15:46:31 -0500 (Sun, 08 Aug 2010) | 1 line
  use parens
........
  r83857 | benjamin.peterson | 2010-08-08 15:59:49 -0500 (Sun, 08 Aug 2010) | 1 line
  things which use touch_import should be pre order
........
  r84042 | george.boutsioukis | 2010-08-14 16:10:19 -0500 (Sat, 14 Aug 2010) | 2 lines
  This revision incorporates into the 2to3 tool the new, faster, tree matching algorithm developed during a GSOC project. The algorithm resides in the two added modules, btm_matcher and btm_utils. New code has been added to drive the new matching process in refactor.py and a few minor changes were made in other modules. A BM_compatible flag(False by default) has been added in fixer_base and it is set to True in most of the current fixers.
........
  r84216 | benjamin.peterson | 2010-08-19 16:44:05 -0500 (Thu, 19 Aug 2010) | 1 line
  allow star_expr in testlist_gexp
........
  r84274 | benjamin.peterson | 2010-08-22 18:40:46 -0500 (Sun, 22 Aug 2010) | 1 line
  wrap long line
........
  r84275 | benjamin.peterson | 2010-08-22 18:42:22 -0500 (Sun, 22 Aug 2010) | 1 line
  cleanup
........
  r84276 | benjamin.peterson | 2010-08-22 18:51:01 -0500 (Sun, 22 Aug 2010) | 1 line
  when there's a None value and a traceback, don't call type with it #9661
........
  r84375 | george.boutsioukis | 2010-08-31 08:38:53 -0500 (Tue, 31 Aug 2010) | 3 lines
  Idiomatic code changes & stylistic issues fixed in the BottomMatcher module. Thanks to Benjamin Peterson for taking the time to review the code.
........
  r85388 | benjamin.peterson | 2010-10-12 17:27:44 -0500 (Tue, 12 Oct 2010) | 1 line
  fix urllib fixer with multiple as imports on a line #10069
........
  r85478 | benjamin.peterson | 2010-10-14 08:09:56 -0500 (Thu, 14 Oct 2010) | 1 line
  stop abusing docstrings
........
  r85506 | benjamin.peterson | 2010-10-14 17:45:19 -0500 (Thu, 14 Oct 2010) | 1 line
  kill sibling import
........
  r85507 | benjamin.peterson | 2010-10-14 17:54:15 -0500 (Thu, 14 Oct 2010) | 1 line
  remove trailing whitespace
........
  r85508 | benjamin.peterson | 2010-10-14 17:55:28 -0500 (Thu, 14 Oct 2010) | 1 line
  typo
........
											
										 
											2010-10-14 23:00:04 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | pattern_grammar = driver.load_grammar(_PATTERN_GRAMMAR_FILE) | 
					
						
							|  |  |  | pattern_symbols = Symbols(pattern_grammar) |