Rename yySymType to lexer; should eventually capture all lexer state.
Embed lexer in parser and access lexer token data directly.
Change-Id: I246194705d594f80426f3ba77d8580af9185daf7
Reviewed-on: https://go-review.googlesource.com/19759
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Somewhat notably, this means long if statement chains are now parsed
recursively, rather than iteratively. This shouldn't be a concern
though, as several other functions (e.g., gen, typecheck, walk)
already use recursion to process the parsed if statement Node trees.
Change-Id: Ic8c12ace9021c870d60c06f5db86a48c4ec57084
Reviewed-on: https://go-review.googlesource.com/19756
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
While here, get drop the lexlineno{++,--} hacks for canned imports.
They were added in commit d3237f9, but don't seem to serve any
purpose.
Change-Id: I00f9e6be0ae9f217f2fa113b85e041dfd0303757
Reviewed-on: https://go-review.googlesource.com/19652
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Merge push_parser and pop_parser into a single parse_import function
and inline unimportfile. Shake out function boundaries a little bit so
that the symmetry is readily visible.
Move the import_package call into parse_import (and inline
import_there into import_package). This means importfile no longer
needs to provide fake import data to be needlessly lexed/parsed every
time it's called.
Also, instead of indicating import success/failure by whether the next
token is "package", import_spec can just check whether importpkg is
non-nil.
Tangentially, this somehow alters the diagnostics produced for
test/fixedbugs/issue11610.go. However, the new diagnostics are more
consistent with those produced when the empty import statement is
absent, which seems more desirable than maintaining the previous
errors.
Change-Id: I5cd1c22aa14da8a743ef569ff084711d137279d5
Reviewed-on: https://go-review.googlesource.com/19650
Reviewed-by: Robert Griesemer <gri@golang.org>
Combine parser's import_stmt and import_here methods as a single new
importdcl method, and cleanup conditional logic slightly to make the
code easier to follow.
Also, eliminate importfile's unused line parameter, and get rid of all
of its duplicate type assertions.
Change-Id: Ic37ae8490afedc533f98ead9feef383e3599bc01
Reviewed-on: https://go-review.googlesource.com/19629
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously, the builtin runtime export data was reparsed before every
Go source file, and the unsafe export data was reparsed for every
import of package unsafe. Now, we parse both of them just once ahead
of time.
This does mean package unsafe's export data will be loaded even when
compiling packages that don't import it, but it's tiny anyway.
Change-Id: Ic6931bc58f6d62f664348bfa932f92d4ccacc3ef
Reviewed-on: https://go-review.googlesource.com/19626
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Changes largely in preparation for eventually switching the builtin
export data to use the new binary format.
Replace fancy incremental line-by-line scanning with simply reading
the entire object file into memory, finding the export data section,
and processing it that way.
Just use "package runtime" and "package unsafe" in the builtin Go
source files so we don't need to rewrite references to "PACKAGE".
Stop looking for init_PACKAGE_function; it doesn't exist anyway.
Compile package runtime with -u so that its export data marks it as a
"safe" package.
Eliminate requirement to pass "runtime" and "unsafe" as command-line
arguments so that just "go run mkbuiltin.go" works.
Only rewrite builtin.go when successful.
Change-Id: I4addfde9e0cfb30607c7a83de686bde0ad1f035a
Reviewed-on: https://go-review.googlesource.com/19624
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It was only really necessary for ensuring that package runtime should
be treated as safe even without a "safe" marker, but mkbuiltin.go now
compiles it with -u.
Change-Id: Ifbcc62436ce40ab732ece667141afd82c1d3b64b
Reviewed-on: https://go-review.googlesource.com/19625
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This code used to be necessary because of the error messages generated
by the YACC-based parser, but they're no longer relevant under the new
recursive descent parser:
- LBRACE no longer exists, so "{ or {" can never occur.
- The parser never generates error messages about "@" or "?" now
(except in import sections, where they're actually legitimate).
- The s/LLITERAL/litbuf/ substitution is handled in p.syntax_error.
Change-Id: Id39f747e4aa492c5830d14a47b161920bd4589ad
Reviewed-on: https://go-review.googlesource.com/17690
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Another (historic) artifact due to partially resolving symbols too early.
Fixes#13539.
Change-Id: Ie720c491cfa399599454f384b3a9735e75d4e8f1
Reviewed-on: https://go-review.googlesource.com/17600
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Following an empty import, a declaration involving a ? symbol
generates an internal compiler error when the name of the
symbol (in newname function).
package a
import""
var?
go.go:2: import path is empty
go.go:3: internal compiler error: newname nil
Make sure dclname is not called when the symbol is nil.
The error message is now:
go.go:2: import path is empty
go.go:3: invalid declaration
go.go:4: syntax error: unexpected EOF
This CL was initially meant to be applied to the old parser,
and has been updated to apply to the new parser.
Fixes#11610
Change-Id: I75e07622fb3af1d104e3a38c89d9e128e3b94522
Reviewed-on: https://go-review.googlesource.com/15268
Reviewed-by: Russ Cox <rsc@golang.org>
Follow-up cleanup for https://go-review.googlesource.com/17248:
Use properly typed local variable op now that that variable use
is not overloaded anymore.
Also: Remove unnecessary if stmt from common lexical path.
Change-Id: I984b0b346f3fdccd5aedc937330c0a5f99acf324
Reviewed-on: https://go-review.googlesource.com/17249
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
- use same local variable name (lno) for line number for LCOLAS everywhere
- remove now unneeded assignment of line number to yylval.i in lexer
Fix per suggestion of mdempsky.
Fixes#13415.
Change-Id: Ie3c7f5681615042a12b81b26724b3a5d8a979c25
Reviewed-on: https://go-review.googlesource.com/17248
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This is a comment/documentation change only but for a minor
code change in the file and package_ methods (move recognition
of semi to match grammar better).
Per request from r.
Change-Id: I81ec985cc5831074d9eb5e8ffbf7e59466284819
Reviewed-on: https://go-review.googlesource.com/17202
Reviewed-by: Rob Pike <r@golang.org>
Get rid of residue after removing old parser.
Change-Id: I0dace1037d50959071a082c276f9f374eef6edb2
Reviewed-on: https://go-review.googlesource.com/17179
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
- moved yySymType and token constants (L...) to lex.go
- removed oldparser flag and related code
- removed go generate that generated y.go
Fixes#13240.
Change-Id: I2576ec61ee1efe482f2a5132175725c9c02ef977
Reviewed-on: https://go-review.googlesource.com/17176
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Introduce a try_ntype function which doesn't return an error upon
not finding a type. Use it instead of having separate repeated
token checks. Simpler, less code, and more efficient.
Change-Id: I81e482158b71901eb179470269349688636aa0ba
Reviewed-on: https://go-review.googlesource.com/17157
Reviewed-by: Chris Manghane <cmang@golang.org>
- fix/check location of popdcl calls where questioned
- remove unnecessary handling of ... (LDDD) in ntype (couldn't be reached)
- inlined and fnret_type and simplified fnres as a consequence
- leave handling of ... (LDDD) in arg_list alone (remove TODO)
- verify that parser requires a ';' after last statement in a case/default
(added test case)
Fixes#13243.
Change-Id: Iad94b498591a5e85f4cb15bbc01e8e101415560d
Reviewed-on: https://go-review.googlesource.com/17155
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Chris Manghane <cmang@golang.org>
Use a combination of follow- and stop-token lists and nesting levels
to better synchronize parser after a syntax error.
Fixes#13319.
Change-Id: I9592e0b5b3ba782fb9f9315fea16163328e204f7
Reviewed-on: https://go-review.googlesource.com/17080
Reviewed-by: Chris Manghane <cmang@golang.org>
Handling of &(T{}) assumed that the parser would not introduce ()'s.
Also: Better comments around handling of OPAREN syntax tree optimization.
Fixes#13261.
Change-Id: Ifc5047a0448f5e7d74cd42f6608b87dcc9c2f2fb
Reviewed-on: https://go-review.googlesource.com/17040
Reviewed-by: Chris Manghane <cmang@golang.org>
Also:
- better error messages in some cases
- factored out function to produce syntax error at given line number
Fixes#13273.
Change-Id: I0192a94731cc23444680a26bd0656ef663e6da0b
Reviewed-on: https://go-review.googlesource.com/16992
Reviewed-by: Chris Manghane <cmang@golang.org>
The line correction when reporting a missing package clause
was removed before since it wasn't clear that it was needed.
Added it again because of issue 13267.
No explicit test case has been added to test/fixedbugs because
it would require a file that contains a single byte and such a
file doesn't fit the existing test harness. Instead documented
the problematic line in the parser for future reference.
Fixes#13267.
Change-Id: I590fe8f358042aab73acf16c2ed9567872b174f4
Reviewed-on: https://go-review.googlesource.com/16975
Reviewed-by: Chris Manghane <cmang@golang.org>
Inlined non_dcl_stmt in stmt since it was called only from there.
Change-Id: I616c41332cfd86bbf3493d0ce7b1256384215220
Reviewed-on: https://go-review.googlesource.com/16893
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Chris Manghane <cmang@golang.org>
- better error messages
- better error recovery by advancing to "follow" token after error
- make sure that we make progress after all errors
- minor cleanups
Change-Id: Ie43b8b02799618d70dc8fc227fab3e4e9e0d8e3a
Reviewed-on: https://go-review.googlesource.com/16892
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Chris Manghane <cmang@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a translation of the yacc-based parser with adjustements
to make the grammar work for a recursive-descent parser followed
by cleanups and simplifications.
The yacc actions were mostly literally copied for correctness
with better temporary names.
A few of the syntax tests were adjusted for slightly different
error messages (it is very difficult to match the yacc-based
error messages in all cases, and sometimes the new parser could
produce better errors).
The new parser is enabled by default.
To switch back to the yacc-based parser, set -oldparser.
To hardwire the switch back, uncomment "oldparser = 1" in lex.go.
- passes all.bash
- ~18% reduced parse time per file on average for make.bash
- ~3% reduced compile time for building cmd/compile
Change-Id: Icb5651bb9d8b9f66261762d2c94a03793050d4ce
Reviewed-on: https://go-review.googlesource.com/16665
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
