Commit graph

138 commits

Author SHA1 Message Date
Russ Cox
e37597f7f0 [dev.regabi] cmd/compile: rename a few 'base' identifiers
We want to introduce a package cmd/compile/internal/base,
and these will shadow it at points where it is needed.

Change-Id: Ic936733fba1ccba8c2ca1fdedbd4d2989df4bbf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/272249
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-11-24 15:07:30 +00:00
Russ Cox
357c576878 [dev.regabi] cmd/compile: clean up error API
Prepare for factoring the error API out of this package by
cleaning it up. The doc comments use the intended new names,
which will be introduced in the next CL.

Change-Id: Ie4c8d4262422da32a9a9f750fda42c225b6b42a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/272248
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-11-24 15:07:09 +00:00
Russ Cox
8bde9b320e cmd/compile: add //go:embed support
This commit contains the compiler support for //go:embed lines.
The go command passes to the compiler an "embed config"
that maps literal patterns like *.txt to the set of files to embed.
The compiler then lays out the content of those files as static data
in the form of an embed.Files or string or []byte in the final object file.

The test for this code is the end-to-end test hooking up the
embed, cmd/compile, and cmd/go changes, in the next CL.

For #41191.

Change-Id: I916e57f8cc65871dc0044c13d3f90c252a3fe1bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/243944
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 00:29:10 +00:00
hk
178b0f59ff cmd/compile/internal/gc: remove an unused method
Change-Id: Ib496d1e9a47d2cf69509ffd0009038b7d34a0149
GitHub-Last-Rev: 5b5589dd03
GitHub-Pull-Request: golang/go#42060
Reviewed-on: https://go-review.googlesource.com/c/go/+/263618
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-19 17:23:00 +00:00
Alberto Donizetti
e2931612b0 cmd/compile: rename strlit, Bool, and Int64 *Node accessors
The Node type has shortcuts to access bool and int Values:

  func (n *Node) Int64() int64
    for n.Val().U.(*Mpint).Int64()

  func (n *Node) Bool() bool
    for n.Val().U.(bool)

I was convinced we didn't have one for string literal nodes, until I
noticed that we do, it's just called strlit, it's not a method, and
it's later in the file:

  func strlit(n *Node) string

This change, for consistency:
- Renames strlit to StringVal and makes it a *Node method
- Renames Bool and Int64 to BoolVal and Int64Val
- Moves StringVal near the other two

Change-Id: I18e635384c35eb3a238fd52b1ccd322b1a74d733
Reviewed-on: https://go-review.googlesource.com/c/go/+/261361
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-10-14 08:00:39 +00:00
Russ Cox
2c6df2e35d cmd/compile: reject misplaced go:build comments
We are converting from using error-prone ad-hoc syntax // +build lines
to less error-prone, standard boolean syntax //go:build lines.
The timeline is:

Go 1.16: prepare for transition
 - Builds still use // +build for file selection.
 - Source files may not contain //go:build without // +build.
 - Builds fail when a source file contains //go:build lines without // +build lines. <<<

Go 1.17: start transition
 - Builds prefer //go:build for file selection, falling back to // +build
   for files containing only // +build.
 - Source files may contain //go:build without // +build (but they won't build with Go 1.16).
 - Gofmt moves //go:build and // +build lines to proper file locations.
 - Gofmt introduces //go:build lines into files with only // +build lines.
 - Go vet rejects files with mismatched //go:build and // +build lines.

Go 1.18: complete transition
 - Go fix removes // +build lines, leaving behind equivalent // +build lines.

This CL provides part of the <<< marked line above in the Go 1.16 step:
rejecting files containing //go:build but not // +build.
The standard go command checks only consider the top of the file.
This compiler check, along with a separate go vet check for ignored files,
handles the remainder of the file.

For #41184.

Change-Id: I014006eebfc84ab5943de18bc90449e534f150a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/240601
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-13 01:16:45 +00:00
Cuong Manh Le
be9ed03f1a cmd/compile/internal/gc: remove unparen
CL 197120 removed the last use of it.

Change-Id: I5fe4f57a47acc712208d831e72cd79205a534c28
Reviewed-on: https://go-review.googlesource.com/c/go/+/252697
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-02 15:49:03 +00:00
Cuong Manh Le
d3f6e2f300 cmd/compile: report error for unexported name only once
Fixes #22921

Change-Id: If29bd962335ac7676ea4f379727db3d55ae1bf8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/250177
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-08-25 16:06:36 +00:00
Emmanuel T Odeke
26de581a70 cmd/compile: omit file:pos for non-existent errors
Omits printing the file:line:column when trying to
open non-existent files

Given:
    go tool compile x.go

* Before:
    x.go:0: open x.go: no such file or directory

* After:
    open x.go: no such file or directory

Reverts the revert in CL 231043 by only fixing the case
of non-existent errors which is what the original bug
was about. The fix for "permission errors" will come later
on when I have bandwidth to investigate the differences
between running with root and why os.Open works for some
builders and not others.

Fixes #36437

Change-Id: I9c8a0981ad708b504bb43990a4105b42266fa41f
Reviewed-on: https://go-review.googlesource.com/c/go/+/230941
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-05-08 20:28:57 +00:00
Russ Cox
768201729d cmd/compile: detect and diagnose invalid //go: directive placement
Thie CL changes cmd/compile/internal/syntax to give the gc half of
the compiler more control over pragma handling, so that it can prepare
better errors, diagnose misuse, and so on. Before, the API between
the two was hard-coded as a uint16. Now it is an interface{}.
This should set us up better for future directives.

In addition to the split, this CL emits a "misplaced compiler directive"
error for any directive that is in a place where it has no effect.
I've certainly been confused in the past by adding comments
that were doing nothing and not realizing it. This should help
avoid that kind of confusion.

The rule, now applied consistently, is that a //go: directive
must appear on a line by itself immediately before the declaration
specifier it means to apply to. See cmd/compile/doc.go for
precise text and test/directive.go for examples.

This may cause some code to stop compiling, but that code
was broken. For example, this code formerly applied the
//go:noinline to f (not c) but now will fail to compile:

	//go:noinline
	const c = 1

	func f() {}

Change-Id: Ieba9b8d90a27cfab25de79d2790a895cefe5296f
Reviewed-on: https://go-review.googlesource.com/c/go/+/228578
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-21 16:47:01 +00:00
Cuong Manh Le
7e1028a9ff cmd/compile: avoid range over copy of array
Passes toostash-check.

Slightly reduce compiler binary size:

file    before    after     Δ       %
compile 21087288  21070776  -16512  -0.078%
total   131847020 131830508 -16512  -0.013%

file                      before    after     Δ       %
cmd/compile/internal/gc.a 9007472   8999640   -7832   -0.087%
total                     127117794 127109962 -7832   -0.006%

Change-Id: I4aadd68d0a7545770598bed9d3a4d05899b67b52
Reviewed-on: https://go-review.googlesource.com/c/go/+/205777
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-03-06 23:24:28 +00:00
Matthew Dempsky
b7d097a4cf cmd/compile: don't apply -lang=go1.X restrictions to imported packages
Previously langSupported applied -lang as though it's a global
restriction, but it's actually a per-package restriction. This CL
fixes langSupported to take a *types.Pkg parameter to reflect this and
updates its callers accordingly.

This is relevant for signed shifts (added in Go 1.12), because they
can be inlined into a Go 1.11 package; and for overlapping interfaces
(added in Go 1.13), because they can be exported as part of the
package's API.

Today we require all Go packages to be compiled with the same
toolchain, and all uses of langSupported are for controlling
backwards-compatible features. So we can simply assume that since the
imported packages type-checked successfully, they must have been
compiled with an appropriate -lang setting.

In the future if we ever want to use langSupported to control
backwards-incompatible language changes, we might need to record the
-lang flag used for compiling a package in its export data.

Fixes #35437.
Fixes #35442.

Change-Id: Ifdf6a62ee80cd5fb4366cbf12933152506d1b36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205977
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-11-08 21:27:51 +00:00
Ariel Mashraki
33fac9bad8 cmd/compile/internal/gc: preallocate slices in noder
The slice capacity is known for all these cases. Therefore,
we can initialize them with the desired capacity.

Change-Id: I1835b49108d157203d62e4aa119c2d7ab5e5e46f
Reviewed-on: https://go-review.googlesource.com/c/go/+/200119
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-11 18:56:30 +00:00
Matthew Dempsky
2197321db1 cmd/compile: split n.Noescape() into separate uses
n.Noescape() was overloaded for two uses: (1) to indicate a function
was annotated with //go:noescape, and (2) to indicate that certain
temporary allocations don't outlive the current statement.

The first use case is redundant with n.Func.Pragma&Noescape!=0, which
is the convention we use for checking other function-level pragmas.

The second use case is better served by renaming "Noescape" to
"Transient".

Passes toolstash-check.

Change-Id: I0f09d2d5767513894b7bf49da9cdabd04aa4a05e
Reviewed-on: https://go-review.googlesource.com/c/go/+/199822
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-08 23:53:55 +00:00
Robert Griesemer
27fc32ff01 cmd/compile: better error message for language version errors
Fixes #33753.
Updates #31747.

Change-Id: Icc42b23405ead4f7f17b0ffa3611405454b6b271
Reviewed-on: https://go-review.googlesource.com/c/go/+/198491
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-03 04:48:44 +00:00
Matthew Dempsky
00b773a4a9 cmd/compile: simplify OPTRLIT handling
Previously, we would recognize &(T{...}) expressions during type
checking, rewrite them into (*T){...}, and then do a lot of extra work
to make sure the user doesn't write (*T){...} themselves and
resynthesizing the OPTRLIT later on.

This CL simply handles &T{...} directly in the straight forward
manner, by changing OADDR directly to OPTRLIT when appropriate.

While here, match go/types's invalid composite literal type error
message.

Passes toolstash-check.

Change-Id: I902b14c7e2cd9fa93e6915dd58272d2352ba38f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/197120
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-26 18:45:53 +00:00
Matthew Dempsky
2fc6366fb5 cmd/compile: remove OCASE and rename OXCASE to OCASE
We used to use OXCASE to represent general, possibly multi-valued
cases, and then desugar these during walk into single-value cases
represented by OCASE.

In CL 194660, we switched to eliminated the desugaring step and
instead handle the multi-valued cases directly, which eliminates the
need for an OCASE Op. Instead, we can simply remove OCASE, and rename
OXCASE to just OCASE.

Passes toolstash-check.

Change-Id: I3cc184340f9081d37453927cca1c059267fdbc12
Reviewed-on: https://go-review.googlesource.com/c/go/+/196117
Reviewed-by: Keith Randall <khr@golang.org>
2019-09-18 02:31:34 +00:00
Cuong Manh Le
75da700d0a cmd/compile: consistently use strlit to access constants string values
Passes toolstash-check.

Change-Id: Ieaef20b7649787727b69469f93ffc942022bc079
Reviewed-on: https://go-review.googlesource.com/c/go/+/195198
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-09-16 11:41:20 +00:00
Robert Griesemer
5411953df5 cmd/compile: avoid follow-on errors for literals with syntax errors
- only convert literal strings if there were no syntax errors
  (some of the conversion routines exit if there is an error)
- mark nodes for literals with syntax errors to avoid follow-on
  errors
- don't attempt to import packages whose path had syntax errors

Fixes #32133.

Change-Id: I1803ad48c65abfecf6f48ddff1e27eded5e282c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/192437
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-08-29 23:37:07 +00:00
Austin Clements
dde7c770ef cmd/compile: make the second argument to go:linkname optional
The //go:linkname directive can be used to make a symbol accessible to
another package (when it wouldn't normally be). Sometimes you want to
do this without actually changing the symbol's object file symbol
name; for example, in gccgo this makes unexported symbols non-static,
and in gc this provides ABI0 wrappers for Go symbols so they can be
called from assembly in other packages. Currently, this results in
stutter like

   //go:linkname entersyscall runtime.entersyscall

This CL makes the second argument to go:linkname optional for the case
where the intent is simply to expose the symbol rather than to rename
it in the object file.

Updates #31230.

Change-Id: Id06d9c4b2ec3d8e27f9b8a0d65212ab8048d734f
Reviewed-on: https://go-review.googlesource.com/c/go/+/179861
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2019-06-06 19:44:10 +00:00
Robert Griesemer
762953be28 cmd/compile: disable Go1.13 language features for -lang=go1.12 and below
Fixes   #31747.
Updates #19308.
Updates #12711.
Updates #29008.
Updates #28493.
Updates #19113.

Change-Id: I76d2fdbc7698cc4e0f31b7ae24cbb4d28afbb6a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174897
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-05-02 00:42:51 +00:00
Robert Griesemer
041d31b882 cmd/compile: don't mix internal float/complex constants of different precision
There are several places where a new (internal) complex constant is allocated
via new(Mpcplx) rather than newMpcmplx(). The problem with using new() is that
the Mpcplx data structure's Real and Imag components don't get initialized with
an Mpflt of the correct precision (they have precision 0, which may be adjusted
later).

In all cases but one, the components of those complex constants are set using
a Set operation which "inherits" the correct precision from the value that is
being set.

But when creating a complex value for an imaginary literal, the imaginary
component is set via SetString which assumes 64bits of precision by default.
As a result, the internal representation of 0.01i and complex(0, 0.01) was
not correct.

Replaced all used of new(Mpcplx) with newMpcmplx() and added a new test.

Fixes #30243.

Change-Id: Ife7fd6ccd42bf887a55c6ce91727754657e6cb2d
Reviewed-on: https://go-review.googlesource.com/c/163000
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-02-19 21:05:17 +00:00
Robert Griesemer
cd47e8944d cmd/compile: avoid multiple errors regarding misuse of ... in signatures
Follow-up on #28450 (golang.org/cl/152417).

Updates #28450.
Fixes #29107.

Change-Id: Ib4b4fe582c35315a4f71cf6dbc7f7f2f24b37ec1
Reviewed-on: https://go-review.googlesource.com/c/152758
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-12-05 20:59:58 +00:00
smasher164
a7af474359 cmd/compile: improve error message for non-final variadic parameter
Previously, when a function signature had defined a non-final variadic
parameter, the error message always referred to the type associated with that
parameter. However, if the offending parameter's name was part of an identifier
list with a variadic type, one could misinterpret the message, thinking the
problem had been with one of the other names in the identifer list.

    func bar(a, b ...int) {}
clear ~~~~~~~^       ^~~~~~~~ confusing

This change updates the error message and sets the column position to that of
the offending parameter's name, if it exists.

Fixes #28450.

Change-Id: I076f560925598ed90e218c25d70f9449ffd9b3ea
Reviewed-on: https://go-review.googlesource.com/c/152417
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-12-05 01:35:59 +00:00
Keith Randall
ca3749230b cmd/compile: allow bodyless function if it is linkname'd
In assembly free packages (aka "complete" or "pure go"), allow
bodyless functions if they are linkname'd to something else.

Presumably the thing the function is linkname'd to has a definition.
If not, the linker will complain. And linkname is unsafe, so we expect
users to know what they are doing.

Note this handles only one direction, where the linkname directive
is in the local package. If the linkname directive is in the remote
package, this CL won't help. (See os/signal/sig.s for an example.)

Fixes #23311

Change-Id: I824361b4b582ee05976d94812e5b0e8b0f7a18a6
Reviewed-on: https://go-review.googlesource.com/c/151318
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-11-26 20:00:59 +00:00
Josh Bleecher Snyder
bc43889566 cmd/compile: bulk rename
This change does a bulk rename of several identifiers in the compiler.
See #27167 and https://docs.google.com/document/d/19_ExiylD9MRfeAjKIfEsMU1_RGhuxB9sA0b5Zv7byVI/
for context and for discussion of these particular renames.

Commands run to generate this change:

gorename -from '"cmd/compile/internal/gc".OPROC' -to OGO
gorename -from '"cmd/compile/internal/gc".OCOM' -to OBITNOT
gorename -from '"cmd/compile/internal/gc".OMINUS' -to ONEG
gorename -from '"cmd/compile/internal/gc".OIND' -to ODEREF
gorename -from '"cmd/compile/internal/gc".OARRAYBYTESTR' -to OBYTES2STR
gorename -from '"cmd/compile/internal/gc".OARRAYBYTESTRTMP' -to OBYTES2STRTMP
gorename -from '"cmd/compile/internal/gc".OARRAYRUNESTR' -to ORUNES2STR
gorename -from '"cmd/compile/internal/gc".OSTRARRAYBYTE' -to OSTR2BYTES
gorename -from '"cmd/compile/internal/gc".OSTRARRAYBYTETMP' -to OSTR2BYTESTMP
gorename -from '"cmd/compile/internal/gc".OSTRARRAYRUNE' -to OSTR2RUNES

gorename -from '"cmd/compile/internal/gc".Etop' -to ctxStmt
gorename -from '"cmd/compile/internal/gc".Erv' -to ctxExpr
gorename -from '"cmd/compile/internal/gc".Ecall' -to ctxCallee
gorename -from '"cmd/compile/internal/gc".Efnstruct' -to ctxMultiOK
gorename -from '"cmd/compile/internal/gc".Easgn' -to ctxAssign
gorename -from '"cmd/compile/internal/gc".Ecomplit' -to ctxCompLit

Not altered: parameters and local variables (mostly in typecheck.go) named top,
which should probably now be called ctx (and which should probably have a named type).
Also not altered: Field called Top in gc.Func.

gorename -from '"cmd/compile/internal/gc".Node.Isddd' -to IsDDD
gorename -from '"cmd/compile/internal/gc".Node.SetIsddd' -to SetIsDDD
gorename -from '"cmd/compile/internal/gc".nodeIsddd' -to nodeIsDDD
gorename -from '"cmd/compile/internal/types".Field.Isddd' -to IsDDD
gorename -from '"cmd/compile/internal/types".Field.SetIsddd' -to SetIsDDD
gorename -from '"cmd/compile/internal/types".fieldIsddd' -to fieldIsDDD

Not altered: function gc.hasddd, params and local variables called isddd
Also not altered: fmt.go prints nodes using "isddd(%v)".

cd cmd/compile/internal/gc; go generate

I then manually found impacted comments using exact string match
and fixed them up by hand. The comment changes were trivial.

Passes toolstash-check.

Fixes #27167. If this experiment is deemed a success,
we will open a new tracking issue for renames to do
at the end of the 1.13 cycles.

Change-Id: I2dc541533d2ab0d06cb3d31d65df205ecfb151e8
Reviewed-on: https://go-review.googlesource.com/c/150140
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-11-19 00:02:53 +00:00
Austin Clements
685aca45dc cmd/compile, cmd/link: separate stable and internal ABIs
This implements compiler and linker support for separating the
function calling ABI into two ABIs: a stable and an internal ABI. At
the moment, the two ABIs are identical, but we'll be able to evolve
the internal ABI without breaking existing assembly code that depends
on the stable ABI for calling to and from Go.

The Go compiler generates internal ABI symbols for all Go functions.
It uses the symabis information produced by the assembler to create
ABI wrappers whenever it encounters a body-less Go function that's
defined in assembly or a Go function that's referenced from assembly.

Since the two ABIs are currently identical, for the moment this is
implemented using "ABI alias" symbols, which are just forwarding
references to the native ABI symbol for a function. This way there's
no actual code involved in the ABI wrapper, which is good because
we're not deriving any benefit from it right now. Once the ABIs
diverge, we can eliminate ABI aliases.

The linker represents these different ABIs internally as different
versions of the same symbol. This way, the linker keeps us honest,
since every symbol definition and reference also specifies its
version. The linker is responsible for resolving ABI aliases.

Fixes #27539.

Change-Id: I197c52ec9f8fc435db8f7a4259029b20f6d65e95
Reviewed-on: https://go-review.googlesource.com/c/147160
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2018-11-12 20:46:55 +00:00
Robert Griesemer
f6f27bfb97 cmd/compile/internal/gc: more direct noder.nod implementation (cleanup)
Also, renamed

- noder.lineno -> noder.setlineno (because that's what it does)
- noder.setlineno -> noder.pos    (and return the src.XPos)

Change-Id: I5d3442cf2af97028afcab028290152ce8d062927
Reviewed-on: https://go-review.googlesource.com/c/146317
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-10-31 19:29:56 +00:00
Ian Lance Taylor
2e9f0817f0 cmd/compile: add -lang flag to specify language version
The default language version is the current one.

For testing purposes, added a check that type aliases require version
go1.9. There is no consistent support for changes made before 1.12.

Updates #28221

Change-Id: Ia1ef63fff911d5fd29ef79d5fa4e20cfd945feb7
Reviewed-on: https://go-review.googlesource.com/c/144340
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-10-30 04:39:53 +00:00
Matthew Dempsky
2dda040f19 cmd/compile/internal/gc: represent labels as bare Syms
Avoids allocating an ONAME for OLABEL, OGOTO, and named OBREAK and
OCONTINUE nodes.

Passes toolstash-check.

Change-Id: I359142cd48e8987b5bf29ac100752f8c497261c1
Reviewed-on: https://go-review.googlesource.com/c/145200
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-10-27 07:32:06 +00:00
Robert Griesemer
5538ecadca cmd/compile: better error for embedded field referring to missing import
Fixes #27938.

Change-Id: I16263ac6c0b8903b8a16f02e8db0e1a16d1c95b4
Reviewed-on: https://go-review.googlesource.com/c/144261
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-10-24 20:08:18 +00:00
Yury Smolsky
e411bd706d cmd/compile: add doc comment to the parseFiles function
Change-Id: Ifa14557ba834865602c207297ccf0c63e93feb4c
Reviewed-on: https://go-review.googlesource.com/117695
Run-TryBot: Yury Smolsky <yury@smolsky.by>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-06-11 17:27:48 +00:00
Daniel Martí
8c67ca1a99 cmd/compile: apply TODO now that gcCompat is gone
Passes toolstash -cmp on std cmd.

Change-Id: I17b9a816752af55b064b8e46146b8f5a878fa560
Reviewed-on: https://go-review.googlesource.com/109075
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-05-01 05:02:27 +00:00
Matthew Dempsky
ae2a2d12f6 cmd/compile: cleaner solution for importing init functions
Using oldname+resolve is how typecheck handles this anyway.

Passes toolstash -cmp, with both -iexport enabled and disabled.

Change-Id: I12b0f0333d6b86ce6bfc4d416c461b5f15c1110d
Reviewed-on: https://go-review.googlesource.com/109715
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-04-26 23:21:28 +00:00
Matthew Dempsky
ca2f85fd3f cmd/compile: add indexed export format
This CL introduces a new indexed data format for package export
data. This improves on the previous (sequential) binary format by
allowing the compiler to selectively (and lazily) load only the data
that's actually needed for compilation.

In large Go projects, the package export data can become very large
due to transitive type declaration dependencies and inline
function/method bodies. By lazily loading these declarations and
bodies as needed, we avoid wasting time and memory processing
unnecessary and/or redundant data.

In the benchmarks below, "old" is -iexport=false and "new" is
-iexport=true. The suffixes indicate the compiler concurrency (-c) and
inlining (-l) settings used for the build (using -gcflags=all=-foo).
Benchmarks were run on an HP Z620.

Juju is "go build -a github.com/juju/juju/cmd/...":

name          old real-time/op  new real-time/op  delta
Juju/c=1/l=0        44.0s ± 1%        38.7s ± 9%  -11.97%  (p=0.001 n=7+7)
Juju/c=1/l=4        53.7s ± 3%        45.3s ± 4%  -15.53%  (p=0.001 n=7+7)
Juju/c=4/l=0        39.7s ± 8%        32.0s ± 4%  -19.38%  (p=0.001 n=7+7)
Juju/c=4/l=4        46.3s ± 4%        38.0s ± 4%  -18.06%  (p=0.001 n=7+7)

name          old user-time/op  new user-time/op  delta
Juju/c=1/l=0         371s ± 1%         300s ± 0%  -19.07%  (p=0.001 n=7+6)
Juju/c=1/l=4         482s ± 0%         374s ± 1%  -22.37%  (p=0.001 n=7+7)
Juju/c=4/l=0         410s ± 1%         340s ± 1%  -17.19%  (p=0.001 n=7+7)
Juju/c=4/l=4         532s ± 1%         424s ± 1%  -20.26%  (p=0.001 n=7+7)

name          old sys-time/op   new sys-time/op   delta
Juju/c=1/l=0        33.4s ± 1%        28.4s ± 2%  -15.02%  (p=0.001 n=7+7)
Juju/c=1/l=4        40.7s ± 2%        32.8s ± 3%  -19.51%  (p=0.001 n=7+7)
Juju/c=4/l=0        39.8s ± 2%        34.4s ± 2%  -13.74%  (p=0.001 n=7+7)
Juju/c=4/l=4        48.4s ± 2%        40.4s ± 2%  -16.50%  (p=0.001 n=7+7)

Kubelet is "go build -a k8s.io/kubernetes/cmd/kubelet":

name             old real-time/op  new real-time/op  delta
Kubelet/c=1/l=0        42.0s ± 1%        34.8s ± 1%  -17.27%  (p=0.008 n=5+5)
Kubelet/c=1/l=4        55.4s ± 3%        45.4s ± 3%  -18.06%  (p=0.002 n=6+6)
Kubelet/c=4/l=0        37.4s ± 3%        29.9s ± 1%  -20.25%  (p=0.004 n=6+5)
Kubelet/c=4/l=4        48.1s ± 2%        39.0s ± 5%  -18.93%  (p=0.002 n=6+6)

name             old user-time/op  new user-time/op  delta
Kubelet/c=1/l=0         291s ± 1%         233s ± 1%  -19.96%  (p=0.002 n=6+6)
Kubelet/c=1/l=4         385s ± 1%         298s ± 1%  -22.51%  (p=0.002 n=6+6)
Kubelet/c=4/l=0         325s ± 0%         268s ± 1%  -17.48%  (p=0.004 n=5+6)
Kubelet/c=4/l=4         429s ± 1%         343s ± 1%  -20.08%  (p=0.002 n=6+6)

name             old sys-time/op   new sys-time/op   delta
Kubelet/c=1/l=0        25.1s ± 2%        20.9s ± 4%  -16.69%  (p=0.002 n=6+6)
Kubelet/c=1/l=4        31.2s ± 3%        24.4s ± 0%  -21.67%  (p=0.010 n=6+4)
Kubelet/c=4/l=0        30.2s ± 2%        25.6s ± 1%  -15.34%  (p=0.002 n=6+6)
Kubelet/c=4/l=4        37.3s ± 1%        30.9s ± 2%  -17.11%  (p=0.002 n=6+6)

Change-Id: Ie43eb3bbe1392cbb61c86792a17a57b33b9561f0
Reviewed-on: https://go-review.googlesource.com/106796
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-04-24 01:05:27 +00:00
Matthew Dempsky
7759b32a62 cmd/compile: replace Field.Nname.Pos with Field.Pos
For struct fields and methods, Field.Nname was only used to store
position information, which means we're allocating an entire ONAME
Node+Name+Param structure just for one field. We can optimize away
these ONAME allocations by instead adding a Field.Pos field.

Unfortunately, we can't get rid of Field.Nname, because it's needed
for function parameters, so Field grows a little bit and now has more
redundant information in those cases. However, that was already the
case (e.g., Field.Sym and Field.Nname.Sym), and it's still a net win
for allocations as demonstrated by the benchmarks below.

Additionally, by moving the ONAME allocation for function parameters
to funcargs, we can avoid allocating them for function parameters that
aren't used in corresponding function bodies (e.g., interface methods,
function-typed variables, and imported functions/methods without
inline bodies).

name       old time/op       new time/op       delta
Template         254ms ± 6%        251ms ± 6%  -1.04%  (p=0.000 n=487+488)
Unicode          128ms ± 7%        128ms ± 7%    ~     (p=0.294 n=482+467)
GoTypes          862ms ± 5%        860ms ± 4%    ~     (p=0.075 n=488+471)
Compiler         3.91s ± 4%        3.90s ± 4%  -0.39%  (p=0.000 n=468+473)

name       old user-time/op  new user-time/op  delta
Template         339ms ±14%        336ms ±14%  -1.02%  (p=0.001 n=498+494)
Unicode          176ms ±18%        176ms ±25%    ~     (p=0.940 n=491+499)
GoTypes          1.13s ± 8%        1.13s ± 9%    ~     (p=0.157 n=496+493)
Compiler         5.24s ± 6%        5.21s ± 6%  -0.57%  (p=0.000 n=485+489)

name       old alloc/op      new alloc/op      delta
Template        38.3MB ± 0%       37.3MB ± 0%  -2.58%  (p=0.000 n=499+497)
Unicode         29.1MB ± 0%       29.1MB ± 0%  -0.03%  (p=0.000 n=500+493)
GoTypes          116MB ± 0%        115MB ± 0%  -0.65%  (p=0.000 n=498+499)
Compiler         492MB ± 0%        487MB ± 0%  -1.00%  (p=0.000 n=497+498)

name       old allocs/op     new allocs/op     delta
Template          364k ± 0%         360k ± 0%  -1.15%  (p=0.000 n=499+499)
Unicode           336k ± 0%         336k ± 0%  -0.01%  (p=0.000 n=500+493)
GoTypes          1.16M ± 0%        1.16M ± 0%  -0.30%  (p=0.000 n=499+499)
Compiler         4.54M ± 0%        4.51M ± 0%  -0.58%  (p=0.000 n=494+495)

Passes toolstash-check -gcflags=-dwarf=false. Changes DWARF output
because position information is now tracked more precisely for
function parameters.

Change-Id: Ib8077d70d564cc448c5e4290baceab3a4396d712
Reviewed-on: https://go-review.googlesource.com/108217
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-04-23 18:05:57 +00:00
Daniel Martí
2b2348ab14 cmd/compile/internal/gc: add some Node methods
Focus on "isfoo" funcs that take a *Node, and conver them to isFoo
methods instead. This makes for more idiomatic Go code, and also more
readable func names.

Found candidates with grep, and applied most changes with sed. The funcs
chosen were isgoconst, isnil, and isblank. All had the same signature,
func(*Node) bool.

While at it, camelCase the isliteral and iszero function names. Don't
move these to methods, as they are only used in the backend part of gc,
which might one day be split into a separate package.

Passes toolstash -cmp on std cmd.

Change-Id: I4df081b12d36c46c253167c8841c5a841f1c5a16
Reviewed-on: https://go-review.googlesource.com/105555
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-04-16 00:16:55 +00:00
Matthew Dempsky
8f6ae33796 cmd/compile, cmd/link: encode cgo directives using JSON
The standard library has plenty of polished encoder/decoder
implementations. No need for another ad-hoc one.

I considered using encoding/gob instead, but these strings go into the
package data part of the object file, so it's important they don't
contain "\n$$\n". Package json escapes newlines in strings, so it's
safe to use here.

Change-Id: I998655524ccee7365c2c8e9a843e6975e95a3e62
Reviewed-on: https://go-review.googlesource.com/106463
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-11 23:35:42 +00:00
Matthew Dempsky
07029254a0 cmd/compile: add package height to export data
A package's height is defined as the length of the longest import path
between itself and a leaf package (i.e., package with no imports).

We can't rely on knowing the path of the package being compiled, so
package height is useful for defining a package ordering.

Updates #24693.

Change-Id: I965162c440b6c5397db91b621ea0be7fa63881f1
Reviewed-on: https://go-review.googlesource.com/105038
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-04-09 23:36:46 +00:00
Matthew Dempsky
26e0e8a840 cmd/compile: improve declaration position precision
Previously, n.Pos was reassigned to lineno when declare was called,
which might not match where the identifier actually appeared in the
source. This caused a loss of position precision for function
parameters (which were all declared at the last parameter's position),
and required some clumsy workarounds in bimport.go.

This CL changes declare to leave n.Pos alone and also fixes a few
places where n.Pos was not being set correctly.

Change-Id: Ibe5b5fd30609c684367207df701f9a1bfa82867f
Reviewed-on: https://go-review.googlesource.com/104275
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-04-03 05:45:44 +00:00
Robert Griesemer
c65a2781be cmd/compile: better handling of incorrect type switches
Don't report errors if we don't have a correct type switch
guard; instead ignore it and leave it to the type-checker
to report the error. This leads to better error messages
concentrating on the type switch guard rather than errors
around (confusing) syntactic details.

Also clean up some code setting up AssertExpr (they never
have a nil Type field) and remove some incorrect TODOs.

Fixes #24470.

Change-Id: I69512f36e0417e3b5ea9c8856768e04b19d654a8
Reviewed-on: https://go-review.googlesource.com/103615
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-04-03 05:34:20 +00:00
Matthew Dempsky
29517daff9 cmd/compile: extract common noding code from func{Decl,Lit}
Passes toolstash-check.

Change-Id: I8290221d6169e077dfa4ea737d685c7fcecf6841
Reviewed-on: https://go-review.googlesource.com/100835
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-03-15 01:09:42 +00:00
Robert Griesemer
363bcd7b4f cmd/compile: use key position for key:val elements in composite literals
Fixes #24339.

Change-Id: Ie47764fed27f76b480834b1fdbed0512c94831d9
Reviewed-on: https://go-review.googlesource.com/100457
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-03-13 21:44:12 +00:00
Matthew Dempsky
e4de522c95 cmd/compile: fix Node.Etype overloading
Add helper methods that validate n.Op and convert to/from the
appropriate type.

Notably, there was a lot of code in walk.go that thought setting
Etype=1 on an OADDR node affected escape analysis.

Passes toolstash-check.

TBR=marvin

Change-Id: Ieae7c67225c1459c9719f9e6a748a25b975cf758
Reviewed-on: https://go-review.googlesource.com/99535
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-03-09 21:44:35 +00:00
Alessandro Arzilli
3fca7306f4 cmd/compile: optimize scope tracking
1. Detect and remove the markers of lexical scopes that don't contain
any variables early in noder, instead of waiting until the end of DWARF
generation.
This saves memory by never allocating some of the markers and optimizes
some of the algorithms that depend on the number of scopes.

2. Assign scopes to Progs by doing, for each Prog, a binary search over
the markers array. This is faster, compared to sorting the Prog list
because there are fewer markers than there are Progs.

completed   15 of   15, estimated time remaining 0s (eta 2:30PM)
name        old time/op       new time/op       delta
Template          274ms ± 5%        260ms ± 6%  -4.91%  (p=0.000 n=15+15)
Unicode           126ms ± 5%        127ms ± 9%    ~     (p=0.856 n=13+15)
GoTypes           861ms ± 5%        857ms ± 4%    ~     (p=0.595 n=15+15)
Compiler          4.11s ± 4%        4.12s ± 5%    ~     (p=1.000 n=15+15)
SSA               10.7s ± 2%        10.9s ± 4%  +2.01%  (p=0.002 n=14+14)
Flate             163ms ± 4%        166ms ± 9%    ~     (p=0.134 n=14+15)
GoParser          203ms ± 4%        205ms ± 6%    ~     (p=0.461 n=15+15)
Reflect           544ms ± 5%        549ms ± 4%    ~     (p=0.174 n=15+15)
Tar               249ms ± 9%        245ms ± 6%    ~     (p=0.285 n=15+15)
XML               286ms ± 4%        291ms ± 5%    ~     (p=0.081 n=15+15)
[Geo mean]        528ms             529ms       +0.14%

name        old user-time/op  new user-time/op  delta
Template          358ms ± 7%        354ms ± 5%    ~     (p=0.242 n=14+15)
Unicode           189ms ±11%        191ms ±10%    ~     (p=0.438 n=15+15)
GoTypes           1.15s ± 4%        1.14s ± 3%    ~     (p=0.405 n=15+15)
Compiler          5.36s ± 6%        5.35s ± 5%    ~     (p=0.588 n=15+15)
SSA               14.6s ± 3%        15.0s ± 4%  +2.58%  (p=0.000 n=15+15)
Flate             214ms ±12%        216ms ± 8%    ~     (p=0.539 n=15+15)
GoParser          267ms ± 6%        270ms ± 5%    ~     (p=0.569 n=15+15)
Reflect           712ms ± 5%        709ms ± 4%    ~     (p=0.894 n=15+15)
Tar               329ms ± 8%        330ms ± 5%    ~     (p=0.974 n=14+15)
XML               371ms ± 3%        381ms ± 5%  +2.85%  (p=0.002 n=13+15)
[Geo mean]        705ms             709ms       +0.62%

name        old alloc/op      new alloc/op      delta
Template         38.0MB ± 0%       38.4MB ± 0%  +1.27%  (p=0.000 n=15+14)
Unicode          28.8MB ± 0%       28.8MB ± 0%  +0.16%  (p=0.000 n=15+14)
GoTypes           112MB ± 0%        114MB ± 0%  +1.64%  (p=0.000 n=15+15)
Compiler          465MB ± 0%        474MB ± 0%  +1.91%  (p=0.000 n=15+15)
SSA              1.48GB ± 0%       1.53GB ± 0%  +3.32%  (p=0.000 n=15+15)
Flate            24.3MB ± 0%       24.8MB ± 0%  +1.77%  (p=0.000 n=14+15)
GoParser         30.7MB ± 0%       31.1MB ± 0%  +1.27%  (p=0.000 n=15+15)
Reflect          76.3MB ± 0%       77.1MB ± 0%  +1.03%  (p=0.000 n=15+15)
Tar              39.2MB ± 0%       39.6MB ± 0%  +1.02%  (p=0.000 n=13+15)
XML              41.5MB ± 0%       42.1MB ± 0%  +1.45%  (p=0.000 n=15+15)
[Geo mean]       77.5MB            78.7MB       +1.48%

name        old allocs/op     new allocs/op     delta
Template           385k ± 0%         387k ± 0%  +0.54%  (p=0.000 n=15+15)
Unicode            342k ± 0%         343k ± 0%  +0.10%  (p=0.000 n=15+15)
GoTypes           1.19M ± 0%        1.19M ± 0%  +0.64%  (p=0.000 n=14+15)
Compiler          4.51M ± 0%        4.54M ± 0%  +0.53%  (p=0.000 n=15+15)
SSA               12.2M ± 0%        12.4M ± 0%  +1.16%  (p=0.000 n=15+15)
Flate              234k ± 0%         236k ± 0%  +0.63%  (p=0.000 n=14+15)
GoParser           318k ± 0%         320k ± 0%  +0.63%  (p=0.000 n=15+15)
Reflect            974k ± 0%         977k ± 0%  +0.28%  (p=0.000 n=15+15)
Tar                395k ± 0%         397k ± 0%  +0.38%  (p=0.000 n=15+13)
XML                404k ± 0%         407k ± 0%  +0.55%  (p=0.000 n=15+15)
[Geo mean]         794k              799k       +0.55%

name        old text-bytes    new text-bytes    delta
HelloSize         680kB ± 0%        680kB ± 0%    ~     (all equal)

name        old data-bytes    new data-bytes    delta
HelloSize        9.62kB ± 0%       9.62kB ± 0%    ~     (all equal)

name        old bss-bytes     new bss-bytes     delta
HelloSize         125kB ± 0%        125kB ± 0%    ~     (all equal)

name        old exe-bytes     new exe-bytes     delta
HelloSize        1.11MB ± 0%       1.12MB ± 0%  +1.11%  (p=0.000 n=15+15)

Change-Id: I95a0173ee28c52be1a4851d2a6e389529e74bf28
Reviewed-on: https://go-review.googlesource.com/95396
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-02 10:38:41 +00:00
Robert Griesemer
0c884d0810 cmd/compile, cmd/compile/internal/syntax: print relative column info
This change enables printing of relative column information if a
prior line directive specified a valid column. If there was no
line directive, or the line directive didn't specify a column
(or the -C flag is specified), no column information is shown in
file positions.

Implementation: Column values (and line values, for that matter)
that are zero are interpreted as "unknown". A line directive that
doesn't specify a column records that as a zero column in the
respective PosBase data structure. When computing relative columns,
a relative value is zero of the base's column value is zero.
When formatting a position, a zero column value is not printed.

To make this work without special cases, the PosBase for a file
is given a concrete (non-0:0) position 1:1 with the PosBase's
line and column also being 1:1. In other words, at the position
1:1 of a file, it's relative positions are starting with 1:1 as
one would expect.

In the package syntax, this requires self-recursive PosBases for
file bases, matching what cmd/internal/src.PosBase was already
doing. In src.PosBase, file and inlining bases also need to be
based at 1:1 to indicate "known" positions.

This change completes the cmd/compiler part of the issue below.

Fixes #22662.

Change-Id: I6c3d2dee26709581fba0d0261b1d12e93f1cba1a
Reviewed-on: https://go-review.googlesource.com/97375
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-02-28 03:51:23 +00:00
Robert Griesemer
515fa58ac9 cmd/compile: track line directives w/ column information
Extend cmd/internal/src.PosBase to track column information,
and adjust the meaning of the PosBase position to mean the
position at which the PosBase's relative (line, col) position
starts (rather than indicating the position of the //line
directive). Because this semantic change is made in the
compiler's noder, it doesn't affect the logic of src.PosBase,
only its test setup (where PosBases are constructed with
corrected incomming positions). In short, src.PosBase now
matches syntax.PosBase with respect to the semantics of
src.PosBase.pos.

For #22662.

Change-Id: I5b1451cb88fff3f149920c2eec08b6167955ce27
Reviewed-on: https://go-review.googlesource.com/96535
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-02-26 18:32:03 +00:00
Robert Griesemer
5c08b9e8bd cmd/compile/internal/syntax: remove dependency on cmd/internal/src
For dependency reasons, the data structure implementing source
positions in the compiler is in cmd/internal/src. It contains
highly compiler specific details (e.g. inlining index).

This change introduces a parallel but simpler position
representation, defined in the syntax package, which removes
that package's dependency on cmd/internal/src, and also removes
the need to deal with certain filename-specific operations
(defined by the needs of the compiler) in the syntax package.
As a result, the syntax package becomes again a compiler-
independent, stand-alone package that at some point might
replace (or augment) the existing top-level go/* syntax-related
packages.

Additionally, line directives that update column numbers
are now correctly tracked through the syntax package, with
additional tests added. (The respective changes also need to
be made in cmd/internal/src; i.e., the compiler accepts but
still ignores column numbers in line directives.)

This change comes at the cost of a new position translation
step, but that step is cheap because it only needs to do real
work if the position base changed (i.e., if there is a new file,
or new line directive).

There is no noticeable impact on overall compiler performance
measured with `compilebench -count 5 -alloc`:

name       old time/op       new time/op       delta
Template         220ms ± 8%        228ms ±18%    ~     (p=0.548 n=5+5)
Unicode          119ms ±11%        113ms ± 5%    ~     (p=0.056 n=5+5)
GoTypes          684ms ± 6%        677ms ± 3%    ~     (p=0.841 n=5+5)
Compiler         3.19s ± 7%        3.01s ± 1%    ~     (p=0.095 n=5+5)
SSA              7.92s ± 8%        7.79s ± 1%    ~     (p=0.690 n=5+5)
Flate            141ms ± 7%        139ms ± 4%    ~     (p=0.548 n=5+5)
GoParser         173ms ±12%        171ms ± 4%    ~     (p=1.000 n=5+5)
Reflect          417ms ± 5%        411ms ± 3%    ~     (p=0.548 n=5+5)
Tar              205ms ± 5%        198ms ± 2%    ~     (p=0.690 n=5+5)
XML              232ms ± 4%        229ms ± 4%    ~     (p=0.690 n=5+5)
StdCmd           28.7s ± 5%        28.2s ± 2%    ~     (p=0.421 n=5+5)

name       old user-time/op  new user-time/op  delta
Template         269ms ± 4%        265ms ± 5%    ~     (p=0.421 n=5+5)
Unicode          153ms ± 7%        149ms ± 3%    ~     (p=0.841 n=5+5)
GoTypes          850ms ± 7%        862ms ± 4%    ~     (p=0.841 n=5+5)
Compiler         4.01s ± 5%        3.86s ± 0%    ~     (p=0.190 n=5+4)
SSA              10.9s ± 4%        10.8s ± 2%    ~     (p=0.548 n=5+5)
Flate            166ms ± 7%        167ms ± 6%    ~     (p=1.000 n=5+5)
GoParser         204ms ± 8%        206ms ± 7%    ~     (p=0.841 n=5+5)
Reflect          514ms ± 5%        508ms ± 4%    ~     (p=0.548 n=5+5)
Tar              245ms ± 6%        244ms ± 3%    ~     (p=0.690 n=5+5)
XML              280ms ± 4%        278ms ± 4%    ~     (p=0.841 n=5+5)

name       old alloc/op      new alloc/op      delta
Template        37.9MB ± 0%       37.9MB ± 0%    ~     (p=0.841 n=5+5)
Unicode         28.8MB ± 0%       28.8MB ± 0%    ~     (p=0.841 n=5+5)
GoTypes          113MB ± 0%        113MB ± 0%    ~     (p=0.151 n=5+5)
Compiler         468MB ± 0%        468MB ± 0%  -0.01%  (p=0.032 n=5+5)
SSA             1.50GB ± 0%       1.50GB ± 0%    ~     (p=0.548 n=5+5)
Flate           24.4MB ± 0%       24.4MB ± 0%    ~     (p=1.000 n=5+5)
GoParser        30.7MB ± 0%       30.7MB ± 0%    ~     (p=1.000 n=5+5)
Reflect         76.5MB ± 0%       76.5MB ± 0%    ~     (p=0.548 n=5+5)
Tar             38.9MB ± 0%       38.9MB ± 0%    ~     (p=0.222 n=5+5)
XML             41.6MB ± 0%       41.6MB ± 0%    ~     (p=0.548 n=5+5)

name       old allocs/op     new allocs/op     delta
Template          382k ± 0%         382k ± 0%  +0.01%  (p=0.008 n=5+5)
Unicode           343k ± 0%         343k ± 0%    ~     (p=0.841 n=5+5)
GoTypes          1.19M ± 0%        1.19M ± 0%  +0.01%  (p=0.008 n=5+5)
Compiler         4.53M ± 0%        4.53M ± 0%  +0.03%  (p=0.008 n=5+5)
SSA              12.4M ± 0%        12.4M ± 0%  +0.00%  (p=0.008 n=5+5)
Flate             235k ± 0%         235k ± 0%    ~     (p=0.079 n=5+5)
GoParser          318k ± 0%         318k ± 0%    ~     (p=0.730 n=5+5)
Reflect           978k ± 0%         978k ± 0%    ~     (p=1.000 n=5+5)
Tar               393k ± 0%         393k ± 0%    ~     (p=0.056 n=5+5)
XML               405k ± 0%         405k ± 0%    ~     (p=0.548 n=5+5)

name       old text-bytes    new text-bytes    delta
HelloSize        672kB ± 0%        672kB ± 0%    ~     (all equal)
CmdGoSize       7.12MB ± 0%       7.12MB ± 0%    ~     (all equal)

name       old data-bytes    new data-bytes    delta
HelloSize        133kB ± 0%        133kB ± 0%    ~     (all equal)
CmdGoSize        390kB ± 0%        390kB ± 0%    ~     (all equal)

name       old exe-bytes     new exe-bytes     delta
HelloSize       1.07MB ± 0%       1.07MB ± 0%    ~     (all equal)
CmdGoSize       11.2MB ± 0%       11.2MB ± 0%    ~     (all equal)

Passes toolstash compare.

For #22662.

Change-Id: I19edb53dd9675af57f7122cb7dba2a6d8bdcc3da
Reviewed-on: https://go-review.googlesource.com/94515
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-02-26 18:27:15 +00:00
Ian Lance Taylor
41d3d153eb cmd/compile: permit go:cgo_import_dynamic anywhere
It's used on Solaris to import symbols from shared libraries, e.g., in
golang.org/x/sys/unix and golang.org/x/net/internal/socket.
We could use a different directive but that would require build tags
in all the places that use it.

Updates #23672
Updates #23749

Change-Id: I47fcf72a6d2862e304204705979c2056c2f78ec5
Reviewed-on: https://go-review.googlesource.com/94018
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-02-16 03:31:46 +00:00
Russ Cox
1dcb5836ad cmd/go: accept only limited compiler and linker flags in #cgo directives
Both gcc and clang accept an option -fplugin=code.so to load
a plugin from the ELF shared object file code.so.
Obviously that plugin can then do anything it wants
during the build. This is contrary to the goal of "go get"
never running untrusted code during the build.
(What happens if you choose to run the result of
the build is your responsibility.)

Disallow this behavior by only allowing a small set of
known command-line flags in #cgo CFLAGS directives
(and #cgo LDFLAGS, etc).

The new restrictions can be adjusted by the environment
variables CGO_CFLAGS_ALLOW, CGO_CFLAGS_DISALLOW,
and so on. See the documentation.

In addition to excluding cgo-defined flags, we also have to
make sure that when we pass file names on the command
line, they don't look like flags. So we now refuse to build
packages containing suspicious file names like -x.go.

A wrinkle in all this is that GNU binutils uniformly accept
@foo on the command line to mean "if the file foo exists,
then substitute its contents for @foo in the command line".
So we must also reject @x.go, flags and flag arguments
beginning with @, and so on.

Fixes #23672, CVE-2018-6574.

Change-Id: I59e7c1355155c335a5c5ae0d2cf8fa7aa313940a
Reviewed-on: https://team-review.git.corp.google.com/209949
Reviewed-by: Ian Lance Taylor <iant@google.com>
2018-02-07 15:35:57 +00:00