The printing of the ptr values can mean that two dump outputs can't easily be
compared for the identical structure, so adding the "-d=dumpptrs" option to make
printing of Node pointer values be an option.
Change-Id: I0e92b02f069e9de2e6fa036a7841645d13cdd7a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/271339
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Within the frontend, we generally don't guarantee uniqueness of
anonymous types. For example, each struct type literal gets
represented by its own types.Type instance.
However, the field tracking code was using the struct type as a map
key. This broke in golang.org/cl/256457, because that CL started
changing the inlined parameter variables from using the types.Type of
the declared parameter to that of the call site argument. These are
always identical types (e.g., types.Identical would report true), but
they can be different pointer values, causing the map lookup to fail.
The easiest fix is to simply get rid of the map and instead use
Node.Opt for tracking the types.Field. To mitigate against more latent
field tracking failures (e.g., if any other code were to start trying
to use Opt on ODOT/ODOTPTR fields), we store this field
unconditionally. I also expect having the types.Field will be useful
to other frontend code in the future.
Finally, to make it easier to test field tracking without having to
run make.bash with GOEXPERIMENT=fieldtrack, this commit adds a
-d=fieldtrack flag as an alternative way to enable field tracking
within the compiler. See also #42681.
Fixes#42686.
Change-Id: I6923d206d5e2cab1e6798cba36cae96c1eeaea55
Reviewed-on: https://go-review.googlesource.com/c/go/+/271217
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Enable pie as a buildmode for linux/riscv64, along with associated tests.
Change-Id: I3fb0234d534dbeb96aa6cee6ae872304fbe02cf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/267317
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
After inlining, add a pass that looks for interface calls where we can
statically determine the interface value's concrete type. If such a
case is found, insert an explicit type assertion to the concrete type
so that escape analysis can see it.
Fixes#33160.
Change-Id: I36932c691693f0069e34384086d63133e249b06b
Reviewed-on: https://go-review.googlesource.com/c/go/+/264837
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This reverts CL 263457.
It turns out that this still missed changes to cmd/link/internal/ld/config.go
and some of these build modes also fail once cgo is enabled. Disable again for
now.
Change-Id: Iaf40d44e1551afd5b040d357f04af134f55a64a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/266317
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Joel Sing <joel@sing.id.au>
Enable c-archive, c-shared, shared and pie build modes for linux/riscv64.
Change-Id: I15a8a51b84dbbb82a5b6592aec84a7f09f0cc37f
Reviewed-on: https://go-review.googlesource.com/c/go/+/263457
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
On top of the merge, the following fixes were applied:
+ Debug["G"] changed to Debug.G, following golang.org/cl/263539.
+ issue42058a.go and issue42058b.go were skipped in
types2/stdlib_test.go. go/types does not produce errors related to
channel element size.
Change-Id: I59fc84e12a2d728ef789fdc616f7afe80e451283
This commit contains the compiler support for //go:embed lines.
The go command passes to the compiler an "embed config"
that maps literal patterns like *.txt to the set of files to embed.
The compiler then lays out the content of those files as static data
in the form of an embed.Files or string or []byte in the final object file.
The test for this code is the end-to-end test hooking up the
embed, cmd/compile, and cmd/go changes, in the next CL.
For #41191.
Change-Id: I916e57f8cc65871dc0044c13d3f90c252a3fe1bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/243944
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
gc debug flags are currently stored in a 256-long array, that is then
addressed using the ASCII numeric value of the flag itself (a quirk
inherited from the old C compiler). It is also a little wasteful,
since we only define 16 flags, and the other 240 array elements are
always empty.
This change makes Debug a struct, which also provides static checking
that we're not referencing flags that does not exist.
Change-Id: I2f0dfef2529325514b3398cf78635543cdf48fe0
Reviewed-on: https://go-review.googlesource.com/c/go/+/263539
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Function symbols defined and referenced by assembly source currently
always default to ABI0; this patch adds preliminary support for
accepting an explicit ABI selector clause for func defs/refs. This
functionality is currently only enabled when compiling runtime-related
packages (runtime, syscall, reflect). Examples:
TEXT ·DefinedAbi0Symbol<ABI0>(SB),NOSPLIT,$0
RET
TEXT ·DefinedAbi1Symbol<ABIInternal>(SB),NOSPLIT,$0
CALL ·AbiZerolSym<ABI0>(SB)
...
JMP ·AbiInternalSym<ABIInternal>(SB)
RET
Also included is a small change to the code in the compiler that reads
the symabis file emitted by the assembler.
New behavior is currently gated under GOEXPERIMENT=regabi.
Updates #27539, #40724.
Change-Id: Ia22221fe26df0fa002191cfb13bdfaaa38d7df38
Reviewed-on: https://go-review.googlesource.com/c/go/+/260477
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Providing the -G flag instructs the compiler to accept type parameters.
For now, the compiler only parses such files and then exits.
Added a new test directory (test/typeparam) and initial test case.
Port from dev.go2go branch.
Change-Id: Ic11e33a9d5f012f8def0bdae205043659562ac73
Reviewed-on: https://go-review.googlesource.com/c/go/+/261660
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This was useful for debugging failures occurring during make.bash.
The added flush also ensures that any hints in the GOSSAFUNC output
are flushed before fatal exit.
The environment variable GOSSADIR specifies where the SSA html debugging
files should be placed. To avoid collisions, each one is written into
the [package].[functionOrMethod].html, where [package] is the filepath
separator separated package name, function is the function name, and method
is either (*Type).Method, or Type.Method, as appropriate. Directories
are created as necessary to make this work.
Change-Id: I420927426b618b633bb1ffc51cf0f223b8f6d49c
Reviewed-on: https://go-review.googlesource.com/c/go/+/252338
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
xtop holds package's top-level declaration statements, but OCLOSURE
only appears in expression contexts. xtop will instead hold the
synthetic ODCLFUNC representing OCLOSURE's function body.
This CL makes the loop consistent with the later phases that only look
for ODCLFUNC nodes in xtop.
Passes toolstash-check.
Change-Id: I852a10ef1bf75bb3351e3da0357ca8b2e26aec6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/255340
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
This allows the global initializers function to go through normal
mid-end optimizations (e.g., inlining, escape analysis) like any other
function.
Updates #33485.
Change-Id: I9bcfe98b8628d1aca09b4c238d8d3b74c69010a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/254839
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
funchdr and funcbody currently assume that either (1) Curfn == nil &&
dclcontext == PEXTERN, or (2) Curfn != nil && dclcontext == PAUTO.
This is a reasonable assumption during parsing. However, these
functions end up getting used in other contexts, and not all callers
are so disciplined about Curfn/dclcontext handling.
This CL changes them to save/restore arbitrary Curfn/dclcontext pairs
instead. This is necessary for the followup CL, which pushes fninit
earlier. Otherwise, Curfn/dclcontext fall out of sync, and funchdr
panics.
Passes toolstash-check.
Updates #33485.
Change-Id: I19b1be23db1bad6475345ae5c81bbdc66291a3a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/254838
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
We're reworking pclntab generation in the linker, and with that we're
moving FuncID generation in to the compiler. Determining the FuncID is
done by a lookup on the package.function name; therefore, we need the
package whenever we make the TEXT symbols.
Change-Id: I805445ffbf2f895f06ce3a91fb09126d012bf86e
Reviewed-on: https://go-review.googlesource.com/c/go/+/245318
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
For content-addressable symbols with relocations, we build a
content hash based on its content and relocations. Depending on
the category of the referenced symbol, we choose different hash
algorithms such that the hash is globally consistent.
For now, we only support content-addressable symbols with
relocations when the current package's import path is known, so
that the symbol names are fully expanded. Otherwise, if the
referenced symbol is a named symbol whose name is not fully
expanded, the hash won't be globally consistent, and can cause
erroneous collisions. This is fine for now, as the deduplication
is just an optimization, not a requirement for correctness (until
we get to type descriptors).
Change-Id: I639e4e03dd749b5d71f0a55c2525926575b1ac30
Reviewed-on: https://go-review.googlesource.com/c/go/+/243142
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This change splits the SDWARFINFO symbol type (a generic container of
DWARF content) into separate sub-classes. The new symbol types are
SDWARFCUINFO comp unit DIE, also CU info and CU packagename syms
SDWARFCONST constant DIE
SDWARFFCN subprogram DIE (default and concrete)
SDWARFABSFCN abstract function DIE
SDWARFTYPE type DIE
SDWARFVAR global variable DIE
Advantage of doing this: in the linker there are several places where
we have to iterate over a symbol's relocations to pick out references
to specific classes of DWARF sub-symbols (for example, looking for all
abstract function DIEs referenced by a subprogram DIE, or looking at
all the type DIEs used in a subprogram DIE). By splitting SDWARFINFO
into parts clients can now look only at the relocation target's sym
type as opposed to having to materialize the target sym name, or do a
lookup.
Change-Id: I4e0ee3216d3c8f1a78bec3d296c01e95b3d025b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/234684
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Now we have ctxt.IsAsm, use that, instead of passing in a
parameter.
Change-Id: I81dedbe6459424fa9a4c2bfbd9abd83d83f3a107
Reviewed-on: https://go-review.googlesource.com/c/go/+/234492
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
We are not going to merge to master until Go 1.16 cycle. The old
object support can go now.
Change-Id: I93e6f584974c7749d0a0c2e7a96def35134dc566
Reviewed-on: https://go-review.googlesource.com/c/go/+/231918
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The new object files use indices for symbol references, instead
of names. Fundamental to the design, it requires that the
importing and imported packages have consistent view of symbol
indices. The Go command should already ensure this, when using
"go build". But in case it goes wrong, it could lead to obscure
errors like run-time crashes. It would be better to check the
index consistency at build time.
To do that, we add a fingerprint to each object file, which is
a hash of symbol indices. In the object file it records the
fingerprints of all imported packages, as well as its own
fingerprint. At link time, the linker checks that a package's
fingerprint matches the fingerprint recorded in the importing
packages, and issue an error if they don't match.
This CL does the first part: introducing the fingerprint in the
object file, and propagating fingerprints through
importing/exporting by the compiler. It is not yet used by the
linker. Next CL will do.
Change-Id: I0aa372da652e4afb11f2867cb71689a3e3f9966e
Reviewed-on: https://go-review.googlesource.com/c/go/+/229617
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
CL 201783 enable -d=checkptr when -race or -msan is specified
everywhere but windows.
But, now that all unsafe pointer conversions in the standard
library are fixed, enable -d=checkptr even on windows.
Updates #34964
Updates #34972
Change-Id: Id912fa83b0d5b46c6f1c134c742fd94d2d185835
Reviewed-on: https://go-review.googlesource.com/c/go/+/227003
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We still disallow inlining for an immediately-recursive function, but allow
inlining if a function is in a recursion chain.
If all functions in the recursion chain are simple, then we could inline
forever down the recursion chain (eventually running out of stack on the
compiler), so we add a map to keep track of the functions we have
already inlined at a call site. We stop inlining when we reach a
function that we have already inlined in the recursive chain. Of course,
normally the inlining will have stopped earlier, because of the cost
function.
We could also limit the depth of inlining by a simple count (say, limit
max inlining of 10 at any given site). Would that limit other
opportunities too much?
Added a test in test/inline.go. runtime.BenchmarkStackCopyNoCache() is
also already a good test that triggers the check to stop inlining
when we reach the start of the recursive chain again.
For the bent benchmark suite, the performance improvement was mostly not
statistically significant, but the geomean averaged out to: -0.68%. The text size
increase was less than .1% for all bent benchmarks. The cmd/go text size increase
was 0.02% and the cmd/compile text size increase was .1%.
Fixes#29737
Change-Id: I892fa84bb07a947b3125ec8f25ed0e508bf2bdf5
Reviewed-on: https://go-review.googlesource.com/c/go/+/226818
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The only merge conflict is the addition of -spectre flag on
master and the addition of -go115newobj flag on dev.link.
Resolved trivially.
Change-Id: I5b46c2b25e140d6c3d8cb129acbd7a248ff03bb9
Add back the newobj flag, renamed to go115newobj, for feature
gating. The flag defaults to true.
This essentially reverts CL 206398 as well as CL 220060.
The old object format isn't working yet. Will fix in followup CLs.
Change-Id: I1ace2a9cbb1a322d2266972670d27bda4e24adbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/224623
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This commit extends the -spectre flag to cmd/asm and adds
a new Spectre mitigation mode "ret", which enables the use
of retpolines.
Retpolines prevent speculation about the target of an indirect
jump or call and are described in more detail here:
https://support.google.com/faqs/answer/7625886
Change-Id: I4f2cb982fa94e44d91e49bd98974fd125619c93a
Reviewed-on: https://go-review.googlesource.com/c/go/+/222661
Reviewed-by: Keith Randall <khr@golang.org>
This commit adds a new cmd/compile flag -spectre,
which accepts a comma-separated list of possible
Spectre mitigations to apply, or the empty string (none),
or "all". The only known mitigation right now is "index",
which uses conditional moves to ensure that x86-64 CPUs
do not speculate past index bounds checks.
Speculating past index bounds checks may be problematic
on systems running privileged servers that accept requests
from untrusted users who can execute their own programs
on the same machine. (And some more constraints that
make it even more unlikely in practice.)
The cases this protects against are analogous to the ones
Microsoft explains in the "Array out of bounds load/store feeding ..."
sections here:
https://docs.microsoft.com/en-us/cpp/security/developer-guidance-speculative-execution?view=vs-2019#array-out-of-bounds-load-feeding-an-indirect-branch
Change-Id: Ib7532d7e12466b17e04c4e2075c2a456dc98f610
Reviewed-on: https://go-review.googlesource.com/c/go/+/222660
Reviewed-by: Keith Randall <khr@golang.org>
It has been a while we have not done this.
Merge conflict resolution:
- deleted/rewritten code modified on master
- CL 214286, ported in CL 217317
(cmd/internal/obj/objfile.go)
- CL 210678, it already includes a fix to new code
(cmd/link/internal/ld/deadcode.go)
- CL 209317, applied in this CL
(cmd/link/internal/loadelf/ldelf.go)
Change-Id: Ie927ea6a1d69ce49e8d03e56148cb2725e377876
Change the type printer to take a map of types that we're currently
printing. When we happen upon a type that we're already in the middle
of printing, print a reference to it instead.
A reference to another type is built using the offset of the first
byte of that type's string representation in the result. To facilitate
that computation (and it's probably more efficient, regardless), we
print the type to a buffer as we go, and build the string at the end.
It would be nice to use string.Builder instead of bytes.Buffer, but
string.Builder wasn't around in Go 1.4, and we'd like to bootstrap
from that version.
Fixes#29312
Change-Id: I49d788c1fa20f770df7b2bae3b9979d990d54803
Reviewed-on: https://go-review.googlesource.com/c/go/+/214239
Reviewed-by: Robert Griesemer <gri@golang.org>
Bring in Than's fix of #35779.
The only merge conflict is cmd/link/internal/loadelf/ldelf.go,
with a modification-deletion conflict.
Change-Id: Id2fcfd2094a31120966a6ea9c462b4ec76646b10
There are more cleanups to do, but I want to keep this CL mostly
a pure deletion.
Change-Id: Icd2ff0a4b648eb4adf3d29386542617e49620818
Reviewed-on: https://go-review.googlesource.com/c/go/+/206398
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is intended to allow IDEs to note where the optimizer
was not able to improve users' code. There may be other
applications for this, for example in studying effectiveness
of optimizer changes more quickly than running benchmarks,
or in verifying that code changes did not accidentally disable
optimizations in performance-critical code.
Logging of nilcheck (bad) for amd64 is implemented as
proof-of-concept. In general, the intent is that optimizations
that didn't happen are what will be logged, because that is
believed to be what IDE users want.
Added flag -json=version,dest
Check that version=0. (Future compilers will support a
few recent versions, I hope that version is always <=3.)
Dest is expected to be one of:
/path (or \path in Windows)
will create directory /path and fill it w/ json files
file://path
will create directory path, intended either for
I:\dont\know\enough\about\windows\paths
trustme_I_know_what_I_am_doing_probably_testing
Not passing an absolute path name usually leads to
json splattered all over source directories,
or failure when those directories are not writeable.
If you want a foot-gun, you have to ask for it.
The JSON output is directed to subdirectories of dest,
where each subdirectory is net/url.PathEscape of the
package name, and each for each foo.go in the package,
net/url.PathEscape(foo).json is created. The first line
of foo.json contains version and context information,
and subsequent lines contains LSP-conforming JSON
describing the missing optimizations.
Change-Id: Ib83176a53a8c177ee9081aefc5ae05604ccad8a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/204338
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Previously langSupported applied -lang as though it's a global
restriction, but it's actually a per-package restriction. This CL
fixes langSupported to take a *types.Pkg parameter to reflect this and
updates its callers accordingly.
This is relevant for signed shifts (added in Go 1.12), because they
can be inlined into a Go 1.11 package; and for overlapping interfaces
(added in Go 1.13), because they can be exported as part of the
package's API.
Today we require all Go packages to be compiled with the same
toolchain, and all uses of langSupported are for controlling
backwards-compatible features. So we can simply assume that since the
imported packages type-checked successfully, they must have been
compiled with an appropriate -lang setting.
In the future if we ever want to use langSupported to control
backwards-incompatible language changes, we might need to record the
-lang flag used for compiling a package in its export data.
Fixes#35437.
Fixes#35442.
Change-Id: Ifdf6a62ee80cd5fb4366cbf12933152506d1b36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205977
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
In the dev.link branch we implemented the new object file format
and (part of) the linker improvements described in
https://golang.org/s/better-linker
The new object file is index-based and provides random access.
The linker maps the object files into read-only memory, and
access symbols on-demand using indices, as opposed to reading
all object files sequentially into the heap with the old format.
The linker carries symbol informations using indices (as opposed
to Symbol data structure). Symbols are created after the
reachability analysis, and only created for reachable symbols.
This reduces the linker's memory usage.
Linking cmd/compile, it creates ~25% fewer Symbols, and reduces
memory usage (inuse_space) by ~15%. (More results from Than.)
Currently, both the old and new object file formats are supported.
The old format is used by default. The new format can be turned
on by using the compiler/assembler/linker's -newobj flag. Note
that the flag needs to be specified consistently to all
compilations, i.e.
go build -gcflags=all=-newobj -asmflags=all=-newobj -ldflags=-newobj
Change-Id: Ia0e35306b5b9b5b19fdc7fa7c602d4ce36fa6abd
Flip back to the old object files for Go 1.14.
Change-Id: I4ad499460fb7156b63fc63e9c6ea4f7099e20af2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204098
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL adds experimental coverage instrumentation similar to what
github.com/dvyukov/go-fuzz produces in its -libfuzzer mode. The
coverage can be enabled by compiling with -d=libfuzzer. It's intended
to be used in conjunction with -buildmode=c-archive to produce an ELF
archive (.a) file that can be linked with libFuzzer. See #14565 for
example usage.
The coverage generates a unique 8-bit counter for each basic block in
the original source code, and emits an increment operation. These
counters are then collected into the __libfuzzer_extra_counters ELF
section for use by libFuzzer.
Updates #14565.
Change-Id: I239758cc0ceb9ca1220f2d9d3d23b9e761db9bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/202117
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Generate inline code at defer time to save the args of defer calls to unique
(autotmp) stack slots, and generate inline code at exit time to check which defer
calls were made and make the associated function/method/interface calls. We
remember that a particular defer statement was reached by storing in the deferBits
variable (always stored on the stack). At exit time, we check the bits of the
deferBits variable to determine which defer function calls to make (in reverse
order). These low-cost defers are only used for functions where no defers
appear in loops. In addition, we don't do these low-cost defers if there are too
many defer statements or too many exits in a function (to limit code increase).
When a function uses open-coded defers, we produce extra
FUNCDATA_OpenCodedDeferInfo information that specifies the number of defers, and
for each defer, the stack slots where the closure and associated args have been
stored. The funcdata also includes the location of the deferBits variable.
Therefore, for panics, we can use this funcdata to determine exactly which defers
are active, and call the appropriate functions/methods/closures with the correct
arguments for each active defer.
In order to unwind the stack correctly after a recover(), we need to add an extra
code segment to functions with open-coded defers that simply calls deferreturn()
and returns. This segment is not reachable by the normal function, but is returned
to by the runtime during recovery. We set the liveness information of this
deferreturn() to be the same as the liveness at the first function call during the
last defer exit code (so all return values and all stack slots needed by the defer
calls will be live).
I needed to increase the stackguard constant from 880 to 896, because of a small
amount of new code in deferreturn().
The -N flag disables open-coded defers. '-d defer' prints out the kind of defer
being used at each defer statement (heap-allocated, stack-allocated, or
open-coded).
Cost of defer statement [ go test -run NONE -bench BenchmarkDefer$ runtime ]
With normal (stack-allocated) defers only: 35.4 ns/op
With open-coded defers: 5.6 ns/op
Cost of function call alone (remove defer keyword): 4.4 ns/op
Text size increase (including funcdata) for go binary without/with open-coded defers: 0.09%
The average size increase (including funcdata) for only the functions that use
open-coded defers is 1.1%.
The cost of a panic followed by a recover got noticeably slower, since panic
processing now requires a scan of the stack for open-coded defer frames. This scan
is required, even if no frames are using open-coded defers:
Cost of panic and recover [ go test -run NONE -bench BenchmarkPanicRecover runtime ]
Without open-coded defers: 62.0 ns/op
With open-coded defers: 255 ns/op
A CGO Go-to-C-to-Go benchmark got noticeably faster because of open-coded defers:
CGO Go-to-C-to-Go benchmark [cd misc/cgo/test; go test -run NONE -bench BenchmarkCGoCallback ]
Without open-coded defers: 443 ns/op
With open-coded defers: 347 ns/op
Updates #14939 (defer performance)
Updates #34481 (design doc)
Change-Id: I63b1a60d1ebf28126f55ee9fd7ecffe9cb23d1ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/202340
Reviewed-by: Austin Clements <austin@google.com>
It can still be manually disabled again using -d=checkptr=0.
It's also still disabled by default for GOOS=windows, because the
Windows standard library code has a lot of unsafe pointer conversions
that need updating.
Updates #34964.
Change-Id: Ie0b8b4fdf9761565e0dcb00d69997ad896ac233d
Reviewed-on: https://go-review.googlesource.com/c/go/+/201783
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When building a program that links against Go shared libraries,
it needs to reference symbols defined in the shared library. At
compile time, we don't know where the shared library boundary is.
If we reference a symbol in package p by index, and package p is
actually part of a shared library, we cannot resolve the index at
link time, as the linker doesn't see the object file of p.
So when linking against Go shared libraries, always use named
reference for now.
To do this, the compiler needs to know whether we will be linking
against Go shared libraries. The -dynlink flag kind of indicates
that (as the document says), but currently it is actually
overloaded: it is also used when building a plugin or a shared
library, which is self-contained (if -linkshared is not otherwise
specified) and could use index for symbol reference. So we
introduce another compiler flag, -linkshared, specifically for
linking against Go shared libraries. The go command will pass
this flag if its -linkshared flag is specified
("go build -linkshared").
There may be better way to handle this. For example, we can
put the symbol indices in a special section in the shared library
that the linker can read. Or we can generate some per-package
description file to include the indices. (Currently we generate
a .shlibname file for each package that is included in a shared
library, which contains the path of the library. We could
consider extending this.) That said, this CL is a stop-gap
solution. And it is no worse than the old object files.
If we were to redesign the build system so that the shared
library boundary is known at compile time, we could use indices
for symbol references that do not cross shared library boundary,
as well as doing other things better.
Change-Id: I9c02aad36518051cc4785dbe25c4b4cef8f3faeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/201818
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Escaping all unsafe.Pointer conversions for -d=checkptr seems like it
might be a little too aggressive to enable for -race/-msan mode, since
at least some tests are written to expect unsafe.Pointer conversions
to not affect escape analysis.
So instead only enable that functionality behind -d=checkptr=2.
Updates #22218.
Updates #34959.
Change-Id: I2f0a774ea5961dabec29bc5b8ebe387a1b90d27b
Reviewed-on: https://go-review.googlesource.com/c/go/+/201840
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL adds -d=checkptr as a compile-time option for adding
instrumentation to check that Go code is following unsafe.Pointer
safety rules dynamically. In particular, it currently checks two
things:
1. When converting unsafe.Pointer to *T, make sure the resulting
pointer is aligned appropriately for T.
2. When performing pointer arithmetic, if the result points to a Go
heap object, make sure we can find an unsafe.Pointer-typed operand
that pointed into the same object.
These checks are currently disabled for the runtime, and can also be
disabled through a new //go:nocheckptr annotation. The latter is
necessary for functions like strings.noescape, which intentionally
violate safety rules to workaround escape analysis limitations.
Fixes#22218.
Change-Id: If5a51273881d93048f74bcff10a3275c9c91da6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/162237
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
