Commit graph

862 commits

Author SHA1 Message Date
Jeremy Faller
664accc7be [dev.link] cmd/link: remove sym.Symbols from loader
Pushing the wavefront forward requires a single source of truth for
symbols. This CL removes sym.Symbols from the loader for host object
loaders, allowing us to have the single source of truth for symbols be
the loader, not some strange combination of sym.Symbols and the loader.

Change-Id: Ib8ff0117ebe6040fade346047535ff342d781081
Reviewed-on: https://go-review.googlesource.com/c/go/+/209217
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-12-12 20:57:54 +00:00
Than McIntosh
29f886e427 [dev.link] cmd/link: avoid allsyms loop in initarray setup
In the linker's symtab() function, avoid looping over the context's
Syms.Allsyms array to locate the entry symbol when setting up the init
array section; do an explicit ABI0 symbol lookup instead. This is a
minor efficiency tweak / code cleanup.

Fixes #20205.

Change-Id: I2ebc17a3cb2cd63e9f5052bc80f1b0ac72c960e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/209838
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-04 18:39:51 +00:00
Cherry Zhang
27c0aeee3f [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: I4a73cd4bac2f4d35d2c07c39f2bc01c6075fadab
2019-11-15 16:33:22 -05:00
Ville Skyttä
440f7d6404 all: fix a bunch of misspellings
Change-Id: I5b909df0fd048cd66c5a27fca1b06466d3bcaac7
GitHub-Last-Rev: 778c5d2131
GitHub-Pull-Request: golang/go#35624
Reviewed-on: https://go-review.googlesource.com/c/go/+/207421
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-15 21:04:43 +00:00
Elias Naur
d856e05d64 cmd/link/internal/ld: set MachO platform to PLATFORM_IOS on iOS
CL 205340 changed the linker to skip certain flags when linking
for iOS. However, for host linking on iOS (such as on the Corellium
darwin/arm64 builders) the MachO platform defaults to PLATFORM_MACOS,
and the flags are not skipped.

Avoids warnings such as

    ld: warning: -no_pie ignored for arm64

Updates #32963

Change-Id: Ib6b4c2375fd14cf89410bf5ff1537b692b7a1c15
Reviewed-on: https://go-review.googlesource.com/c/go/+/206337
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-15 21:02:07 +00:00
Elias Naur
d3c2b1f176 cmd/link/internal/ld,misc/cgo/testcshared: don't -fuse-ld=gold on Android
The NDK is switching to ldd, and will stop including the gold linker.

Change-Id: If74168017c9874134b34010906ab1d94001528b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/206840
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-14 17:43:48 +00:00
Joel Sing
bf5f64107a cmd/internal/objabi,cmd/link: initial linker support for riscv64
Provide initial linker support for riscv64.

Based on riscv-go port.

Updates #27532

Change-Id: I8a881ce41cd49efef0358bad9171d4d18aaf7ab2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204624
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-13 08:07:51 +00:00
Cherry Zhang
62dfb27827 [dev.link] cmd/link: delete old object file format support
There are more cleanups to do, but I want to keep this CL mostly
a pure deletion.

Change-Id: I30f4891a2ea54545fd6b83041746ab65895537e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/206558
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-11-12 19:59:04 +00:00
Jeremy Faller
aacffb6576 [dev.link] cmd/link: remove old objs from host obj loaders
Change-Id: I9ee853e37090ea015aba817d9aeb2f4cc9ff5a1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/206539
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-11 20:19:37 +00:00
Than McIntosh
795e8c2317 cmd/go: address DWARF linker issues with -buildmode=plugin on Darwin
Assorted fixups in the linker needed to enable turning back on
DWARF generation when building plugins for Darwin. Includes:

 - don't suppress import of runtime/cgo in the linker for
   Darwin if we are linking in plugin mode

 - in calcCompUnitRanges handle the case where we encounter
   linker-generated functions that have no associated Unit (and
   also have no DWARF)

 - generalize a guard in relocsym() include so as to avoid
   triggering a spurious error on go.info symbols in plugin mode

Updates #21647.
Updates #27502.

Change-Id: I317fea97bef2f3461e31498e63f9fd6d8b8f4b23
Reviewed-on: https://go-review.googlesource.com/c/go/+/182959
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-11 17:37:48 +00:00
Cherry Zhang
8cdb769b08 [dev.link] cmd: reenable newobj mode by default
Change-Id: I6e820a77c516363f350c2aad5bc17a2d7e7501e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/206457
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-11 16:39:35 +00:00
Gerrit Code Review
bababde766 Merge "cmd: merge branch 'dev.link' into master" 2019-11-08 20:24:43 +00:00
Joel Sing
a2b1dc863f cmd/link,cmd/internal/objabi: factor out direct call identification
Factor out the direct CALL identification code from objabi.IsDirectJump and
use this in two places that have separately maintained lists of reloc types.
Provide an objabi.IsDirectCallOrJump function that implements the original
behaviour of objabi.IsDirectJump.

Change-Id: I48131bae92b2938fd7822110d53df0b4ffb35766
Reviewed-on: https://go-review.googlesource.com/c/go/+/196577
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-06 17:03:51 +00:00
Elias Naur
1bd974eeb7 cmd/link/internal/ld: omit bitcode-incompatible flags on iOS simulator
The -Wl,-headerpad, -Wl,-no_pie, -Wl,-pagezero_size flags are
incompatible with the bitcode-related flags used for iOS.

We already omitted the flags on darwin/arm and darwin/arm64; this change
omits the flags on all platforms != macOS so that building for the iOS
simulator works.

Updates #32963

Change-Id: Ic9af0daf01608f5ae0f70858e3045e399de7e95b
Reviewed-on: https://go-review.googlesource.com/c/go/+/205340
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-06 14:56:38 +00:00
Clément Chigot
6dc250f456 cmd/link: fix the size of typerel.* with c-archive buildmode
With buildmode=c-archive, "runtime.types" type isn't STYPE but
STYPERELRO.
On AIX, this symbol is present in the symbol table and not under
typerel.* outersymbol. Therefore, the size of typerel.* must be adapted.

Fixes #35342

Change-Id: Ib982c6557d9b41bc3d8775e4825650897f9e0ee6
Reviewed-on: https://go-review.googlesource.com/c/go/+/205338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-06 13:55:04 +00:00
Cherry Zhang
bbae923d20 cmd: merge branch 'dev.link' into master
In the dev.link branch we implemented the new object file format
and (part of) the linker improvements described in
https://golang.org/s/better-linker

The new object file is index-based and provides random access.
The linker maps the object files into read-only memory, and
access symbols on-demand using indices, as opposed to reading
all object files sequentially into the heap with the old format.

The linker carries symbol informations using indices (as opposed
to Symbol data structure). Symbols are created after the
reachability analysis, and only created for reachable symbols.
This reduces the linker's memory usage.

Linking cmd/compile, it creates ~25% fewer Symbols, and reduces
memory usage (inuse_space) by ~15%. (More results from Than.)

Currently, both the old and new object file formats are supported.
The old format is used by default. The new format can be turned
on by using the compiler/assembler/linker's -newobj flag. Note
that the flag needs to be specified consistently to all
compilations, i.e.

go build -gcflags=all=-newobj -asmflags=all=-newobj -ldflags=-newobj

Change-Id: Ia0e35306b5b9b5b19fdc7fa7c602d4ce36fa6abd
2019-11-05 14:57:48 -05:00
Cherry Zhang
9cf6c65ca3 [dev.link] cmd: default to old object file format
Flip back to the old object files for Go 1.14.

Change-Id: I4ad499460fb7156b63fc63e9c6ea4f7099e20af2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204098
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 19:51:52 +00:00
Matthew Dempsky
e341e93c51 cmd/compile, cmd/link: add coverage instrumentation for libfuzzer
This CL adds experimental coverage instrumentation similar to what
github.com/dvyukov/go-fuzz produces in its -libfuzzer mode. The
coverage can be enabled by compiling with -d=libfuzzer. It's intended
to be used in conjunction with -buildmode=c-archive to produce an ELF
archive (.a) file that can be linked with libFuzzer. See #14565 for
example usage.

The coverage generates a unique 8-bit counter for each basic block in
the original source code, and emits an increment operation. These
counters are then collected into the __libfuzzer_extra_counters ELF
section for use by libFuzzer.

Updates #14565.

Change-Id: I239758cc0ceb9ca1220f2d9d3d23b9e761db9bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/202117
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05 00:00:36 +00:00
Cherry Zhang
d9ee4b2859 cmd/link/internal/ld: fix TestArchiveBuildInvokeWithExec
TestArchiveBuildInvokeWithExec is failing on darwin due to
duplicated symbols, because the C definition (int fortytwo;) is
copied to two generated cgo sources. In fact, this test is about
building c-archive, but doesn't need to import "C". Removed the
"C" import.

Change-Id: I3a17546e01272a7ae37e6417791ab949fb44597e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205278
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-04 21:38:36 +00:00
Than McIntosh
bf7e55b618 cmd/link: directly exec archive command if external tmpdir
When linking a Go archive, if the archiver invocation is the very last
thing that needs to happen in the link (no "atexit" cleanups required
remove the locally created tmpdir) then call syscall.Exec to invoke
the archiver command instead of the usual exec.Command. This has the
effect of reducing peak memory use for the linker overall, since we
don't be holding onto all of the linker's live memory while the
archiver is running.

Change-Id: Ibbe22d8d67a70cc2a4f91c68aab56d19fb77c393
Reviewed-on: https://go-review.googlesource.com/c/go/+/203821
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-04 18:51:13 +00:00
Cherry Zhang
d3660e8ef8 cmd/link: enable PIE on darwin/arm
We used to pass -no_pie to external linker on darwin/arm, which
is incompatible with -fembed-bitcode. CL 201358 attempted to
remove the -no_pie flag, but it resulted the darwin linker to
complain about absolute addressing in TEXT segment.

On darwin/arm, we already get away from absolute addressing in
the TEXT section. The complained absolute addressing is in
RODATA, which was embedded in the TEXT segment. This CL moves
RODATA to the DATA segment, like what we already did on ARM64
and on AMD64 in c-archive/c-shared buildmodes for the same reason.
So there is no absolute addressing in the TEXT segment, which
allows us to remove -no_pie flag.

Fixes #35252.
Updates #32963.

Change-Id: Id6e3a594cb066d257d4f58fadb4a3ee4672529f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/205060
Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-11-04 15:18:34 +00:00
Cherry Zhang
db75205a9b [dev.link] cmd/link: restore -strictdups flag in newobj mode
Change-Id: I93ad769595fa343400afa342af12e1445abff084
Reviewed-on: https://go-review.googlesource.com/c/go/+/204918
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-03 20:19:00 +00:00
Cherry Zhang
ab4a71fca7 [dev.link] cmd/link: use the start of compilation unit for R_ADDRCUOFF
R_ADDRCUOFF refers to the offset from the start of the
compilation unit. Now that we could have multiple compilation
units per package, we should use the start of the compilation
unit, instead of the start of the package.

Change-Id: I3d6a3c5a5c47c9ccf88091ed05413ced826f9934
Reviewed-on: https://go-review.googlesource.com/c/go/+/204828
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-03 19:43:59 +00:00
Cherry Zhang
c6621d9241 [dev.link] cmd/link: restore -dumpdep in new deadcode pass
Change-Id: I2e52206a95c9463df5661664726a8b3bbf3ad1fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/204825
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-03 04:57:50 +00:00
Cherry Zhang
36dbdbe9bd [dev.link] cmd/link: use min-heap for work queue for better locality
In the deadcode pass, we use a work queue for the flood algorithm.
Currently this is a simple LIFO queue. In this order, there is
poor locality in accessing object files.

Since the global indices are assigned in package DAG order, edges
are mostly either within a package or from a smaller index to a
larger one. (With named symbols, there can be backward edges, but
shouldn't be too many.) Using a min-heap for the work queue, we
access all symbols in one object, then move to next one. It
rarely needs to revisit an object that is already visted. This
should result in better locality.

Benchmark result from Than (thanks!):

name                      old time/op       new time/op       delta
LinkCompiler                    1.74s ±11%        1.61s ± 9%  -7.80%  (p=0.000 n=20+19)
LinkWithoutDebugCompiler        1.27s ±11%        1.15s ± 9%  -9.02%  (p=0.000 n=20+20)

Currently this uses the container/heap package, which uses
interface elements. If this allocates too much, we may consider
to hand-code the min heap.

Change-Id: I216d5291c432fe1f40b0b8f4f1b9d388807bf6c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/204438
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-11-02 19:22:12 +00:00
Cherry Zhang
24ea07d592 [dev.link] cmd/link: add internal packages at the end
Currently in the linker we load internal packges first, then the
main package, and then load imported packages following the
dependency graph. As a result, packages are loaded mostly in the
dependency order, except the internal packages. The global symbol
indices are assigned the same way.

By loading the internal packages at the end, the packages are
loaded in the dependency order, so are the global indices. This
way, a relocation edge is mostly either within a packge or a
forward edge from a smaller index to a larger one. This allows
us to use a min-heap work queue in the deadcode pass, to achieve
better spatial locality (in the next CL).

Change-Id: I01fa9b3cf0c9e9e66006040f6378a51fd78f0f39
Reviewed-on: https://go-review.googlesource.com/c/go/+/204437
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-11-02 19:21:37 +00:00
Jeremy Faller
496e4273e4 [dev.link] cmd/link: fix loadpe to work with new obj file
Change-Id: I0fe88df182f13e7f04c8de0b82e111db441a26e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204341
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-02 19:21:08 +00:00
Cherry Zhang
48a0b97902 [dev.link] cmd/link: set cgo attributes early when internal cgo linking
In newobj mode, cgo attributes are typically set later, as we
create sym.Symbols later. But when internal cgo linking, the
host object loaders still work with sym.Symbols, and the cgo
attributes need to be set for them to work properly. Therefore,
set them early. This will cause creating some Symbols eagerly,
but they are mostly host object symbols and will need to be
created anyway.

Now all cgo internal linking tests pass on ELF systems.

Change-Id: I023a4df4429acc8ebf5e185f62e6809198497a78
Reviewed-on: https://go-review.googlesource.com/c/go/+/204857
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-02 00:36:59 +00:00
Jeremy Faller
e961b26c27 [dev.link] cmd/link: fix xcoff loader for new obj format
config.go needs to be removed from this CL.

Change-Id: I04a267feeae1551bb18f6a03a725adc9db593fdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204099
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-01 15:36:44 +00:00
Than McIntosh
c0555a2a7a [dev.link] all: merge branch 'master' into dev.link
Fixed a couple of minor conflicts in lib.go and deadcode.go
relating to debug logging.

Change-Id: I58335fc42ab1f1f3409fd8354da4f26419e8fb22
2019-11-01 10:45:24 -04:00
Than McIntosh
219922e95b [dev.link] cmd/link: add support to new deadcode for field tracking
Fix up the new dead code pass to include support for populating the
ctxt "Reachparent" map, which is needed to support field tracking.
Since we don't have sym.Symbols created at the point where new dead
code runs, keep track of reachability using global symbol indices, and
then once loader.LoadFull is complete we can translate the index
mappings into symbol mappings.

The fieldtracking output is unfortunately different relative to
master, due to differences in the order in which symbols are
encountered in deadcode, but I have eyeballed the results to make sure
they look reasonable.

Change-Id: I48c7a4597f05c00f15af3bfd37fc15ab4d0017c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204342
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-01 13:54:58 +00:00
Cherry Zhang
63815923fc [dev.link] cmd/link: resolve ABI aliases for external symbols
ABI alias references in Go symbols are resolved during
loadObjFull. But for external symbols they are not resolved. If
there is a reference from an external symbol to a Go ABIInternal
symbol, this reference will be invalid as it is not resolved.

The old code resolve ABI aliases in the deadcode pass. But the
new deadcode pass doesn't do it, as it works with indices instead
of Symbols. We do this in LoadFull.

This makes all internal cgo linking tests pass on Mach-O.

Change-Id: Iac6c084c03f5ddbcc9455527800ce7ed7313f9a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204698
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-01 13:51:36 +00:00
Cherry Zhang
4a43a50791 [dev.link] cmd/link: process cgo_import_static before host object loading
In internal linking mode, we need to process cgo_import_static
directives before loading host objects, because the directive is
to tell the host object loader how to deal with imported symbols.

This should fix linking with old object files. I think there
needs some similar logic for new object files, but I'll leave
that for later.

Change-Id: Icaa286de626ea1876086dbdd015047084c92caf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/204697
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-01 03:40:20 +00:00
Jeremy Faller
396a9b98ba [dev.link] cmd/link: fix macho host objects
Small updates to macho loader, fixing some misunderstandings I had about
using the new object file format.

Change-Id: I9224b01ca327e3a087ebfa36800bd6eef6abcc80
Reviewed-on: https://go-review.googlesource.com/c/go/+/204097
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-31 15:22:28 +00:00
Jeremy Faller
0e76e2f4e5 [dev.link] cmd/link: elf host obj support w/ new obj files
Add support for elf host objects with new object file format.

Change-Id: Ic5be1953359b9b6b78d9a0b715af69763aefd227
Reviewed-on: https://go-review.googlesource.com/c/go/+/201728
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-31 13:11:51 +00:00
Cherry Zhang
5a210b5858 [dev.link] cmd/link: keep DWARF constant DIE symbols live
DWARF constant DIE symbols are not referenced by any other symbol,
but are needed by the DWARF pass, where they get linked to the
compilation unit.

Reenable gdb constant test.

Change-Id: If77a0d379d9a6f1591939345bc31b027c2567f22
Reviewed-on: https://go-review.googlesource.com/c/go/+/204397
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-31 01:09:58 +00:00
Cherry Zhang
43d25a61c1 [dev.link] cmd/link: ensure deterministic order in dynexp
dynexp is used for generating the dynamic symbol table. It is
created from a map. Sort it to ensure deterministic order.

Should fix solaris build.

Change-Id: I561b9da3a4136a7ea41139073f76c98fb069d4fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/204378
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-31 01:09:34 +00:00
Than McIntosh
851b1f40b4 [dev.link] cmd/link: add new slice interface for querying aux symbols
Add a new loader.Loader.ReadAuxSyms method that returns a slice
containing the ids of the aux symbols for a specified global symbol.
This is similar to the new interface recently added that allows you to
get back a slice of relocations (as opposed to making calls into the
loader for each one). This was idea suggested by Cherry. Compilebench
numbers:

name                      old time/op       new time/op       delta
LinkCompiler                    1.63s ± 9%        1.57s ± 7%  -3.84%  (p=0.006 n=20+20)
LinkWithoutDebugCompiler        1.15s ±11%        1.11s ±11%    ~     (p=0.108 n=20+20)

name                      old user-time/op  new user-time/op  delta
LinkCompiler                    1.99s ± 8%        2.00s ±12%    ~     (p=0.751 n=19+19)
LinkWithoutDebugCompiler        1.14s ±11%        1.19s ±21%    ~     (p=0.183 n=20+20)

Change-Id: Iab6cbe18419aaa61d9cadb3f626a4515c71f2686
Reviewed-on: https://go-review.googlesource.com/c/go/+/203501
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-31 01:09:14 +00:00
Elias Naur
f4e32aeed1 cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode
The flags -headerpad, -Wl,-no_pie and -pagezero_size are incompatible with
the -fembed-bitcode flag used by `gomobile build`. Than McIntosh
suggested we might not need the offending flags; this change removes
the flags on darwin/arm64 and -headerpad, -pagezero_size on darwin/arm.

The -Wl,-no_pie flag is left for darwin/arm because linking fails
without it:

ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in _runtime.rodata from /var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/tmp/go-link-225285265/go.o. To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie

Discussion: https://groups.google.com/d/msg/golang-dev/U1jK3xmmGAk/j0_ty46EDAAJ

I've verified the CL on the builders, built the "flappy" example from
gomobile with `gomobile build`, and verified that flappy runs on an
iPhone 5S.

Updates #32963

Change-Id: I783abc93ccf3c1d2b7ca00144b7164ba223d3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/201358
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-30 08:17:29 +00:00
John Papandriopoulos
59a6847039 cmd/link: pass-through undefined call targets in external link mode
Allows Go asm calls referencing a function in a .syso file to be
passed through to the external linker, that would have otherwise
raised a "relocation target X not defined" error in cmd/link.

Fixes #33139

Change-Id: I2a8eb6063ebcd05fac96f141acf7652cf9189766
Reviewed-on: https://go-review.googlesource.com/c/go/+/198798
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-28 22:10:06 +00:00
Koichi Shiraishi
18e9cdc70e cmd/link: use filepath.Clean rather than path.Clean
pname already translated package path into a file path on for statement.
Use same as filepath.Clean rather than path.Clean.

Change-Id: I9782e2396313abc39ac55e3fdcec8f53bcaff84e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202377
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-10-28 19:31:17 +00:00
Dmitri Goutnik
152dddee7e cmd/link: switch to ld.bfd on freebsd/arm64
Updates golang/go#35197

Change-Id: I4fd85c84475761d71d2c17e62796e0a411cf91d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/203519
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2019-10-28 09:26:37 +00:00
Jason A. Donenfeld
31bfab4ac6 cmd/link: enable DEP for Windows executables
There's no reason not to enable DEP in 2019, especially given Go's
minimum operating system level.

RELNOTE=yes

Change-Id: I9c3bbc5b05a1654876a218123dd57b9c9077b780
Reviewed-on: https://go-review.googlesource.com/c/go/+/203601
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-10-27 09:49:02 +00:00
nikita-vanyasin
c3e8a20a65 cmd/link/internal: eliminate all ld.Cputime() usages
Also a similar 'elapsed' function and its usages were deleted.

Fixes #19865.

Change-Id: Ib125365e69cf2eda60de64fa74290c8c7d1fd65a
Reviewed-on: https://go-review.googlesource.com/c/go/+/171730
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-26 05:40:14 +00:00
Cherry Zhang
d77b809df9 [dev.link] all: merge branch 'master' into dev.link
The only conflict is in cmd/internal/obj/link.go and the
resolution is trivial.

Change-Id: Ic79b760865a972a0ab68291d06386531d012de86
2019-10-25 13:41:36 -04:00
Than McIntosh
376ef734a7 [dev.link] cmd/link: rework relocation handling in new deadcode
Do a better job of reading relocations in the new deadcode pass.
Specifically, during method type processing, read relocations for the
symbol we're working on into a slice, and then pass the slice to
helper functions, as opposed to rereading relocs at each stage.

Change-Id: I95e3737ae91bb09b4da8e6ee68112ec255ceb0fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/201722
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 16:51:18 +00:00
Dan Scales
be64a19d99 cmd/compile, cmd/link, runtime: make defers low-cost through inline code and extra funcdata
Generate inline code at defer time to save the args of defer calls to unique
(autotmp) stack slots, and generate inline code at exit time to check which defer
calls were made and make the associated function/method/interface calls. We
remember that a particular defer statement was reached by storing in the deferBits
variable (always stored on the stack). At exit time, we check the bits of the
deferBits variable to determine which defer function calls to make (in reverse
order). These low-cost defers are only used for functions where no defers
appear in loops. In addition, we don't do these low-cost defers if there are too
many defer statements or too many exits in a function (to limit code increase).

When a function uses open-coded defers, we produce extra
FUNCDATA_OpenCodedDeferInfo information that specifies the number of defers, and
for each defer, the stack slots where the closure and associated args have been
stored. The funcdata also includes the location of the deferBits variable.
Therefore, for panics, we can use this funcdata to determine exactly which defers
are active, and call the appropriate functions/methods/closures with the correct
arguments for each active defer.

In order to unwind the stack correctly after a recover(), we need to add an extra
code segment to functions with open-coded defers that simply calls deferreturn()
and returns. This segment is not reachable by the normal function, but is returned
to by the runtime during recovery. We set the liveness information of this
deferreturn() to be the same as the liveness at the first function call during the
last defer exit code (so all return values and all stack slots needed by the defer
calls will be live).

I needed to increase the stackguard constant from 880 to 896, because of a small
amount of new code in deferreturn().

The -N flag disables open-coded defers. '-d defer' prints out the kind of defer
being used at each defer statement (heap-allocated, stack-allocated, or
open-coded).

Cost of defer statement  [ go test -run NONE -bench BenchmarkDefer$ runtime ]
  With normal (stack-allocated) defers only:         35.4  ns/op
  With open-coded defers:                             5.6  ns/op
  Cost of function call alone (remove defer keyword): 4.4  ns/op

Text size increase (including funcdata) for go binary without/with open-coded defers:  0.09%

The average size increase (including funcdata) for only the functions that use
open-coded defers is 1.1%.

The cost of a panic followed by a recover got noticeably slower, since panic
processing now requires a scan of the stack for open-coded defer frames. This scan
is required, even if no frames are using open-coded defers:

Cost of panic and recover [ go test -run NONE -bench BenchmarkPanicRecover runtime ]
  Without open-coded defers:        62.0 ns/op
  With open-coded defers:           255  ns/op

A CGO Go-to-C-to-Go benchmark got noticeably faster because of open-coded defers:

CGO Go-to-C-to-Go benchmark [cd misc/cgo/test; go test -run NONE -bench BenchmarkCGoCallback ]
  Without open-coded defers:        443 ns/op
  With open-coded defers:           347 ns/op

Updates #14939 (defer performance)
Updates #34481 (design doc)

Change-Id: I63b1a60d1ebf28126f55ee9fd7ecffe9cb23d1ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/202340
Reviewed-by: Austin Clements <austin@google.com>
2019-10-24 13:54:11 +00:00
Alex Brainman
e4c3925925 cmd/link: use libmsvcrt.a during internal link
When using recent versions of gcc with cgo, internal link fails with

c:\>go test debug/pe
--- FAIL: TestInternalLinkerDWARF (0.94s)
    file_test.go:394: building test executable for linktype 2 failed: exit status 2 # command-line-arguments
        runtime/cgo(.text): relocation target __acrt_iob_func not defined for ABI0 (but is defined for ABI0)
        runtime/cgo(.text): relocation target __acrt_iob_func not defined for ABI0 (but is defined for ABI0)
        runtime/cgo(.text): relocation target __acrt_iob_func not defined for ABI0 (but is defined for ABI0)
FAIL
FAIL    debug/pe        4.572s
FAIL

It appears that __acrt_iob_func is defined in libmsvcrt.a. And this
change adds libmsvcrt.a to the list of libraries always used byi
internal linker.

libmsvcrt.a also implements __imp___acrt_iob_func. So this change
also prevents rewriting __imp___acrt_iob_func name into
__acrt_iob_func, otherwise we end up with duplicate __acrt_iob_func
symbol error.

Fixes #23649

Change-Id: Ie9864cd17e907501e9a8a3672bbc33e02ca20e5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/197977
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-20 06:38:39 +00:00
Cherry Zhang
bd229936cf [dev.link] cmd/link: restore export data hash
With the previous CL, the export data will not change whether it
is compiled with -dynlink flag or not. Restore the export data
hash, and reenable plugin version check.

TODO: it may be still better to just generate a fingerprint for
each package at compile time.

Change-Id: I1f298ac97c3ab9b8d05d1c95e8be74d10ca7cd0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201720
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-17 20:28:28 +00:00
Than McIntosh
6cbf37b30b [dev.link] cmd/link: record go.itablink symbols during object file read
Change the new loader to keep a note of the set of "go.itablink.*"
symbols (using a small map), and add a method that clients can use to
query whether a given global index corresponds to a "go.itablink.*"
sym. This eliminates one instance of raw symbol name reading/matching
during new deadcode, which should produce a minor speedup.

Change-Id: I5915773a3f33c16099ccd68592dbba783d909bc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/201400
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-17 14:20:00 +00:00