Introduces a parallel OutBuf implementation, and POC on amd64. Due to
some of the weird behaviors I saw on MacOS (SIGBUS while calling msync),
I will wait for feedback to port to other architectures.
On my mac, sped up Asmb by ~78% for cmd/compile (below). Will likely
have an appreciable speedup on kubelet benchmark.
Asmb 39.1ms ±11% 8.5ms ±10% -78.17% (p=0.000 n=10+9)
TotalTime 596ms ± 2% 577ms ± 8% -3.07% (p=0.034 n=8+10)
Change-Id: Id2a2577c3f4da155d8dccc862897f43b941877ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/223742
Run-TryBot: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Add some new loader.Sym equivalents to the archSyms struct so that we
can run setArchSyms earlier in the pipeline, and add a "mode" to
setArchSyms to control whether it should create loader.Sym symbols or
their *sym.Symbol equivalents.
These change needed for a subsequent patch in which addexport() is run
earlier as well
Change-Id: I0475c9388c39f13e045dd4aa9c90eaec42624810
Reviewed-on: https://go-review.googlesource.com/c/go/+/226361
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This should restore deterministic order of dynexp, and fix
Solaris build.
Change-Id: Icb796babaa3238bff90fd8255ee9f023f2306c26
Reviewed-on: https://go-review.googlesource.com/c/go/+/225538
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
It is needed in Xcoffadddynrel. And it will be needed once we
move more things to new style (the sym.Symbol parameters will
also need to change).
Change-Id: Ie12683f9b44e21f1e6ea711bf2f4c5c32282e5b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/225203
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Populate ArchSyms with loader.Sym versions of important symbols,
so as to be able to convert the code that uses ArchSyms to the
new loader interfaces.
Change-Id: If6766f7164642a9dc2e31fcf7f9280a6e95e5d23
Reviewed-on: https://go-review.googlesource.com/c/go/+/224383
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rewrite setupdynexp to work with loader.Sym, and move the call to it
before the body of loadlibfull. After loadlibfull is complete,
construct the old *sym.Symbol version of dynexp, since not all all
clients that access this list are converted to the loader APIs.
Change-Id: I347d24958e2f3e2332fbe33f2eb6ec25cc126bdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/224382
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add an Errorf method to ErrorReporter. The hope is that we can
consolidate error handling/reporting in this helper, and eventually
do away with Link.Errorf and the global Errorf function (which
can be removed once we've eliminated enough uses of *sym.Symbol).
Change-Id: Ie1147020b8409b9c57acfd71c942b287b214afca
Reviewed-on: https://go-review.googlesource.com/c/go/+/224380
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Use symbol's Extname, instead of symbol renaming, for the mangled
names.
The old symbol Rename has an interesting logic of "merging"
symbols, when a symbol is renamed to the name of an existing
symbol. It turns out that this is needed for linking against
shared libraries, where the Go object has a reference to a symbol
with the original name, and the shared libary provides a symbol
under the mangled name. Implement this logic with the loader.
Change-Id: Ib95d7a9c93a52f8e02f4a51ac67240d6ebfc1c6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/224939
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This change moves datap from global space into the link context. Rather
than having it exist in context, we could have it returned from dodata,
and pass it as a parameter, but it is used in awkward places in the
Arch functions. Easiest for now is just keeping it in the context, until
we more formally move it to slices of loader.Sym.
This is a largely non-functional change.
Change-Id: Ica93bd857c39913ad470a61c63bc8d21704d6308
Reviewed-on: https://go-review.googlesource.com/c/go/+/222664
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This is a non-functional change, just moving things around, making
dodata a more approachable piece of code.
Change-Id: I06fa047cbef2313040a31998fa8d242ccb2fedea
Reviewed-on: https://go-review.googlesource.com/c/go/+/222662
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
I thought that only non-static symbols need TOC, but apparently
this is not true. We need to use versioned .TOC. symbols.
Partially fix AIX build. There is still another problem, but this
gets us a bit farther in make.bash.
Change-Id: I40ba4a4dd27da0fc6ab26e4a54ff76bd2930aef1
Reviewed-on: https://go-review.googlesource.com/c/go/+/223379
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Use a different mechanism to access relocations from the object
files, and use it in the stack bounds check pass. This shows some
speedup.
(linking cmd/compile)
Dostkcheck 76.9ms ± 1% 55.1ms ± 1% -28.36% (p=0.008 n=5+5)
Change-Id: I2ac42da515dccd64719fb557ffff6cdc69e4319b
Reviewed-on: https://go-review.googlesource.com/c/go/+/222240
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Remove the temporary "-newdw2" linker command line option, along with
the remainder of the legacy sym.Symbol based DWARF generation code.
Change-Id: I86c0581dd021cd4e2209ca9bc45f34037d42323c
Reviewed-on: https://go-review.googlesource.com/c/go/+/222766
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
dodata is a pile of dependencies on *Link. Pull some of these
dependencies out, and tighten up the visibility on the functions.
Change-Id: Id213f19ae7b63dd246ddb47b77ffde99f615aa80
Reviewed-on: https://go-review.googlesource.com/c/go/+/222061
Reviewed-by: Than McIntosh <thanm@google.com>
Implement a TODO in CL 220059, now that CL 220985 is in.
Change-Id: I7273a7b9e1e7c0b916780571c9c954da21366429
Reviewed-on: https://go-review.googlesource.com/c/go/+/222239
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Change the loader method AssignTextSymbolOrder to return a slice of
all reachable textp symbols, since it will be needed in second-phase
DWARF gen.
Change-Id: Iaf16ee9cf0d5266aeb0d3df596e8117263b35d8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/220985
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This is the last step requried before relocsym can go parallel.
Change-Id: Id1c1c530c2b9277917208c3767eeb29e02c17a9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/220841
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Also correct a few places I hadn't switched over to Target from *Link
context.
Change-Id: I6de6b39723762de3fa5f8d12800085421aca76fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/220840
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This CL converts the stack bound check pass to use the loader and
new symbol representation.
Change-Id: I0e4c102909db20dc2d5a367379146728862e53cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/220059
Reviewed-by: Than McIntosh <thanm@google.com>
We need to propogate Target through the relocation functions.
Change-Id: I1192d54a20a34569fd13c1511d2b21ebf2d1272e
Reviewed-on: https://go-review.googlesource.com/c/go/+/220839
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Remove the "-newdw" command line option, a temporary placeholder/gate
for the first phase of new DWARF gen, along with some portions of the
sym.Symbol based DWARF type gen code. The remainder of the old code
will be removed in a subsequent patch.
Change-Id: I7870539f020fbf05d0345d3af2acf383fae94892
Reviewed-on: https://go-review.googlesource.com/c/go/+/220984
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This CL implements windows version of -buildmode=pie code in both
cmd/go and cmd/link.
Windows executables built with -buildmode=pie set (unlike the one
built with -buildmode=exe) will have extra .reloc PE section, and
will have no IMAGE_FILE_RELOCS_STRIPPED flag set. They will also
have IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE flag set, and
IMAGE_DLLCHARACTERISTICS_HIGH_ENTROPY_VA flag set for windows/amd64.
Both cgo and non-cgo versions are implemented. And TestBuildmodePIE
is extended to test both cgo and non-cgo versions on windows and
linux.
This CL used some code from CLs 152759 and 203602.
RELNOTE=yes
Fixes#27144
Updates #35192
Change-Id: I1249e4ffbd79bd4277efefb56db321c390c0f76f
Reviewed-on: https://go-review.googlesource.com/c/go/+/214397
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The bootstrapping process (make.bash) on all other platforms use
internal linking. This change brings android/arm64 in line, fixing the
scary warning on our self-hosted Corellium builders:
warning: unable to find runtime/cgo.a
The linkmode default is changed to internal for all Android programs,
but in practice that won't matter outside our builders: using Go with
Android apps requires buildmode=c-shared which uses linkmode external.
Fixes#31343
Updates #31819
Change-Id: I3b3ada5ed69a7989e6d8e5960bbebf5e1c22aada
Reviewed-on: https://go-review.googlesource.com/c/go/+/207299
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
It was needed for the old softfloat implementation, which has
long gone.
Change-Id: Ib8d53689209a3b003d62f84c7c6047d8ec5df859
Reviewed-on: https://go-review.googlesource.com/c/go/+/220719
Reviewed-by: Than McIntosh <thanm@google.com>
Converts the portion of DWARF generation that deals with creation of
type DIEs and constant DIEs to use the new loader interfaces. Creation
of subprogram DIE and compilation unit DIE content still operates on
sym.Symbols at the moment, and happens much later in the linker.
The new code for type DIE generation is gated/guarded by the linker
flag "-newdw", which currently defaults to true. At some point in the
near future this flag should be removed, but it is handy for triage at
the moment.
This patch also includes shim code designed to run after loadlibfull()
that walks through the DIE chains and to converts loader.Sym
references back into sym.Symbol references for the remainder of the
compilation, since the second phase of DWARF has not yet been
converted.
Change-Id: I681a00fb8a1f3c37884a79b373d86411332e07c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/208230
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
With unique global indices, MakeSymbolUpdater will not change the
symbol's index. So no need to return a new index.
Change-Id: I5b4fd6a0167cc74476880bbf4382c524ecde7721
Reviewed-on: https://go-review.googlesource.com/c/go/+/219227
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Currently, the loader adds defined package symbols and
non-package symbols to the global index space object by object.
This CL changes it to add all the defined package symbols first,
then all the non-package symbols. The advantage of doing this is
that when adding package symbols, by definition they cannot be
dup to each other, so we don't need to do a name lookup when
adding them.
We still add them to the lookup table (for now), since they may
still be referenced by name (e.g. through linkname). This CL is
also a prerequisite if we want to move to not adding package
symbols to the lookup table entirely (e.g. by using
pre-generated in-file lookup table).
Also update some comments to reflect the current state.
Change-Id: Ib757e070b48a9ef6215e47dc3421fc5c055b746c
Reviewed-on: https://go-review.googlesource.com/c/go/+/219078
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Now that we have local-global index mappings, just use that for
symbol reference resolution.
Change-Id: I6bc5405853fe040ff21b624ccd8da7965d66ec8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/217065
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
No real reason to keep the old code around as the new stuff is working
just fine.
Change-Id: I5e886f8274344738663a0ead181c5d58d9f5a45f
Reviewed-on: https://go-review.googlesource.com/c/go/+/215997
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Switch the linker's Link.linksetup method to work with loader.Sym
instead of sym.Symbols. Currently enabled when the new ELF host object
loader is turned on.
Change-Id: I336cc9f36166767baac574455531e195b6f1ac57
Reviewed-on: https://go-review.googlesource.com/c/go/+/213423
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This is a rewrite of the ELF host object loader to use just the Loader
interfaces for symbol creation, without constructing sym.Symbols. At
the moment this is gated under the temporary linker command line
option "-newldelf". This version is able to get through all.bash
on linux/amd64.
Change-Id: I99f41368f75b0df9e35ef3c2cf2a702b732540c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/210779
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add SymbolBuilder helper type -- this type provides a set of methods
intended to make it easy to manipulate the content of a symbol (type,
relocations, data, etc).
Change-Id: I579bf8d04650e66d33a9780a6c2347a576c94c6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/210178
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Pushing the wavefront forward requires a single source of truth for
symbols. This CL removes sym.Symbols from the loader for host object
loaders, allowing us to have the single source of truth for symbols be
the loader, not some strange combination of sym.Symbols and the loader.
Change-Id: Ib8ff0117ebe6040fade346047535ff342d781081
Reviewed-on: https://go-review.googlesource.com/c/go/+/209217
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The NDK is switching to ldd, and will stop including the gold linker.
Change-Id: If74168017c9874134b34010906ab1d94001528b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/206840
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There are more cleanups to do, but I want to keep this CL mostly
a pure deletion.
Change-Id: I30f4891a2ea54545fd6b83041746ab65895537e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/206558
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
