Pushing the wavefront forward requires a single source of truth for
symbols. This CL removes sym.Symbols from the loader for host object
loaders, allowing us to have the single source of truth for symbols be
the loader, not some strange combination of sym.Symbols and the loader.
Change-Id: Ib8ff0117ebe6040fade346047535ff342d781081
Reviewed-on: https://go-review.googlesource.com/c/go/+/209217
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Bring in Than's fix of #35779.
The only merge conflict is cmd/link/internal/loadelf/ldelf.go,
with a modification-deletion conflict.
Change-Id: Id2fcfd2094a31120966a6ea9c462b4ec76646b10
The previous fix for this issue (CL 208479) was not general enough;
this patch revises it to handle more cases.
The problem with the original fix was that once a sym.Symbol is
created for a given static symbol and given a bogus anonymous version
of -1, we hit problems if some other non-anonymous symbol (created by
host object loading) had relocations targeting the static symbol.
In this patch instead of assigning a fixed anonymous version of -1 to
such symbols, each time loader.Create is invoked we create a new
(unique) anonymous version for the sym.Symbol, then enter the result
into the loader's extStaticSyms map, permitting it to be found in
lookups when processing relocation targets.
NB: this code will hopefully get a lot simpler once we can move host
object loading away from early sym.Symbol creation.
Updates #35779.
Change-Id: I450ff577e17549025565d355d6707a2d28a5a617
Reviewed-on: https://go-review.googlesource.com/c/go/+/208778
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When the ELF host object loader encounters a static/hidden symbol, it
creates a sym.Symbol for it but does not enter it into the sym.Symbols
lookup table. Under -newobj mode, this was not happening correctly; we
were adding the sym via loader.LookupOrCreate, which resulted in
collisions when it encountered symbols with the same name + version +
section (this can happen for "ld -r" objects).
Fixes#35779.
Change-Id: I36d40fc1efc03fc1cd8ae6b76cb6a0d2a957389c
Reviewed-on: https://go-review.googlesource.com/c/go/+/208479
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
In loadObjFull when populating the sym.Reloc vector for live symbols,
avoid using the loader.SymType() method to determine if a relocation
targets an ABI alias; since invoking loader.SymType requires a
global-to-local index translation and a read from the object file.
Instead just look at the target symbol itself, which has already been
created at this point.
Hyperkube performance numbers for this change:
name old time/op new time/op delta
RelinkHyperkube 29.9s ± 2% 29.2s ± 3% -2.42% (p=0.000 n=20+20)
RelinkWithoutDebugHyperkube 22.0s ± 3% 21.4s ± 3% -2.58% (p=0.000 n=20+20)
Change-Id: Ib7696d8760dd0485240246d6d640668fbf451d71
Reviewed-on: https://go-review.googlesource.com/c/go/+/205257
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
In newobj mode, cgo attributes are typically set later, as we
create sym.Symbols later. But when internal cgo linking, the
host object loaders still work with sym.Symbols, and the cgo
attributes need to be set for them to work properly. Therefore,
set them early. This will cause creating some Symbols eagerly,
but they are mostly host object symbols and will need to be
created anyway.
Now all cgo internal linking tests pass on ELF systems.
Change-Id: I023a4df4429acc8ebf5e185f62e6809198497a78
Reviewed-on: https://go-review.googlesource.com/c/go/+/204857
Reviewed-by: Than McIntosh <thanm@google.com>
Change the loader to do more bulk allocation when making slices of
small objects (sym.Reloc, etc) as part of creating and populating
sym.Symbols in loader.LoadFull(). This replaces a large number of
small allocations with a smaller number of large allocations,
improving performace. Compilebench numbers (linker portion) for this
change:
name old time/op new time/op delta
LinkCompiler 1.71s ±11% 1.57s ± 9% -8.35% (p=0.000 n=19+20)
LinkWithoutDebugCompiler 1.19s ±14% 1.10s ±13% -7.93% (p=0.000 n=20+19)
name old user-time/op new user-time/op delta
LinkCompiler 1.86s ±15% 1.34s ±10% -28.02% (p=0.000 n=20+20)
LinkWithoutDebugCompiler 1.05s ±14% 0.95s ± 9% -9.17% (p=0.000 n=19+20)
Hyperkube from kubernetes doesn't show any significant benefit (which
seems a little surprising).
Change-Id: Ide97f78532fb60b08bb6e4cfa097e9058f7ea8ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/203457
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix up the new dead code pass to include support for populating the
ctxt "Reachparent" map, which is needed to support field tracking.
Since we don't have sym.Symbols created at the point where new dead
code runs, keep track of reachability using global symbol indices, and
then once loader.LoadFull is complete we can translate the index
mappings into symbol mappings.
The fieldtracking output is unfortunately different relative to
master, due to differences in the order in which symbols are
encountered in deadcode, but I have eyeballed the results to make sure
they look reasonable.
Change-Id: I48c7a4597f05c00f15af3bfd37fc15ab4d0017c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204342
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
ABI alias references in Go symbols are resolved during
loadObjFull. But for external symbols they are not resolved. If
there is a reference from an external symbol to a Go ABIInternal
symbol, this reference will be invalid as it is not resolved.
The old code resolve ABI aliases in the deadcode pass. But the
new deadcode pass doesn't do it, as it works with indices instead
of Symbols. We do this in LoadFull.
This makes all internal cgo linking tests pass on Mach-O.
Change-Id: Iac6c084c03f5ddbcc9455527800ce7ed7313f9a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204698
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Small updates to macho loader, fixing some misunderstandings I had about
using the new object file format.
Change-Id: I9224b01ca327e3a087ebfa36800bd6eef6abcc80
Reviewed-on: https://go-review.googlesource.com/c/go/+/204097
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add support for elf host objects with new object file format.
Change-Id: Ic5be1953359b9b6b78d9a0b715af69763aefd227
Reviewed-on: https://go-review.googlesource.com/c/go/+/201728
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Add a new loader.Loader.ReadAuxSyms method that returns a slice
containing the ids of the aux symbols for a specified global symbol.
This is similar to the new interface recently added that allows you to
get back a slice of relocations (as opposed to making calls into the
loader for each one). This was idea suggested by Cherry. Compilebench
numbers:
name old time/op new time/op delta
LinkCompiler 1.63s ± 9% 1.57s ± 7% -3.84% (p=0.006 n=20+20)
LinkWithoutDebugCompiler 1.15s ±11% 1.11s ±11% ~ (p=0.108 n=20+20)
name old user-time/op new user-time/op delta
LinkCompiler 1.99s ± 8% 2.00s ±12% ~ (p=0.751 n=19+19)
LinkWithoutDebugCompiler 1.14s ±11% 1.19s ±21% ~ (p=0.183 n=20+20)
Change-Id: Iab6cbe18419aaa61d9cadb3f626a4515c71f2686
Reviewed-on: https://go-review.googlesource.com/c/go/+/203501
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
To speed up the loader.Loader.toLocal() method, cache the index of the
most recently accessed object file and check that object's sym range
in toLocal() before doing a full binary search over all object symbol
ranges. This speeds up relink of kubernetes/hyperkube by about 2%, and
improves compilebench (relative to the dev.link branch) by about 5%:
name old time/op new time/op delta
LinkCompiler 1.62s ± 8% 1.50s ± 9% -7.21% (p=0.000 n=20+19)
LinkWithoutDebugCompiler 1.13s ± 8% 1.09s ±12% ~ (p=0.052 n=20+20)
name old user-time/op new user-time/op delta
LinkCompiler 1.94s ±18% 1.97s ±16% ~ (p=0.813 n=19+20)
LinkWithoutDebugCompiler 1.15s ±16% 1.13s ±12% ~ (p=0.547 n=20+20)
Change-Id: Id5a8a847b533858373c0462f03972d436eda6748
Reviewed-on: https://go-review.googlesource.com/c/go/+/204337
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add a cache for the loader.Loader.resolve() method to use when
looking mapping local PkgNone symbols to global symbol indices.
This helps avoid repeated map lookups during deadcode and other
early phases of the linker when we haven't fully read in all
of object file symbols. Benchstat numbers:
name old time/op new time/op delta
LinkCompiler 1.97s ±13% 1.67s ± 8% -15.34% (p=0.000 n=20+20)
LinkWithoutDebugCompiler 1.48s ±12% 1.21s ±11% -18.14% (p=0.000 n=20+20)
name old user-time/op new user-time/op delta
LinkCompiler 2.19s ± 9% 2.04s ±17% -6.98% (p=0.002 n=19+20)
LinkWithoutDebugCompiler 1.29s ±13% 1.20s ±13% -7.70% (p=0.000 n=20+20)
Change-Id: I4b0b05c8208ee44ee9405b24774b84443e486831
Reviewed-on: https://go-review.googlesource.com/c/go/+/203197
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Compiler-generated function references (e.g. call to
runtime.newobject) appear frequently. We assign special indices
for them, so they don't need to be referenced by name.
Change-Id: I2072594cbc56c9e1037a26e4aae12e68c2436e9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/202085
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
As we no longer include static symbols into the name lookup table,
it is basically just two maps, one for ABI0, one for ABIInternal.
Just use two maps instead. It may be slightly faster to use
string-keyed maps than struct-keyed maps (still need performance
data to confirm).
For now, allow external symbols being referenced by name, as
external objects don't use index.
Change-Id: I60cedaa7346fce7535970780bc67f93c82160646
Reviewed-on: https://go-review.googlesource.com/c/go/+/201999
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Incorporate a change suggested by Cherry for CL 201721 that I missed
accidentally.
Change-Id: I65e6532e78888505573169e56bc4ace9a0f8c510
Reviewed-on: https://go-review.googlesource.com/c/go/+/202760
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Since the previous CL, we will not reference static symbols by
name. Therefore no need to put them into the name lookup table.
On Linux/ARM, in runtime/internal/atomic/sys_linux_arm.s, the
kernelcas function has a definition and a reference written in
two different forms, one with package prefix, one without. This
way, the assembler cannot know they are the same symbol, only the
linker knows. This is quite unusual, unify the names to so the
assembler can resolve it to index.
Change-Id: Ie7223097be6a3b65f3fa43ed4575da9972ef5b69
Reviewed-on: https://go-review.googlesource.com/c/go/+/201998
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
If two symbols have the same name, the old code allows either one
being dupok (preferably both, but either is ok). Currently, the
new code only works when the new symbol being dupok (or both).
Allow only old symbol being dupok as well.
One example for this is the tls_g variable on ARM64 and PPC64
when the race detector is enabled.
Should fix Linux/ARM64 build.
Change-Id: I8dd21c017e826847f13471c30dfd71bf225d8076
Reviewed-on: https://go-review.googlesource.com/c/go/+/201642
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Add a new loader.Relocs method that reads all of the relocations for a
symbol into a slice. Handy in cases where the client knows in advance
that it wants to visit all the relocations on a symbol (as opposed to
just one or two).
Change-Id: I1a420513e160c8bb4b90c9824ae8d5b5de060c15
Reviewed-on: https://go-review.googlesource.com/c/go/+/201721
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Change the new loader to keep a note of the set of "go.itablink.*"
symbols (using a small map), and add a method that clients can use to
query whether a given global index corresponds to a "go.itablink.*"
sym. This eliminates one instance of raw symbol name reading/matching
during new deadcode, which should produce a minor speedup.
Change-Id: I5915773a3f33c16099ccd68592dbba783d909bc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/201400
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add a flag bit to mark symbols in the new object file as containing Go
type information. The use of a flag eliminates the need to do symbol
name matching as part of the new dead code elimination pass, which
should produce a minor speedup.
Change-Id: Iec8700e1139e2c4e310644c0766379865d2d6f82
Reviewed-on: https://go-review.googlesource.com/c/go/+/201399
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
For the new object file format, don't tag individual symbols with a
"shared" flag, since that characteristic is better off as an attribute
of the containing object file as opposed to the individual symbol. Add
a new flags field in the object file header and put a bit in the flags
if the shared flags is in effect during compilation.
Change-Id: I2cf6d33bf7bf2fd8a7614ae0cd6ef03914777498
Reviewed-on: https://go-review.googlesource.com/c/go/+/201398
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This CL adds basic shared library support in newobj mode. This is
not complete -- there are still tests in misc/cgo/testshared
failing. But at least a simple program works, and some tests
there pass.
Add the mechanism of loading external symbols with contents.
(Before, external symbols are always contentless.) This may
potentially be also used for other host objects.
Change-Id: I68dbf71e7949cc01ebf37ea159084e798ae16925
Reviewed-on: https://go-review.googlesource.com/c/go/+/201537
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Third change of several to update the loader API to reflect the final
consensus version of the loader API as described in Cherry's doc.
This piece:
- move objfile.Loader into its own separate package, and update
clients accordingly.
This includes a few minor cleanups, including converting a couple
of loader-related functions to methods, and privatizing some of the
loader methods such as ToGlobal/ToLocal.
Change-Id: Iae20585751a45491d8b19dcffc096aadae6bbfc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/200998
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-16 13:24:55 +00:00
Renamed from src/cmd/link/internal/objfile/objfile2.go (Browse further)
