Commit graph

558 commits

Author SHA1 Message Date
Russ Cox
e5407501cb [dev.boringcrypto] cmd: use notsha256 instead of md5, sha1, sha256
When we add GOEXPERIMENT=boringcrypto, the bootstrap process
will not converge if the compiler itself depends on the boringcrypto
cgo-based implementations of sha1 and sha256.

Using notsha256 avoids boringcrypto and makes bootstrap converge.
Removing md5 is not strictly necessary but it seemed worthwhile to
be consistent.

For #51940.

Change-Id: Iba649507e0964d1a49a1d16e463dd23c4e348f14
Reviewed-on: https://go-review.googlesource.com/c/go/+/402595
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29 14:23:19 +00:00
Chressie Himpel
e07d63964b [dev.boringcrypto] all: merge master into dev.boringcrypto
Change-Id: I52009bf809dda4fbcff03aa82d0ea8aa2a978fa2
2022-04-20 16:57:46 +02:00
Austin Clements
e25f46e596 cmd/link: faster algorithm for nosplit stack checking, better errors
The linker performs a global analysis of all nosplit call chains to
check they fit in the stack space ensured by splittable functions.
That analysis has two problems right now:

1. It's inefficient. It performs a top-down analysis, starting with
every nosplit function and the nosplit stack limit and walking *down*
the call graph to compute how much stack remains at every call. As a
result, it visits the same functions over and over, often with
different remaining stack depths. This approach is historical: this
check was originally written in C and this approach avoided the need
for any interesting data structures.

2. If some call chain is over the limit, it only reports a single call
chain. As a result, if the check does fail, you often wind up playing
whack-a-mole by guessing where the problem is in the one chain, trying
to reduce the stack size, and then seeing if the link works or reports
a different path.

This CL completely rewrites the nosplit stack check. It now uses a
bottom-up analysis, computing the maximum stack height required by
every function's call tree. This visits every function exactly once,
making it much more efficient. It uses slightly more heap space for
intermediate storage, but still very little in the scheme of the
overall link. For example, when linking cmd/go, the new algorithm
virtually eliminates the time spent in this pass, and reduces overall
link time:

           │   before    │                after                │
           │   sec/op    │   sec/op     vs base                │
Dostkcheck   7.926m ± 4%   1.831m ± 6%  -76.90% (p=0.000 n=20)
TotalTime    301.3m ± 1%   296.4m ± 3%   -1.62% (p=0.040 n=20)

           │    before    │                 after                  │
           │     B/op     │     B/op       vs base                 │
Dostkcheck   40.00Ki ± 0%   212.15Ki ± 0%  +430.37% (p=0.000 n=20)

Most of this time is spent analyzing the runtime, so for larger
binaries, the total time saved is roughly the same, and proportionally
less of the overall link.

If the new implementation finds an error, it redoes the analysis,
switching to preferring quality of error reporting over performance.
For error reporting, it computes stack depths top-down (like the old
algorithm), and reports *all* paths that are over the stack limit,
presented as a tree for compactness. For example, this is the output
from a simple test case from test/nosplit with two over-limit paths
from f1:

        main.f1: nosplit stack overflow
        main.f1
            grows 768 bytes, calls main.f2
                grows 56 bytes, calls main.f4
                    grows 48 bytes
                    80 bytes over limit
            grows 768 bytes, calls main.f3
                grows 104 bytes
                80 bytes over limit

While we're here, we do a few nice cleanups:

- We add a debug output flag, which will be useful for understanding
  what our nosplit chains look like and which ones are close to
  running over.

- We move the implementation out of the fog of lib.go to its own file.

- The implementation is generally more Go-like and less C-like.

Change-Id: If1ab31197f5215475559b93695c44a01bd16e276
Reviewed-on: https://go-review.googlesource.com/c/go/+/398176
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-19 15:59:26 +00:00
Austin Clements
5f625de4d0 cmd/compile,cmd/internal/obj: replace Ctxt.FixedFrameSize method with Arch field
And delete now-unused FixedFrameSize methods.

Change-Id: Id257e1647dbeb4eb4ab866c53744010c4efeb953
Reviewed-on: https://go-review.googlesource.com/c/go/+/400819
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-19 15:59:22 +00:00
Austin Clements
8af7fac5a1 internal/sys: add LR and fixed frame size to sys.Arch
Storing this information in the Arch eliminates some code duplication
between the compiler and linker. This information is entirely
determined by the Arch, so the current approach of attaching it to an
entire Ctxt is a little silly. This will also make it easier to use
this information from tests.

The next CL will be a rote refactoring to eliminate the
Ctxt.FixedFrameSize methods.

Change-Id: I315c524fa66a0ea99f63ae5a2a6fdc367d843bad
Reviewed-on: https://go-review.googlesource.com/c/go/+/400818
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-19 15:59:19 +00:00
Lasse Folger
4739b353bb [dev.boringcrypto] all: merge master into dev.boringcrypto
Change-Id: Iaf618444dd2d99721c19708df9ce2c1f35854efd
2022-04-05 18:53:15 +02:00
Than McIntosh
cc3a3519af cmd/link/internal/ld: revise recipe for ASLR enable on windows
When doing external linking on windows, the existing Go linker code
assumed that the external linker defaulted to "--no-dynamicbase" (if
no explicit option was given). This assumption doesn't hold for LLD,
which turns on "--dynamicbase" by default for 64-bit apps. Change the
linker to detect whether a more modern toolchain is in use and to
explicitly pass "--dynamicbase" either way , so as to take the
external linker default out of the equation. This also applies to the
"--high-entropy-va" option as well.

Updates #35006.

Change-Id: I3e12cf6d331c9d003e3d2bd566d45de5710588b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/384156
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-31 19:40:29 +00:00
Than McIntosh
cdee8004ab cmd/link: resolve __CTOR_LIST__/__DTOR_LIST__ refs for PE
When doing an internal link on Windows, it's possible to see
unresolved references to the symbols "__CTOR_LIST__" and/or
"__DTOR_LIST__" (which are needed in some circumstances). If these are
still unresolved at the point where we're done reading host objects,
then synthesize dummy versions of them.

Updates #35006.

Change-Id: I408bf18499bba05752710cf5a41621123bd84a3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/383836
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-31 14:57:20 +00:00
Than McIntosh
a9d13a9c23 cmd/link: read crt2.o for windows internal-linking CGO
For Windows internal linking with CGO, when using more modern
LLVM-based compilers, we may need to read in the object file "crt2.o"
so as to have a definition of "atexit" (for example when linking the
runtime/cgo test), and we also need to allow for the possibility that
a given host archive might have to be looked at more than once. The goal
here is to get all.bash working on Windows when using an up to date
mingw C compiler (including those based on clang + LLD).

This patch also adds a new "hostObject" helper routine, similar to
"hostArchive" but specific to individual object files. There is also a
change to hostArchive to modify the pseudo-package name assigned when
reading archive elements: up until this point, a package name of
"libgcc" was used (even when reading a host archive like
"libmingex.a"), which led to very confusing errors messages if symbols
were missing or there were duplicate definitions.

Updates #35006.

Change-Id: I19c17dea9cfffa9e79030fc23064c7c63a612097
Reviewed-on: https://go-review.googlesource.com/c/go/+/382838
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-31 14:08:58 +00:00
Nicolas Hillegeer
50b8f490e1 [dev.boringcrypto] all: merge master into dev.boringcrypto
Change-Id: I04d511ed8e3e7ca4a3267f226a0c3e248c0f84a9
2022-03-29 06:58:14 -07:00
Bryan C. Mills
9a932c5712 internal/buildcfg: initialize GOROOT to runtime.GOROOT
In the beginning the Go compiler was in C, and C had a function
'getgoroot' that returned GOROOT from either the environment or a
generated constant. 'getgoroot' was mechanically converted to Go
(as obj.Getgoroot) in CL 3046.

obj.Getgoroot begat obj.GOROOT. obj.GOROOT begat objabi.GOROOT,
which begat buildcfg.GOROOT.

As far as I can tell, today's buildcfg.GOROOT is functionally
identical to runtime.GOROOT(). Let's reduce some complexity by
defining it in those terms.

While we're thinking about buildcfg.GOROOT, also check whether it is
non-empty: if the toolchain is built with -trimpath, the value of
GOROOT might not be valid or meaningful if the user invokes
cmd/compile or cmd/link directly, or via a build tool other than
cmd/go that doesn't care as much about GOROOT. (As of CL 390024,
runtime.GOROOT will return the empty string instead of a bogus one
when built with -trimpath.)

For #51461.

Change-Id: I9fec020d5fa65d4aff0dd39b805f5ca93f86c36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/393155
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-18 21:52:11 +00:00
Meng Zhuo
d34287a4f6 cmd/link: default generic ABI compression for ELF
This CL change all debug dwarf headers to generic ABI
"Compression header" for ELF
(http://www.sco.com/developers/gabi/latest/ch4.sheader.html#compression_header)

Fixes #50796

Change-Id: I188625e596f11cd120dbd802ac2d79341d5eaf41
Reviewed-on: https://go-review.googlesource.com/c/go/+/380755
Trust: mzh <mzh@golangcn.org>
Run-TryBot: mzh <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-16 05:32:52 +00:00
Chressie Himpel
e14fee553a [dev.boringcrypto] all: merge master into dev.boringcrypto
Change-Id: I18dbf4f9fa7e2334fccedd862a523126cf38164e
2022-02-03 19:30:02 +01:00
Cherry Mui
c5fee935bb cmd/link: force eager binding when using plugins on darwin
When building/using plugins on darwin, we need to use flat
namespace so the same symbol from the main executable and the
plugin can be resolved to the same address. Apparently, when using
flat namespace the dynamic linker can hang at forkExec when
resolving a lazy binding. Work around it by forcing early bindings.

Fixes #38824.

Change-Id: I983aa0a0960b15bf3f7871382e8231ee244655f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/372798
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-17 16:05:31 +00:00
Cherry Mui
e9f0381a80 cmd/link: don't unmap output file at error exit
When the link exits on error it currently calls Out.Close, which
will munmap the output buffer and close the file. This may be
called in concurrent phase where other goroutines may be writing
to the output buffer. The munmap can race with the write, causing
it to write to unmapped memory and crash. This CL changes it to
just close the file without unmapping. We're exiting on error
anyway so no need to unmap.

Fixes #47816.

Change-Id: I0e89aca991bdada3d017b7d5c8efc29e46308c03
Reviewed-on: https://go-review.googlesource.com/c/go/+/363357
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-11-12 14:55:44 +00:00
Clément Chigot
01103d533a cmd/link: fix GCC startfiles names on AIX
Since GCC version 11, the 64-bit version of GCC starting files are
now suffixed by "_64" instead of being stored without suffix under
"ppc64" multilib directory.

Change-Id: Ibe53521ed24d36e5f6282e3574849b9ae11a1e9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/362594
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
2021-11-09 18:37:32 +00:00
Roland Shoemaker
ed07c49cb6 [dev.boringcrypto] all: merge master into dev.boringcrypto
Change-Id: I1aa33cabd0c55fe64994b08f8a3f7b6bbfb3282c
2021-11-05 13:13:16 -07:00
fanzha02
ae4d67c89d cmd/link: add -asan option
The -asan option causes the linker to link against the runtime/asan
package in order to use the C/C++ address sanitizer.

This CL passes tests but is not usable by itself.  The actual
runtime/asan package, and support for -asan in the go tool and the
compiler, and tests, are in separate CLs.

Updates #44853.

Change-Id: Ifc6046c1f75ba52777cbb3d937a4b66e91d5798d
Reviewed-on: https://go-review.googlesource.com/c/go/+/298610
Trust: fannie zhang <Fannie.Zhang@arm.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-22 21:48:34 +00:00
Cherry Mui
17674e2f17 cmd/internal/obj, cmd/link: move symbol alignment logic to object file writer
Change-Id: I827a9702dfa01b712b88331668434f8db94df249
Reviewed-on: https://go-review.googlesource.com/c/go/+/353569
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-10-04 22:46:23 +00:00
Cherry Mui
eb9f090d1b cmd/link: use "CC --print-prog-name" to locate tools
When building for macOS with external linking, we currently use
"xcrun" to invoke "dsymutil" and "strip" tools. That doesn't work
well for cross compilation. Use "CC --print-prog-name" to find the
tool path instead.

Fixes #47316.

Change-Id: Ib30c6494c48bfb6a505dc26fe644ef543d777076
Reviewed-on: https://go-review.googlesource.com/c/go/+/336769
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-30 19:56:06 +00:00
Cherry Mui
587b3c1192 cmd/link: access pcdata via aux symbols
Pcdata are now separate aux symbols. Read them from aux, instead
of using funcinfo.

Now we can remove pcdata fields from funcinfo.

Change-Id: Ie65e3962edecc0f39127a5f6963dc59d1f141e67
Reviewed-on: https://go-review.googlesource.com/c/go/+/352893
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-29 13:53:07 +00:00
Cherry Mui
55d22f7490 cmd/link: remove ABI aliases
It is now gone.

Change-Id: I59f68b324af706476695de2f291dd3aa5734e192
Reviewed-on: https://go-review.googlesource.com/c/go/+/351332
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 15:49:52 +00:00
Cherry Mui
98989f2a74 cmd/compile, cmd/link: avoid ABI aliases
In the past we introduced ABI aliases, in preparation for ABI
wrappers. Now that we have ABI wrappers implemented, we don't
need ABI aliases. If ABI wrappers are not enabled, ABI0 and
ABIInternal are actually identical, so we can resolve symbol
references without distinguish them. This CL does so by
normalizing ABIInternal to ABI0 at link time. This way, we no
longer need to generate ABI aliases.

This CL doesn't clean up everything related to ABI aliases, which
will be done in followup CLs.

Change-Id: I5b5db43370d29b8ad153078c70a853e3263ae6f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/351271
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 13:50:24 +00:00
Paul E. Murphy
9295723079 cmd/link: rework .TOC. handling for ppc64le
Setup .TOC. to point to the same place for all objects. Today, the linker
assumes all call relocations can use the local function entry point of
imported object files. This requires a consistent pointer across all
objects.

This intentionally computes the .TOC. pointer in all linking configurations.
In some cases the .TOC. is not used today (e.g linking position-dependent go
only code). It is harmless and simple to compute in all cases, so just
do it for easier maintenance.

Notably, .TOC. is used in some cases when static linking is requested on
ppc64le/linux:

* Position-independent C code using a PC-rel relocation against .TOC.. cgo
  generated C object files are usually compiled PIC even if the go binary
  itself is not.

* Anything which causes PLT stub generation. The stubs always generate
  a .TOC. relative relocation.

* The race detector. Today, this links in an externally compiled archive which
  contains position-independent object files.

Similarly, position-independent linking is always punted to the external
linker on ppc64 today.

Updates #21961
Fixes #15409

Change-Id: Ifd8294b9249e16ba8b92eaf876d15d162f9c61fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/304458
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-09-08 11:57:03 +00:00
Jay Conrod
742dcba7bb cmd: support space and quotes in CC and CXX
The CC and CXX environment variables now support spaces and quotes
(both double and single). This fixes two issues: first, if CC is a
single path that contains spaces (like 'c:\Program
Files\gcc\bin\gcc.exe'), that should now work if the space is quoted
or escaped (#41400). Second, if CC or CXX has multiple arguments (like
'gcc -O2'), they are now split correctly, and the arguments are passed
before other arguments when invoking the C compiler. Previously,
strings.Fields was used to split arguments, and the arguments were
placed later in the command line. (#43078).

Fixes golang/go#41400
Fixes golang/go#43078

NOTE: This change also includes a fix (CL 341929) for a test that was
broken by the original CL. Commit message for the fix is below.

[dev.cmdgo] cmd/link: fix TestBuildForTvOS

This test was broken in CL 334732 on darwin.

The test invokes 'go build' with a CC containing the arguments
-framework CoreFoundation. Previously, the go command split CC on
whitespace, and inserted the arguments after the command line when
running CC directly. Those arguments weren't passed to cgo though,
so cgo ran CC without -framework CoreFoundation (or any of the other
flags).

In CL 334732, we pass CC through to cgo, and cgo splits arguments
using str.SplitQuotedFields. So -framework CoreFoundation actually
gets passed to the C compiler. It appears that -framework flags are
only meant to be used in linking operations, so when cgo invokes clang
with -E (run preprocessor only), clang emits an error that -framework
is unused.

This change fixes the test by moving -framework CoreFoundation out of
CC and into CGO_LDFLAGS.

Change-Id: I2d5d89ddb19c94adef65982a8137b01f037d5c11
Reviewed-on: https://go-review.googlesource.com/c/go/+/334732
Trust: Jay Conrod <jayconrod@google.com>
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/341936
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-08-16 20:23:11 +00:00
Roland Shoemaker
c7e7ce5ec1 [dev.boringcrypto] all: merge commit 57c115e1 into dev.boringcrypto
Change-Id: I9e2b83c8356372034e4e3bfc6539b813e73611c9
2021-08-16 10:33:17 -07:00
Cherry Mui
c92f5ee170 cmd/link: start at address 0 when external linking
When external linking, we are creating an object file, instead of
a executable. The absolute address is irrelevant. The external
linker will set it up. Start at address 0.

Change-Id: I3a2e0b8087b328d5c3144f29ca8ba6311aa39cba
Reviewed-on: https://go-review.googlesource.com/c/go/+/319830
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-08-16 16:20:06 +00:00
Ian Lance Taylor
24e9707cbf cmd/link, cmd/cgo: support -flto in CFLAGS
The linker now accepts unrecognized object files in external linking mode.
These objects will simply be passed to the external linker.
This permits using -flto which can generate pure byte code objects,
whose symbol table the linker does not know how to read.

The cgo tool now passes -fno-lto when generating objects whose symbols
it needs to read. The cgo tool now emits matching types in different
objects, so that the lto linker does not report a mismatch.

This is based on https://golang.org/cl/293290 by Derek Parker.

For #43505
Fixes #43830
Fixes #46295

Change-Id: I6787de213417466784ddef5af8899e453b4ae1ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/322614
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
2021-06-01 19:59:18 +00:00
Filippo Valsorda
ed1f812cef [dev.boringcrypto] all: merge commit 9d0819b27c (CL 314609) into dev.boringcrypto
There used to be two BoringCrypto-specific behaviors related to cipher
suites in crypto/tls:

1. in FIPS-only mode, only a restricted set of AES ciphers is allowed

2. NOT in FIPS-only mode, AES would be prioritized over ChaCha20 even if
   AES hardware was not available

The motivation of (2) is unclear, and BoringSSL doesn't have equivalent
logic. This merge drops (2), and keeps (1). Note that the list of
FIPS-only ciphers does not have priority semantics anymore, but the
default logic still sorts them the same way as they used to be.

Change-Id: I50544011085cfa2b087f323aebf5338c0bd2dd33
2021-05-13 12:59:22 -04:00
Cherry Zhang
be1da9cdee cmd/link: unify text segment write
Currently we have two code paths of writing the text segment. They
are semantically the same:

- if we split text sections, we write all ".text" sections as
  text and the the rest as data.
- if we do not split text sections, we write the first section
  as text and the rest as data. The first section is named ".text"
  and is the only one in this case.

Unify the code.

Change-Id: Ic639eed625615be3c8a8d41f5b47e901552f587a
Reviewed-on: https://go-review.googlesource.com/c/go/+/316049
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-05-03 16:20:10 +00:00
Cherry Zhang
d7473fd907 cmd/link: mangle ABI name for shared linkage
Currently, when ABI wrappers are used, we don't use ABI aliases.
One exception is shared linkage. When loading a shared library, if
a symbol has only one ABI, and the name is not mangled, we don't
know what ABI it is, so we have to use ABI aliases.

This CL makes it always mangle ABIInternal function name in shared
linkage, so we know what ABI to choose when loading a shared
library. And we now can fully stop using ABI aliases when ABI
wrappers are used.

Change-Id: Id15d9cd72a59f391f54574710ebba7dc44cb6e23
Reviewed-on: https://go-review.googlesource.com/c/go/+/315869
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-05-01 19:17:12 +00:00
Cherry Zhang
18852e8372 cmd/link: use a two-pass approach for trampoline insertion
Currently in the linker, for trampoline insertion it does a one-pass
approach, where it assigns addresses for each function and inserts
trampolines on the go. For this to work and not to emit too many
unnecessary trampolines, the functions need to be laid out in
dependency order, so a direct call's target is always as a known
address (or known to be not too far).

This mostly works, but there are a few exceptions:
- linkname can break dependency tree and cause cycles.
- in internal linking mode, on some platforms, some calls are turned
  into calls via PLT, but the PLT stubs are inserted rather late.

Also, this is expensive in that it has to investigate all CALL
relocations.

This CL changes it to use a two-pass approach. The first pass is
just to assign addresses without inserting any trampolines, assuming
the program is not too big. If this succeeds, no extra work needs to
be done. If this fails, start over and insert trampolines for too-
far targets as well as targets with unknown addresses. This should
make it faster for small programs (most cases) and generate fewer
conservative trampolines.

Change-Id: Ib13e01f38ec6dfbef1cd446b06da33ee17bded5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/314450
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-04-29 14:11:08 +00:00
Ian Lance Taylor
24ea547a21 cmd/link: don't pass -Wl,--dynamic-linker if -static
As with -rdynamic, clang will pass -Wl,--dynamic-linker to the linker
even when linking statically. When using lld this will produce a statically
linked executable with a dynamic interpreter, which will crash at runtime.
This CL changes the linker to drop -Wl,--dynamic-linker when using -static,
as it already does with -rdynamic.

This has become more important since CL 310349, which changes the linker
to always pass a -Wl,--dynamic-linker option if the Go linker is invoked
with a -I option.

Change-Id: I68ed431064f02c70018bc0547585e5b0ebd20a41
Reviewed-on: https://go-review.googlesource.com/c/go/+/314412
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-28 15:52:58 +00:00
Paul E. Murphy
983dea90c1 cmd/link: disable plugin support if cgo is disabled
Functional plugin support requires cgo to be enabled. Disable
it if the environment has disabled cgo.

This prevents unexpected linker failures when linking large
binaries with cgo disabled which use the plugin package.

Fixes #45564

Change-Id: Ib71f0e089f7373b7b3e3cd53da3612291e7bc473
Reviewed-on: https://go-review.googlesource.com/c/go/+/314449
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-04-27 22:46:28 +00:00
Russ Cox
e5a6c5243c cmd/link: force external link for windows/arm64 cgo binaries
Change-Id: I833cac61125513fde4b93ebf14da8bab88908eab
Reviewed-on: https://go-review.googlesource.com/c/go/+/312040
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-23 21:43:06 +00:00
Russ Cox
cef3a442ea cmd/asm, cmd/link: use full objabi header
The cmd/link check of the objabi header was a bit lax because
historically the assembler has not included the full version string.
And the assembler didn't do that because it didn't have access to it:
that was buried inside the compiler.

But now that we have cmd/internal/objabi, all the tools have full
access to the expected string, and they can use it, which simplifies
the cmd/link consistency check.

Do that.

Change-Id: I33bd2f9d36c373cc3c32ff02ec6368365088b011
Reviewed-on: https://go-review.googlesource.com/c/go/+/312030
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-23 21:42:54 +00:00
Michael Hudson-Doyle
7252e1e5b6 cmd/link: convert -I foo to -Wl,--dynamic-linker,foo when externally linking
Fixes #22446

Change-Id: Id5b3fbc9cd3a7d6c4bf4e28428b8cb6d45a9ca92
Reviewed-on: https://go-review.googlesource.com/c/go/+/310349
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-04-19 23:02:15 +00:00
Russ Cox
95ed5c3800 internal/buildcfg: move build configuration out of cmd/internal/objabi
The go/build package needs access to this configuration,
so move it into a new package available to the standard library.

Change-Id: I868a94148b52350c76116451f4ad9191246adcff
Reviewed-on: https://go-review.googlesource.com/c/go/+/310731
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-04-16 19:20:53 +00:00
Austin Clements
48531da9e7 cmd/link: build dynexp symbol list directly
Currently, setCgoAttr populates the cgo_export_{static,dynamic} maps
with symbol names of exported symbols, which are then re-looked-up by
deadcode and setupdynexp, which in turn puts the re-looked-up symbols
in ctxt.dynexp. setCgoAttr already looked up the Syms, so simplify all
of this by making setCgoAttr populate ctxt.dynexp directly and
eliminating the cgo_export_{static,dynamic} maps. Recording Syms
directly also sets us up to use correct symbol versions for these
exports, rather than just assuming version 0 for all lookups.

Since setupdynexp doesn't really do any "setting up" of dynexp any
more with this change, we fold the remaining logic from setupdynexp
directly into addexport, where it has better context anyway. This also
eliminates a sorting step, since we no longer do a non-deterministic
map iteration to build the dynexp slice.

For #40724.

Change-Id: I3e1a65165268da8c2bf50d7485f2624133433260
Reviewed-on: https://go-review.googlesource.com/c/go/+/309340
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-04-13 20:07:46 +00:00
Austin Clements
6208b10d1e cmd/link: refactor setCgoAttr
setCgoAttr takes a lookup function, but there's only a single call and
setCgoAttr already has access to the lookup function passed at that
call. Simplify setCgoAttr by eliminating the lookup parameter and
calling the lookup function directly.

For #40724.

Change-Id: Ib27c0fa2b88c387e30423365f7757e3ba02cf7d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/309338
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-13 20:07:43 +00:00
Jason A. Donenfeld
c432917061 cmd/link: link libgcc archive after mingw archives
When compiling with the race detector using modern mingw, this prevents:

    libgcc(.text): relocation target ___chkstk_ms not defined

Change-Id: I2095ad09a535505b54f9ff2d3075fd20ac85e515
Reviewed-on: https://go-review.googlesource.com/c/go/+/295910
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2021-04-09 13:10:36 +00:00
Changkun Ou
972e883925 runtime/cgo: add Handle for managing (c)go pointers
A non-trivial Cgo program may need to use callbacks and interact with
go objects per goroutine. Because of the rules for passing pointers
between Go and C, such a program needs to store handles to associated
Go values. This often causes much extra effort to figure out a way to
correctly deal with: 1) map collision; 2) identifying leaks and 3)
concurrency.

This CL implements a Handle representation in runtime/cgo package, and
related methods such as Value, Delete, etc. which allows Go users can
use a standard way to handle the above difficulties.

In addition, the CL allows a Go value to have multiple handles, and the
NewHandle always returns a different handle compare to the previously
returned handles. In comparison, CL 294670 implements a different
behavior of NewHandle that returns a unique handle when the Go value is
referring to the same object.

Benchmark:
name                      time/op
Handle/non-concurrent-16  487ns ± 1%
Handle/concurrent-16      674ns ± 1%

Fixes #37033

Change-Id: I0eadb9d44332fffef8fb567c745246a49dd6d4c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/295369
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Cherry Zhang <cherryyz@google.com>
2021-04-06 23:39:42 +00:00
Tobias Klauser
565e70fcef cmd/link/internal/ld: use linkerFlagSupported to check -Qunused-arguments
Rather than checking the linker name or its path for the string "clang",
use linkerFlagSupported to determine whether the -Qunused-arguments flag
may be passed to the linker.

Fixes #45241

Change-Id: I4c1e4d4ecba4cf5823e8f39cfda5d20404ebf513
Reviewed-on: https://go-review.googlesource.com/c/go/+/304692
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-03-29 08:11:13 +00:00
Cherry Zhang
e8700f1ce6 cmd/compile, cmd/link: use weak reference in itab
When converting a type T to a non-empty interface I, we build the
itab which contains the code pointers of the methods. Currently,
this brings those methods live (if the itab is live), even if the
interface method is never used. This CL changes the itab to use
weak references, so the methods can be pruned if not otherwise
live.

Fixes #42421.

Change-Id: Iee5de2ba11d603c5a102a2ba60440d839a7f9702
Reviewed-on: https://go-review.googlesource.com/c/go/+/268479
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-03-24 14:38:53 +00:00
Paul E. Murphy
e4253cd023 cmd/link: Add section data slice to Archrelocvariant
PPC64 needs to preserve bits when applying some relocations. DS form
relocations must preserve the lower two bits, and thus needs to inspect
the section data as it streams out.

Similarly, the overflow checking requires inspecting the primary
opcode to see if the value is sign or zero extended.

The existing PPC64 code no longer works as the slice returned by
(loader*).Data is cleared as we layout the symbol and process
relocations.  This data is always the section undergoing relocation,
thus we can directly inspect the contents to preserve bits or
check for overflows.

Change-Id: I239211f7e5e96208673663b6553b3017adae7e01
Reviewed-on: https://go-review.googlesource.com/c/go/+/300555
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-18 19:15:12 +00:00
Austin Clements
eaa1ddee84 all: explode GOEXPERIMENT=regabi into 5 sub-experiments
This separates GOEXPERIMENT=regabi into five sub-experiments:
regabiwrappers, regabig, regabireflect, regabidefer, and regabiargs.
Setting GOEXPERIMENT=regabi now implies the working subset of these
(currently, regabiwrappers, regabig, and regabireflect).

This simplifies testing, helps derisk the register ABI project,
and will also help with performance comparisons.

This replaces the -abiwrap flag to the compiler and linker with
the regabiwrappers experiment.

As part of this, regabiargs now enables registers for all calls
in the compiler. Previously, this was statically disabled in
regabiEnabledForAllCompilation, but now that we can control it
independently, this isn't necessary.

For #40724.

Change-Id: I5171e60cda6789031f2ef034cc2e7c5d62459122
Reviewed-on: https://go-review.googlesource.com/c/go/+/302070
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
2021-03-18 16:51:27 +00:00
Cherry Zhang
db4adb1a9b cmd/link: print symbol versions in stack bound check
When the stack bound check fails, print the call chain with
symbol versions (along with the names). Now that we have ABI
wrappers and wrappers do consume stack space, it is clearer to
distinguish the wrappers vs. the underlying functions.

Change-Id: Id1d922e3e7934b31317f233aff3d9667b6ac90c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/302869
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-03-18 14:09:57 +00:00
Brad Fitzpatrick
18510ae88f runtime, cmd/link/internal/ld: disable memory profiling when data unreachable
If runtime.MemProfile is unreachable, default to not collecting any
memory profiling samples, to save memory on the hash table.

Fixes #42347

Change-Id: I9a4894a5fc77035fe59b1842e1ec77a1182e70c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/299671
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Keith Randall <khr@golang.org>
2021-03-09 20:09:52 +00:00
Jason A. Donenfeld
f41460145e cmd/link: recognize ARM64 PE files and relocations
For now, this only add a single relocation type, which is sufficient for
Windows resources. Later we'll see if we need more for cgo.

In order to ensure these code paths are actually tested, this expands
the rsrc tests to include all the architectures of PE objects that we
need to be recognizing, and splits things more clearly between binutils
and llvm objects, which have a slightly different layout, so that we
test both.

This CL is part of a stack adding windows/arm64
support (#36439), intended to land in the Go 1.17 cycle.

Change-Id: Ia1ee840265e9d12c0b12dd1c5d0810f8b300e557
Reviewed-on: https://go-review.googlesource.com/c/go/+/289429
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-02-26 18:15:09 +00:00
Jason A. Donenfeld
811167e2c9 cmd/link: do not pass -Bsymbolic for PE DLLs
This is only a valid option on ELF. Binutils accepts it, but LLVM
rejects it, so for Windows, it's best to just omit it.

Updates #44250.
Updates #39326.
Updates #38755.
Updates #36439.
Updates #43800.

Change-Id: Iffd2345d757f23dd737e63bd464cd412527077c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/291632
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-02-23 15:03:39 +00:00