This CL is part of a set of CLs that attempt to reduce how much work the
GC must do. See the design in https://go.dev/design/74299-runtime-freegc
This CL adds runtime.freegc:
func freegc(ptr unsafe.Pointer, uintptr size, noscan bool)
Memory freed via runtime.freegc is made immediately reusable for
the next allocation in the same size class, without waiting for a
GC cycle, and hence can dramatically reduce pressure on the GC. A sample
microbenchmark included below shows strings.Builder operating roughly
2x faster.
An experimental modification to reflect to use runtime.freegc
and then using that reflect with json/v2 gave reported memory
allocation reductions of -43.7%, -32.9%, -21.9%, -22.0%, -1.0%
for the 5 official real-world unmarshalling benchmarks from
go-json-experiment/jsonbench by the authors of json/v2, covering
the CanadaGeometry through TwitterStatus datasets.
Note: there is no intent to modify the standard library to have
explicit calls to runtime.freegc, and of course such an ability
would never be exposed to end-user code.
Later CLs in this stack teach the compiler how to automatically
insert runtime.freegc calls when it can prove it is safe to do so.
(The reflect modification and other experimental changes to
the standard library were just that -- experiments. It was
very helpful while initially developing runtime.freegc to see
more complex uses and closer-to-real-world benchmark results
prior to updating the compiler.)
This CL only addresses noscan span classes (heap objects without
pointers), such as the backing memory for a []byte or string. A
follow-on CL adds support for heap objects with pointers.
If we update strings.Builder to explicitly call runtime.freegc on its
internal buf after a resize operation (but without freeing the usually
final incarnation of buf that will be returned to the user as a string),
we can see some nice benchmark results on the existing strings
benchmarks that call Builder.Write N times and then call Builder.String.
Here, the (uncommon) case of a single Builder.Write is not helped (given
it never resizes after first alloc if there is only one Write), but the
impact grows such that it is up to ~2x faster as there are more resize
operations due to more strings.Builder.Write calls:
│ disabled.out │ new-free-20.txt │
│ sec/op │ sec/op vs base │
BuildString_Builder/1Write_36Bytes_NoGrow-4 55.82n ± 2% 55.86n ± 2% ~ (p=0.794 n=20)
BuildString_Builder/2Write_36Bytes_NoGrow-4 125.2n ± 2% 115.4n ± 1% -7.86% (p=0.000 n=20)
BuildString_Builder/3Write_36Bytes_NoGrow-4 224.0n ± 1% 188.2n ± 2% -16.00% (p=0.000 n=20)
BuildString_Builder/5Write_36Bytes_NoGrow-4 239.1n ± 9% 205.1n ± 1% -14.20% (p=0.000 n=20)
BuildString_Builder/8Write_36Bytes_NoGrow-4 422.8n ± 3% 325.4n ± 1% -23.04% (p=0.000 n=20)
BuildString_Builder/10Write_36Bytes_NoGrow-4 436.9n ± 2% 342.3n ± 1% -21.64% (p=0.000 n=20)
BuildString_Builder/100Write_36Bytes_NoGrow-4 4.403µ ± 1% 2.381µ ± 2% -45.91% (p=0.000 n=20)
BuildString_Builder/1000Write_36Bytes_NoGrow-4 48.28µ ± 2% 21.38µ ± 2% -55.71% (p=0.000 n=20)
See the design document for more discussion of the strings.Builder case.
For testing, we add tests that attempt to exercise different aspects
of the underlying freegc and mallocgc behavior on the reuse path.
Validating the assist credit manipulations turned out to be subtle,
so a test for that is added in the next CL. There are also
invariant checks added, controlled by consts (primarily the
doubleCheckReusable const currently).
This CL also adds support in runtime.freegc for GODEBUG=clobberfree=1
to immediately overwrite freed memory with 0xdeadbeef, which
can help a higher-level test fail faster in the event of a bug,
and also the GC specifically looks for that pattern and throws
a fatal error if it unexpectedly finds it.
A later CL (currently experimental) adds GODEBUG=clobberfree=2,
which uses mprotect (or VirtualProtect on Windows) to set
freed memory to fault if read or written, until the runtime
later unprotects the memory on the mallocgc reuse path.
For the cases where a normal allocation is happening without any reuse,
some initial microbenchmarks suggest the impact of these changes could
be small to negligible (at least with GOAMD64=v3):
goos: linux
goarch: amd64
pkg: runtime
cpu: AMD EPYC 7B13
│ base-512M-v3.bench │ ps16-512M-goamd64-v3.bench │
│ sec/op │ sec/op vs base │
Malloc8-16 11.01n ± 1% 10.94n ± 1% -0.68% (p=0.038 n=20)
Malloc16-16 17.15n ± 1% 17.05n ± 0% -0.55% (p=0.007 n=20)
Malloc32-16 18.65n ± 1% 18.42n ± 0% -1.26% (p=0.000 n=20)
MallocTypeInfo8-16 18.63n ± 0% 18.36n ± 0% -1.45% (p=0.000 n=20)
MallocTypeInfo16-16 22.32n ± 0% 22.65n ± 0% +1.50% (p=0.000 n=20)
MallocTypeInfo32-16 23.37n ± 0% 23.89n ± 0% +2.23% (p=0.000 n=20)
geomean 18.02n 18.01n -0.05%
These last benchmark results include the runtime updates to support
span classes with pointers (which was originally part of this CL,
but later split out for ease of review).
Updates #74299
Change-Id: Icceaa0f79f85c70cd1a718f9a4e7f0cf3d77803c
Reviewed-on: https://go-review.googlesource.com/c/go/+/673695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
This change mechanically replaces all occurrences of interface{}
by 'any' (where deemed safe by the 'any' modernizer) throughout
std and cmd, minus their vendor trees.
Since this fix is relatively numerous, it gets its own CL.
Also, 'go generate go/types'.
Change-Id: I14a6b52856c3291c1d27935409bca8d5fd4242a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/719702
Commit-Queue: Alan Donovan <adonovan@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>
Symbols loaded from host files can have the N_WEAK_REF bit set,
which is used to instruct the loader to not fail if that symbol
can't be resolved.
The Go internal linker should honor this information by setting the
BIND_SYMBOL_FLAGS_WEAK_IMPORT flag in the corresponding bind table
entry.
Fixes#76023
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64-longtest,gotip-darwin-amd64_12,gotip-darwin-arm64_12,gotip-darwin-arm64_15,gotip-darwin-arm64-longtest,gotip-darwin-amd64_14
Change-Id: Id2cef247ec7a9cb08455844f3c30ff874772bb7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/713760
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This code path became useless after CL 231220.
Change-Id: I35c25368652eeb107350dcd9d1b283429ad3d5e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/714500
Auto-Submit: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Found by github.com/mdempsky/unconvert
Change-Id: I88ce10390a49ba768a4deaa0df9057c93c1164de
GitHub-Last-Rev: 3b0f7e8f74
GitHub-Pull-Request: golang/go#75974
Reviewed-on: https://go-review.googlesource.com/c/go/+/712940
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
Binutils defaults to exporting all symbols when building a Windows DLL.
To avoid that we were marking symbols with __declspec(dllexport) in
the cgo-generated headers, which instructs ld to export only those
symbols. However, that approach makes the headers hard to reuse when
importing the resulting DLL into other projects, as imported symbols
should be marked with __declspec(dllimport).
A better approach is to generate a .def file listing the symbols to
export, which gets the same effect without having to modify the headers.
Updates #30674Fixes#56994
Change-Id: I22bd0aa079e2be4ae43b13d893f6b804eaeddabf
Reviewed-on: https://go-review.googlesource.com/c/go/+/705776
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Although a comment on addmoduledata warns that it should not be
called from Go code, the linker still allow it to be accessed via
go:linkname. Using linkname on this function will cause a linker
crash when building with buildmode=plugin.
Fixes#75180
Change-Id: Ib72c367a8afaef712ca5e29b1d0a4fc98ed8f40f
Reviewed-on: https://go-review.googlesource.com/c/go/+/699655
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When we do a size fixup, we need to clone the symbol to an
external symbol so we can modify it. This includes cloning the
relocations, which includes resolving the relocations. If the
symbol being fixed has a relocation referencing a non-Go symbol,
that symbol has not yet been created, it will be resolved to an
empty symbol. Load the references first, so the referenced symbol,
even if it is a non-Go symbol, exists.
Fixes#74537.
Change-Id: I81525bd7c3e232b80eefeb0f18e13ba5331e1510
Reviewed-on: https://go-review.googlesource.com/c/go/+/687315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Currently, if there is a BSS reference and a DATA symbol
definition with the same name, we pick the DATA symbol, as it
contains the right content. In this case, if the BSS reference
has a larger size, we error out, because it is not safe to access
a smaller symbol as if it has a larger size.
Sometimes code declares a global variable in Go and defines it
in assembly with content. They are expected to be of the same
size. However, in ASAN mode, we insert a red zone for the variable
on the Go side, making it have a larger size, whereas the assembly
symbol is unchanged. This causes the Go reference (BSS) has a
larger size than the assembly definition (DATA). It results in an
error currently. This code is valid and safe, so we should permit
that.
We support this case by increasing the symbol size to match the
larger size (of the BSS side). The symbol content (from the DATA
side) is kept. In some sense, we merge the two symbols. When
loading symbols, it is not easy to change its size (as the object
file may be mapped read-only), so we add it to a fixup list, and
fix it up later after all Go symbols are loaded. This is a very
rare case, so the list should not be long.
We could limit this to just ASAN mode. But it seems okay to allow
this in general. As long as the symbol has the larger size, it is
safe to access it with the larger size.
Fixes#74314.
Change-Id: I3ee6e46161d8f59500e2b81befea11e563355a57
Reviewed-on: https://go-review.googlesource.com/c/go/+/684236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
I missed one in the previous CL.
Change-Id: I448a871523d7fb8f429b4482839d7f101ea003b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/681497
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In Go 1.25 we added a number of new linknames for standard library
internal uses. Add them to the linker's blocklist to keep them
internal.
Change-Id: I5b6051a669b7ff132a1d2c05deefbbf74701c5d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/681475
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In CL 660696, we made the linker to choose the symbol of the
larger size in case there are multiple contentless declarations of
the same symbol. We also made it emit an error in the case that
there are a contentless declaration of a larger size and a
definition with content of a smaller size. In this case, we should
choose the definition with content, but the code accesses it
through the declaration of the larger size could fall into the
next symbol, potentially causing data corruption. So we disallowed
it.
There is one spcial case, though, that some code uses a linknamed
variable declaration to reference a function in assembly, in order
to take its address. The variable is often declared as uintptr.
The function symbol is the definition, which could sometimes be
shorter. This would trigger the error case above, causing existing
code failing to build.
This CL allows it as a special case. It is still not safe to
access the variable's content. But it is actually okay to just
take its address, which the existing code often do.
Fixes#73617.
Change-Id: I467381bc5f6baa16caee6752a0a824c7185422f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/676636
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
It's not currently possible to build cgo programs that are partially
compiled with gcc-15 on riscv64 using the internal linker. There are
two reasons for this.
1. When gcc-15 compiles _cgo_export.c, which contains no actual code,
for a riscv64 target, it emits a label in the .text section called
.Letext0. This label is referred to by another section, .debug_line,
and an entry is generated in the symbol table for it. The Go linker
panics when processing the .Letext0 symbol in _cgo_export.o, as it
occurs in an empty section.
2. GCC-15 is generating additional debug symbols with the .LVUS
prefix, e.g., .LVUS33, that need to be ignored.
We fix the issue by removing the check in
cmd/link/internal/loader/loader.go that panics if we encounter a
symbol in an empty section (the comments preceding this check suggest
it's safe to remove it) and by adding .LVUS to the list of symbol
prefixes to ignore.
Fixes#72840
Change-Id: I00658b6bdd01606dde1581b5bc2f42edfc37de82
Reviewed-on: https://go-review.googlesource.com/c/go/+/668276
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
When two packages declare a variable with the same name (with
linkname at least on one side), the linker will choose one as the
actual definition of the symbol if one has content (i.e. a DATA
symbol) and the other does not (i.e. a BSS symbol). When both have
content, it is redefinition error. When neither has content,
currently the choice is sort of arbitrary (depending on symbol
loading order, etc. which are subject to change).
One use case for that is that one wants to reference a symbol
defined in another package, and the reference side just wants to
see some of the fields, so it may be declared with a smaller type.
In this case, we want to choose the one with the larger size as
the true definition. Otherwise the code accessing the larger
sized one may read/write out of bounds, corrupting the next
variable. This CL makes the linker do so.
Fixes#72032.
Change-Id: I160aa9e0234702066cb8f141c186eaa89d0fcfed
Reviewed-on: https://go-review.googlesource.com/c/go/+/660696
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
blockedLinknames was updated in CL 635676 after the lib/fips140 zip
mechanism was last tested. linknames from crypto/internal/fips140/v1.0.0
need to be allowed if they'd be allowed from crypto/internal/fips140.
Change-Id: I6a6a4656022118d4739ae14831f2ad721951c192
Reviewed-on: https://go-review.googlesource.com/c/go/+/645195
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
In Go 1.24 we added a number of new linknames for standard library
internal uses. Add them to the linker's blocklist to keep them
internal.
Change-Id: Ibb7fa095506c161604e978ae196a7cf248475b2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/635676
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Sometimes we've used the 140 suffix (GOFIPS140, crypto/fips140)
and sometimes not (crypto/internal/fips, cmd/go/internal/fips).
Use it always, to avoid having to remember which is which.
Also, there are other FIPS standards, like AES (FIPS 197), SHA-2 (FIPS 180),
and so on, which have nothing to do with FIPS 140. Best to be clear.
For #70123.
Change-Id: I33b29dabd9e8b2703d2af25e428f88bc81c7c307
Reviewed-on: https://go-review.googlesource.com/c/go/+/630115
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
The updates are:
- API documentation changes.
- Removal of the old package documentation discouraging linkname.
- Addition of new package documentation with some advice.
- Renaming of weak.Pointer.Strong -> weak.Pointer.Value.
Fixes#67552.
Change-Id: Ifad7e629b6d339dacaf2ca37b459d7f903e31bf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/628455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
For FIPS init-time code+data verification, we need to arrange to
put the FIPS symbols into contiguous regions of the executable
and then record those sections along with the expected checksum.
The cmd/internal/obj changes identify the FIPS symbols and give
them distinguished types, which the linker then places in contiguous
regions. The linker also writes out information to use at run time
to find the FIPS sections, along with the expected hash.
See cmd/internal/obj/fips.go and cmd/link/internal/ld/fips.go
for more details.
The code is disabled in this commit.
CL 625998 and 625999 adds tests.
CL 626000 enables the code.
For #69536.
Change-Id: I48da6db94bc0bea7428c43d4abcf999527bccfcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/625997
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
These will be necessary when we start using the new FIPS symbols.
Split into a separate CL so that these refactoring changes can be
tested separate from any FIPS-specific changes.
Passes golang.org/x/tools/cmd/toolstash/buildall.
Change-Id: I73e5873fcb677f1f572f0668b4dc6f3951d822bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/625996
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
This CL adds a compiler directive go:wasmexport, which applies to
a Go function and makes it an exported function of the Wasm module
being built, so it can be called directly from the host. As
proposed in #65199, parameter and result types are limited to
32-bit and 64-bit integers and floats, and there can be at most
one result.
As the Go and Wasm calling conventions are different, for a
wasmexport function we generate a wrapper function does the ABI
conversion at compile time.
Currently this CL only adds basic support. In particular,
- it only supports executable mode, i.e. the Go wasm module calls
into the host via wasmimport, which then calls back to Go via
wasmexport. Library (c-shared) mode is not implemented yet.
- only supports wasip1, not js.
- if the exported function unwinds stacks (goroutine switch, stack
growth, etc.), it probably doesn't work.
TODO: support stack unwinding, c-shared mode, js.
For #65199.
Change-Id: Id1777c2d44f7d51942c1caed3173c0a82f120cc4
Reviewed-on: https://go-review.googlesource.com/c/go/+/603055
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Randy Reddig <randy.reddig@fastly.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL does some minor refactoring of the code handling
wasmimport.
- Put the WasmImport aux reading and writing code together for
symmetry.
- Define WasmFuncType, embedded in WasmImport. WasmFuncType could
also be used (later) for wasmexport.
- Move code generation code to a separate function. The containing
function is already pretty large.
- Simplify linker code a little bit. The loader convention is to
return the 0 Sym for nonexistent symbol, instead of a separate
boolean.
No change in generated code. Passes toolstash -cmp
(GOARCH=wasm GOOS=wasip1 go build -toolexec "toolstash -cmp" -a std cmd).
Change-Id: Idc2514f84a08621333841ae4034b81130e0ce411
Reviewed-on: https://go-review.googlesource.com/c/go/+/603135
Reviewed-by: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
When a symbol is cloned to external (in order to edit it),
propagate the FromAssembly attribute, so the linker knows it is
(originally) an assembly symbol, and can treat it specially (e.g.
for stack maps).
This should fix the Linux/RISCV64 builder.
Change-Id: Icc956bcc43b79f328983a60835b05fd50f22326a
Reviewed-on: https://go-review.googlesource.com/c/go/+/587926
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
As mentioned in CL 584598, linkname is a mechanism that, when
abused, can break API integrity and even safety of Go programs.
CL 584598 is a first step to restrict the use of linknames, by
implementing a blocklist. This CL takes a step further, tightening
up the restriction by allowing linkname references ("pull") only
when the definition side explicitly opts into it, by having a
linkname on the definition (possibly to itself). This way, it is at
least clear on the definition side that the symbol, despite being
unexported, is accessed outside of the package. Unexported symbols
without linkname can now be actually private. This is similar to
the symbol visibility rule used by gccgo for years (which defines
unexported non-linknamed symbols as C static symbols).
As there can be pull-only linknames in the wild that may be broken
by this change, we currently only enforce this rule for symbols
defined in the standard library. Push linknames are added in the
standard library to allow things build.
Linkname references to external (non-Go) symbols are still allowed,
as their visibility is controlled by the C symbol visibility rules
and enforced by the C (static or dynamic) linker.
Assembly symbols are treated similar to linknamed symbols.
This is controlled by -checklinkname linker flag, currently not
enabled by default. A follow-up CL will enable it by default.
Change-Id: I07344f5c7a02124dbbef0fbc8fec3b666a4b2b0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/585358
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Go API is defined through exported symbols. When a package is
imported, the compiler ensures that only exported symbols can be
accessed, and the go command ensures that internal packages cannot
be imported. This ensures API integrity. But there is a hole:
using linkname, one can access internal or non-exported symbols.
Linkname is a mechanism to give access of a symbol to a package
without adding it to the public API. It is intended for coupled
packages to share some implementation details, or to break
circular dependencies, and both "push" (definition) and "pull"
(reference) sides are controlled, so they can be updated in sync.
Nevertheless, it is abused as a mechanism to reach into internal
details of other packages uncontrolled by the user, notably the
runtime. As the other package evolves, the code often breaks,
because the linknamed symbol may no longer exist, or change its
signature or semantics.
This CL adds a mechanism to enforce the integrity of linknames.
Generally, "push" linkname is allowed, as the package defining
the symbol explicitly opt in for access outside of the package.
"Pull" linkname is checked and only allowed in some circumstances.
Given that there are existing code that use "pull"-only linkname
to access other package's internals, disallowing it completely is
too much a change at this point in the release cycle. For a start,
implement a hard-coded blocklist, which contains some newly added
internal functions that, if used inappropriately, may break memory
safety or runtime integrity. All blocked symbols are newly added
in Go 1.23. So existing code that builds with Go 1.22 will
continue to build.
For the implementation, when compiling a package, we mark
linknamed symbols in the current package with an attribute. At
link time, marked linknamed symbols are checked against the
blocklist. Care is taken so it distinguishes a linkname reference
in the current package vs. a reference of a linkname from another
package and propagated to the current package (e.g. through
inlining or instantiation).
Symbol references in assembly code are similar to linknames, and
are treated similarly.
Change-Id: I8067efe29c122740cd4f1effd2dec2d839147d5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/584598
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
This indirection appears to be unnecessary for linking or linker debugging, and therefore hinders readability.
Since all *oReaders are added to loader.objs *only* via the Preload -> addObj path, before any symbols are examined, there is no possible way the "i" member of this structure is still useful; and is likely a remnant of an earlier design.
Change-Id: Icd880f40bf3299bf1aa0a14cf217268e49ee90c5
GitHub-Last-Rev: dd2d512cbe
GitHub-Pull-Request: golang/go#57460
Reviewed-on: https://go-review.googlesource.com/c/go/+/459456
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Get rid of a couple of unused methods in the loader and symbol
builder.
Change-Id: I3822891757dc56356295a9bc99545b725d485eac
Reviewed-on: https://go-review.googlesource.com/c/go/+/540260
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Rather than making these two different types, we can type alias them
together. This will ease converting cmd/internal/dwarf to use generics
in a subsequent CL.
The one unfortunate quirk is that while we'd currently like loader.Sym
to be the authoritative type, to break the cycle we have to instead
make loader.Sym an alias of sym.LoaderSym.
Change-Id: I6dde0d492ca89a478c2470c426bb4eed3393d680
Reviewed-on: https://go-review.googlesource.com/c/go/+/525195
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Fix spelling errors discovered using https://github.com/codespell-project/codespell. Errors in data files and vendored packages are ignored.
Change-Id: I83c7818222f2eea69afbd270c15b7897678131dc
GitHub-Last-Rev: 3491615b1b
GitHub-Pull-Request: golang/go#60758
Reviewed-on: https://go-review.googlesource.com/c/go/+/502576
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Change-Id: I915eff34fcfe82f3514254f7d8998baa88a91da6
Reviewed-on: https://go-review.googlesource.com/c/go/+/501997
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Currently, we pass elfsetstring to the loader as a callback, for
a special case of Addstring. This is only used for ELF when adding
strings to the section header string table. Move the logic to the
caller instead, so the loader would not have this special case.
Change-Id: Icfb91f380fe4ba435985c3019681597932f58242
Reviewed-on: https://go-review.googlesource.com/c/go/+/492718
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The type name symbol is always from a Go object file and we never
change it. Convert the data to string using unsafe conversion
without allocation.
Linking cmd/go (on macOS/amd64),
name old alloc/op new alloc/op delta
Deadcode_GC 1.25MB ± 0% 1.17MB ± 0% -6.29% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
Deadcode_GC 8.98k ± 0% 0.10k ± 3% -98.91% (p=0.000 n=20+20)
Change-Id: I33117ad1f991e4f14ce0b38cceec50b041e3c0a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/490915
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
This CL adds a .pdata section to the PE file generated by the Go linker.
The .pdata section is a standard section [1] that contains an array of
function table entries that are used for stack unwinding.
The table entries layout is taken from [2].
This CL just generates the table entries without any unwinding
information, which is enough to start doing some E2E tests
between the Go linker and the Win32 APIs.
The goal of the .pdata table is to allow Windows retrieve
unwind information for a function at a given PC. It does so by doing
a binary search on the table, looking for an entry that meets
BeginAddress >= PC < EndAddress.
Each table entry takes 12 bytes and only non-leaf functions with
frame pointer needs an entry on the .pdata table.
The result is that PE binaries will be ~0.7% bigger due to the unwind
information, a reasonable amount considering the benefits in
debuggability.
Updates #57302
[1] https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#the-pdata-section
[2] https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64#struct-runtime_function
Change-Id: If675d10c64452946dbab76709da20569651e3e9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/461738
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Currently, for a global variable, its debug info symbol is a named
symbol with the variable's name with a special prefix. And the
linker looks it up by name. This CL makes the debug info symbol an
aux symbol of the variable symbol.
Change-Id: I55614d0ef2af9c53eb40144ad80e09339bf3cbee
Reviewed-on: https://go-review.googlesource.com/c/go/+/490816
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This indicates the symbol does not use or preserve the TOC pointer in
R2. Likewise, it does not have a distinct local entry point. This
happens when gcc compiles an object with -mcpu=power10.
Recycle the SymLocalentry field of a text symbol to pass through this
hint as the bogus value 1 (A valid offset must be a multiple of 4
bytes), and update the usage to check and generate errors further into
the linking process. This matches the behavior of st_other as used by
ELFv2.
Change-Id: Ic89ce17b57f400ab44213b21a3730a98c7cdf842
Reviewed-on: https://go-review.googlesource.com/c/go/+/490295
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Currently, a symbol's global index, the Sym type, is defined as an
int, which is 64-bit on 64-bit machines. We're unlikely to have
more than 4 billion symbols in the near future. Even if we will,
we will probably hit some other limit (e.g. section size) before
the symbol number limit. Use a 32-bit type to reduce memory usage.
E,g, linking cmd/compile in external linking mode (on macOS/amd64)
Munmap_GC 43.2M ± 0% 35.5M ± 1% -17.74% (p=0.000 n=16+20)
This brings the memory usage back before the previous CL, and even
lower.
Change-Id: Ie185f1586638fe70d8121312bfa9410942d518c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/487416
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Currently, a symbol's outer symbol, the "special" attribute, and
whether a symbol is a generator symbol are represented as maps,
and are accessed in some loops over nearly all reachable symbols.
The map lookups are a bit expensive.
For outer symbol, a non-trivial portion of the symbols have outer
symbol set (e.g. type symbols, which we put into container symbols
like "type:*"). Using a slice to access more efficiently.
For the special and generator symbol attributes, use a bitmap.
There are not many symbols have those attributes, so the bitmap is
quite sparse. The bitmap is not too large anyway, so use it for
now. If we want to further reduce memory usage we could consider
some other data structure like a Bloom filter.
Linking cmd/compile in external linking mode (on macOS/amd64)
Symtab 12.9ms ± 9% 6.4ms ± 5% -50.08% (p=0.000 n=19+18)
Dodata 64.9ms ±12% 57.1ms ±12% -11.90% (p=0.000 n=20+20)
Asmb 36.7ms ±11% 32.8ms ± 9% -10.61% (p=0.000 n=20+18)
Asmb2 26.6ms ±15% 21.9ms ±12% -17.75% (p=0.000 n=20+18)
There is some increase of memory usage
Munmap_GC 40.9M ± 1% 43.2M ± 0% +5.54% (p=0.000 n=20+19)
The next CL will bring the memory usage back.
Change-Id: Ie4347eb96c51f008b9284270de37fc880bb52d2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/487415
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Go programs can now use the //go:wasmimport module_name function_name
directive to import functions from the WebAssembly runtime.
For now, the directive is restricted to the runtime and syscall/js
packages.
* Derived from CL 350737
* Original work modified to work with changes to the IR conversion code.
* Modification of CL 350737 changes to fully exist in Unified IR path (emp)
* Original work modified to work with changes to the ABI configuration code.
* Fixes#38248
Co-authored-by: Vedant Roy <vroy101@gmail.com>
Co-authored-by: Richard Musiol <mail@richard-musiol.de>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Change-Id: I740719735d91c306ac718a435a78e1ee9686bc16
Reviewed-on: https://go-review.googlesource.com/c/go/+/463018
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Introduce a flag in the object file indicating whether a given
function corresponds to a compiler-generated (not user-written) init
function, such as "os.init" or "syscall.init". Add code to the
compiler to fill in the correct value for the flag, and add support to
the loader package in the linker for testing the flag. The new loader
API is currently unused, but will be needed in the next CL in this
stack.
Updates #2559.
Updates #36021.
Updates #14840.
Change-Id: Iea7ad2adda487e4af7a44f062f9817977c53b394
Reviewed-on: https://go-review.googlesource.com/c/go/+/463855
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Since this may add a large number of --export-dynamic-symbol options,
use a response file if the command line gets large.
Fixes#53579
Change-Id: Ic226bf372bf1e177a3dae886d1c48f4ce3569c0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/414654
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When ctxt.Debugvlog > 1, produce additional trace output to describe
which object files are being pulled out of host archive libraries and
why they were pulled (e.g. which symbol had a reference to something
in a library). Intended to make it easier to debug problems with cgo
internal linking.
Change-Id: Icd64aff244b9145162a00cb51642ef32f26adfba
Reviewed-on: https://go-review.googlesource.com/c/go/+/451736
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This adds the function "start line number" to runtime._func and
runtime.inlinedCall objects. The "start line number" is the line number
of the func keyword or TEXT directive for assembly.
Subtracting the start line number from PC line number provides the
relative line offset of a PC from the the start of the function. This
helps with source stability by allowing code above the function to move
without invalidating samples within the function.
Encoding start line rather than relative lines directly is convenient
because the pprof format already contains a start line field.
This CL uses a straightforward encoding of explictly including a start
line field in every _func and inlinedCall. It is possible that we could
compress this further in the future. e.g., functions with a prologue
usually have <line of PC 0> == <start line>. In runtime.test, 95% of
functions have <line of PC 0> == <start line>.
According to bent, this is geomean +0.83% binary size vs master and
-0.31% binary size vs 1.19.
Note that //line directives can change the file and line numbers
arbitrarily. The encoded start line is as adjusted by //line directives.
Since this can change in the middle of a function, `line - start line`
offset calculations may not be meaningful if //line directives are in
use.
For #55022.
Change-Id: Iaabbc6dd4f85ffdda294266ef982ae838cc692f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/429638
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
PPC64 ELFv2 uses the st_other field of a symbol to specify an offset
from the global entry point to its local entry point. Similarly, some
values (i.e 1) may also require additional linker support which is
missing today.
For now, generate an error if we encounter unsupported local entry
values on PPC64, and update the Localentry values to use bytes, not
32b instruction words.
Similarly, ELFv2 1.5 also updates the wording of values 2-6. They
now map to a specific number of bytes.
Change-Id: Id1b71c3b0fea982bdcfb7eac91d9f93e04ae43f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/431876
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
