mirror of
https://github.com/golang/go.git
synced 2025-12-08 06:10:04 +00:00
This CL is part of a set of CLs that attempt to reduce how much work the GC must do. See the design in https://go.dev/design/74299-runtime-freegc This CL updates the compiler to examine append calls to prove whether or not the slice is aliased. If proven unaliased, the compiler automatically inserts a call to a new runtime function introduced with this CL, runtime.growsliceNoAlias, which frees the old backing memory immediately after slice growth is complete and the old storage is logically dead. Two append benchmarks below show promising results, executing up to ~2x faster and up to factor of ~3 memory reduction with this CL. The approach works with multiple append calls for the same slice, including inside loops, and the final slice memory can be escaping, such as in a classic pattern of returning a slice from a function after the slice is built. (The final slice memory is never freed with this CL, though we have other work that tackles that.) An example target for this CL is we automatically free the intermediate memory for the appends in the loop in this function: func f1(input []int) []int { var s []int for _, x := range input { s = append(s, g(x)) // s cannot be aliased here if h(x) { s = append(s, x) // s cannot be aliased here } } return s // slice escapes at end } In this case, the compiler and the runtime collaborate so that the heap allocated backing memory for s is automatically freed after a successful grow. (For the first grow, there is nothing to free, but for the second and subsequent growths, the old heap memory is freed automatically.) The new runtime.growsliceNoAlias is primarily implemented by calling runtime.freegc, which we introduced in CL 673695. The high-level approach here is we step through the IR starting from a slice declaration and look for any operations that either alias the slice or might do so, and treat any IR construct we don't specifically handle as a potential alias (and therefore conservatively fall back to treating the slice as aliased when encountering something not understood). For loops, some additional care is required. We arrange the analysis so that an alias in the body of a loop causes all the appends in that same loop body to be marked aliased, even if the aliasing occurs after the append in the IR: func f2() { var s []int for i := range 10 { s = append(s, i) // aliased due to next line alias = s } } For nested loops, we analyse the nesting appropriately so that for example this append is still proven as non-aliased in the inner loop even though it aliased for the outer loop: func f3() { for range 10 { var s []int for i := range 10 { s = append(s, i) // append using non-aliased slice } alias = s } } A good starting point is the beginning of the test/escape_alias.go file, which starts with ~10 introductory examples with brief comments that attempt to illustrate the high-level approach. For more details, see the new .../internal/escape/alias.go file, especially the (*aliasAnalysis).analyze method. In the first benchmark, an append in a loop builds up a slice from nothing, where the slice elements are each 64 bytes. In the table below, 'count' is the number of appends. With 1 append, there is no opportunity for this CL to free memory. Once there are 2 appends, the growth from 1 element to 2 elements means the compiler-inserted growsliceNoAlias frees the 1-element array, and we see a ~33% reduction in memory use and a small reported speed improvement. As the number of appends increases for example to 5, we are at a ~20% speed improvement and ~45% memory reduction, and so on until we reach ~40% faster and ~50% less memory allocated at the end of the table. There can be variation in the reported numbers based on -randlayout, so this table is for 30 different values of -randlayout with a total n=150. (Even so, there is still some variation, so we probably should not read too much into small changes.) This is with GOAMD64=v3 on a VM that gcc reports is cascadelake. goos: linux goarch: amd64 pkg: runtime cpu: Intel(R) Xeon(R) CPU @ 2.80GHz │ old-1bb1f2bf0c │ freegc-8ba7421-ps16 │ │ sec/op │ sec/op vs base │ Append64Bytes/count=1-4 31.09n ± 2% 31.69n ± 1% +1.95% (n=150) Append64Bytes/count=2-4 73.31n ± 1% 70.27n ± 0% -4.15% (n=150) Append64Bytes/count=3-4 142.7n ± 1% 124.6n ± 1% -12.68% (n=150) Append64Bytes/count=4-4 149.6n ± 1% 127.7n ± 0% -14.64% (n=150) Append64Bytes/count=5-4 277.1n ± 1% 213.6n ± 0% -22.90% (n=150) Append64Bytes/count=6-4 280.7n ± 1% 216.5n ± 1% -22.87% (n=150) Append64Bytes/count=10-4 544.3n ± 1% 386.6n ± 0% -28.97% (n=150) Append64Bytes/count=20-4 1058.5n ± 1% 715.6n ± 1% -32.39% (n=150) Append64Bytes/count=50-4 2.121µ ± 1% 1.404µ ± 1% -33.83% (n=150) Append64Bytes/count=100-4 4.152µ ± 1% 2.736µ ± 1% -34.11% (n=150) Append64Bytes/count=200-4 7.753µ ± 1% 4.882µ ± 1% -37.03% (n=150) Append64Bytes/count=400-4 15.163µ ± 2% 9.273µ ± 1% -38.84% (n=150) geomean 601.8n 455.0n -24.39% │ old-1bb1f2bf0c │ freegc-8ba7421-ps16 │ │ B/op │ B/op vs base │ Append64Bytes/count=1-4 64.00 ± 0% 64.00 ± 0% ~ (n=150) Append64Bytes/count=2-4 192.0 ± 0% 128.0 ± 0% -33.33% (n=150) Append64Bytes/count=3-4 448.0 ± 0% 256.0 ± 0% -42.86% (n=150) Append64Bytes/count=4-4 448.0 ± 0% 256.0 ± 0% -42.86% (n=150) Append64Bytes/count=5-4 960.0 ± 0% 512.0 ± 0% -46.67% (n=150) Append64Bytes/count=6-4 960.0 ± 0% 512.0 ± 0% -46.67% (n=150) Append64Bytes/count=10-4 1.938Ki ± 0% 1.000Ki ± 0% -48.39% (n=150) Append64Bytes/count=20-4 3.938Ki ± 0% 2.001Ki ± 0% -49.18% (n=150) Append64Bytes/count=50-4 7.938Ki ± 0% 4.005Ki ± 0% -49.54% (n=150) Append64Bytes/count=100-4 15.938Ki ± 0% 8.021Ki ± 0% -49.67% (n=150) Append64Bytes/count=200-4 31.94Ki ± 0% 16.08Ki ± 0% -49.64% (n=150) Append64Bytes/count=400-4 63.94Ki ± 0% 32.33Ki ± 0% -49.44% (n=150) geomean 1.991Ki 1.124Ki -43.54% │ old-1bb1f2bf0c │ freegc-8ba7421-ps16 │ │ allocs/op │ allocs/op vs base │ Append64Bytes/count=1-4 1.000 ± 0% 1.000 ± 0% ~ (n=150) Append64Bytes/count=2-4 2.000 ± 0% 1.000 ± 0% -50.00% (n=150) Append64Bytes/count=3-4 3.000 ± 0% 1.000 ± 0% -66.67% (n=150) Append64Bytes/count=4-4 3.000 ± 0% 1.000 ± 0% -66.67% (n=150) Append64Bytes/count=5-4 4.000 ± 0% 1.000 ± 0% -75.00% (n=150) Append64Bytes/count=6-4 4.000 ± 0% 1.000 ± 0% -75.00% (n=150) Append64Bytes/count=10-4 5.000 ± 0% 1.000 ± 0% -80.00% (n=150) Append64Bytes/count=20-4 6.000 ± 0% 1.000 ± 0% -83.33% (n=150) Append64Bytes/count=50-4 7.000 ± 0% 1.000 ± 0% -85.71% (n=150) Append64Bytes/count=100-4 8.000 ± 0% 1.000 ± 0% -87.50% (n=150) Append64Bytes/count=200-4 9.000 ± 0% 1.000 ± 0% -88.89% (n=150) Append64Bytes/count=400-4 10.000 ± 0% 1.000 ± 0% -90.00% (n=150) geomean 4.331 1.000 -76.91% The second benchmark is similar, but instead uses an 8-byte integer for the slice element. The first 4 appends in the loop never call into the runtime thanks to the excellent CL 664299 introduced by Keith in Go 1.25 that allows some <= 32 byte dynamically-sized slices to be on the stack, so this CL is neutral for <= 32 bytes. Once the 5th append occurs at count=5, a grow happens via the runtime and heap allocates as normal, but freegc does not yet have anything to free, so we see a small ~1.4ns penalty reported there. But once the second growth happens, the older heap memory is now automatically freed by freegc, so we start to see some benefit in memory reductions and speed improvements, starting at a tiny speed improvement (close to a wash, or maybe noise) by the second growth before count=10, and building up to ~2x faster with ~68% fewer allocated bytes reported. goos: linux goarch: amd64 pkg: runtime cpu: Intel(R) Xeon(R) CPU @ 2.80GHz │ old-1bb1f2bf0c │ freegc-8ba7421-ps16 │ │ sec/op │ sec/op vs base │ AppendInt/count=1-4 2.978n ± 0% 2.969n ± 0% -0.30% (p=0.000 n=150) AppendInt/count=4-4 4.292n ± 3% 4.163n ± 3% ~ (p=0.528 n=150) AppendInt/count=5-4 33.50n ± 0% 34.93n ± 0% +4.25% (p=0.000 n=150) AppendInt/count=10-4 76.21n ± 1% 75.67n ± 0% -0.72% (p=0.000 n=150) AppendInt/count=20-4 150.6n ± 1% 133.0n ± 0% -11.65% (n=150) AppendInt/count=50-4 284.1n ± 1% 225.6n ± 0% -20.59% (n=150) AppendInt/count=100-4 544.2n ± 1% 392.4n ± 1% -27.89% (n=150) AppendInt/count=200-4 1051.5n ± 1% 702.3n ± 0% -33.21% (n=150) AppendInt/count=400-4 2.041µ ± 1% 1.312µ ± 1% -35.70% (n=150) AppendInt/count=1000-4 5.224µ ± 2% 2.851µ ± 1% -45.43% (n=150) AppendInt/count=2000-4 11.770µ ± 1% 6.010µ ± 1% -48.94% (n=150) AppendInt/count=3000-4 17.747µ ± 2% 8.264µ ± 1% -53.44% (n=150) geomean 331.8n 246.4n -25.72% │ old-1bb1f2bf0c │ freegc-8ba7421-ps16 │ │ B/op │ B/op vs base │ AppendInt/count=1-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=150) AppendInt/count=4-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=150) AppendInt/count=5-4 64.00 ± 0% 64.00 ± 0% ~ (p=1.000 n=150) AppendInt/count=10-4 192.0 ± 0% 128.0 ± 0% -33.33% (n=150) AppendInt/count=20-4 448.0 ± 0% 256.0 ± 0% -42.86% (n=150) AppendInt/count=50-4 960.0 ± 0% 512.0 ± 0% -46.67% (n=150) AppendInt/count=100-4 1.938Ki ± 0% 1.000Ki ± 0% -48.39% (n=150) AppendInt/count=200-4 3.938Ki ± 0% 2.001Ki ± 0% -49.18% (n=150) AppendInt/count=400-4 7.938Ki ± 0% 4.005Ki ± 0% -49.54% (n=150) AppendInt/count=1000-4 24.56Ki ± 0% 10.05Ki ± 0% -59.07% (n=150) AppendInt/count=2000-4 58.56Ki ± 0% 20.31Ki ± 0% -65.32% (n=150) AppendInt/count=3000-4 85.19Ki ± 0% 27.30Ki ± 0% -67.95% (n=150) geomean ² -42.81% │ old-1bb1f2bf0c │ freegc-8ba7421-ps16 │ │ allocs/op │ allocs/op vs base │ AppendInt/count=1-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=150) AppendInt/count=4-4 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=150) AppendInt/count=5-4 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=150) AppendInt/count=10-4 2.000 ± 0% 1.000 ± 0% -50.00% (n=150) AppendInt/count=20-4 3.000 ± 0% 1.000 ± 0% -66.67% (n=150) AppendInt/count=50-4 4.000 ± 0% 1.000 ± 0% -75.00% (n=150) AppendInt/count=100-4 5.000 ± 0% 1.000 ± 0% -80.00% (n=150) AppendInt/count=200-4 6.000 ± 0% 1.000 ± 0% -83.33% (n=150) AppendInt/count=400-4 7.000 ± 0% 1.000 ± 0% -85.71% (n=150) AppendInt/count=1000-4 9.000 ± 0% 1.000 ± 0% -88.89% (n=150) AppendInt/count=2000-4 11.000 ± 0% 1.000 ± 0% -90.91% (n=150) AppendInt/count=3000-4 12.000 ± 0% 1.000 ± 0% -91.67% (n=150) geomean ² -72.76% ² Of course, these are just microbenchmarks, but likely indicate there are some opportunities here. The immediately following CL 712422 tackles inlining and is able to get runtime.freegc working automatically with iterators such as used by slices.Collect, which becomes able to automatically free the intermediate memory from its repeated appends (which earlier in this work required a temporary hand edit to the slices package). For now, we only use the NoAlias version for element types without pointers while waiting on additional runtime support in CL 698515. Updates #74299 Change-Id: I1b9d286aa97c170dcc2e203ec0f8ca72d84e8221 Reviewed-on: https://go-review.googlesource.com/c/go/+/710015 Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@golang.org> |
||
|---|---|---|
| .. | ||
| addrcalc.go | ||
| alloc.go | ||
| append.go | ||
| append_freegc.go | ||
| arithmetic.go | ||
| atomics.go | ||
| bitfield.go | ||
| bits.go | ||
| bmi.go | ||
| bool.go | ||
| clobberdead.go | ||
| clobberdeadreg.go | ||
| compare_and_branch.go | ||
| comparisons.go | ||
| condmove.go | ||
| constants.go | ||
| copy.go | ||
| divmod.go | ||
| floats.go | ||
| fuse.go | ||
| generics.go | ||
| ifaces.go | ||
| issue22703.go | ||
| issue25378.go | ||
| issue31618.go | ||
| issue33580.go | ||
| issue38554.go | ||
| issue42610.go | ||
| issue48054.go | ||
| issue52635.go | ||
| issue54467.go | ||
| issue56440.go | ||
| issue58166.go | ||
| issue59297.go | ||
| issue60324.go | ||
| issue60673.go | ||
| issue61356.go | ||
| issue63332.go | ||
| issue66585.go | ||
| issue68845.go | ||
| issue69635.go | ||
| issue70409.go | ||
| issue72832.go | ||
| issue74485.go | ||
| issue74788.go | ||
| issue75203.go | ||
| load_type_from_itab.go | ||
| logic.go | ||
| mapaccess.go | ||
| maps.go | ||
| math.go | ||
| mathbits.go | ||
| memcombine.go | ||
| memcse.go | ||
| memops.go | ||
| memops_bigoffset.go | ||
| multiply.go | ||
| noextend.go | ||
| race.go | ||
| README | ||
| reflect_type.go | ||
| regabi_regalloc.go | ||
| retpoline.go | ||
| rotate.go | ||
| schedule.go | ||
| select.go | ||
| shift.go | ||
| shortcircuit.go | ||
| simd.go | ||
| slices.go | ||
| smallintiface.go | ||
| spectre.go | ||
| spills.go | ||
| stack.go | ||
| strings.go | ||
| structs.go | ||
| switch.go | ||
| typeswitch.go | ||
| unique.go | ||
| unsafe.go | ||
| writebarrier.go | ||
| zerosize.go | ||
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
The codegen directory contains code generation tests for the gc
compiler.
- Introduction
The test harness compiles Go code inside files in this directory and
matches the generated assembly (the output of `go tool compile -S`)
against a set of regexps to be specified in comments that follow a
special syntax (described below). The test driver is implemented as
an action within the GOROOT/test test suite, called "asmcheck".
The codegen harness is part of the all.bash test suite, but for
performance reasons only the codegen tests for the host machine's
GOARCH are enabled by default, and only on GOOS=linux.
To perform comprehensive tests for all the supported architectures
(even on a non-Linux system), one can run the following command:
$ ../../bin/go test cmd/internal/testdir -run='Test/codegen' -all_codegen -v
This is recommended after any change that affect the compiler's code.
The test harness compiles the tests with the same go toolchain that is
used to run the test. After writing tests for a newly added codegen
transformation, it can be useful to first run the test harness with a
toolchain from a released Go version (and verify that the new tests
fail), and then re-running the tests using the devel toolchain.
- Regexps comments syntax
Instructions to match are specified inside plain comments that start
with an architecture tag, followed by a colon and a quoted Go-style
regexp to be matched. For example, the following test:
func Sqrt(x float64) float64 {
// amd64:"SQRTSD"
// arm64:"FSQRTD"
return math.Sqrt(x)
}
verifies that math.Sqrt calls are intrinsified to a SQRTSD instruction
on amd64, and to a FSQRTD instruction on arm64.
It is possible to put multiple architectures checks into the same
line, as:
// amd64:"SQRTSD" arm64:"FSQRTD"
although this form should be avoided when doing so would make the
regexps line excessively long and difficult to read.
Comments that are on their own line will be matched against the first
subsequent non-comment line. Inline comments are also supported; the
regexp will be matched against the code found on the same line:
func Sqrt(x float64) float64 {
return math.Sqrt(x) // arm:"SQRTD"
}
It's possible to specify a comma-separated list of regexps to be
matched. For example, the following test:
func TZ8(n uint8) int {
// amd64:"BSFQ","ORQ\t\\$256"
return bits.TrailingZeros8(n)
}
verifies that the code generated for a bits.TrailingZeros8 call on
amd64 contains both a "BSFQ" instruction and an "ORQ $256".
Note how the ORQ regex includes a tab char (\t). In the Go assembly
syntax, operands are separated from opcodes by a tabulation.
Regexps can be quoted using either " or `. Special characters must be
escaped accordingly. Both of these are accepted, and equivalent:
// amd64:"ADDQ\t\\$3"
// amd64:`ADDQ\t\$3`
and they'll match this assembly line:
ADDQ $3
Negative matches can be specified using a - before the quoted regexp.
For example:
func MoveSmall() {
x := [...]byte{1, 2, 3, 4, 5, 6, 7}
copy(x[1:], x[:]) // arm64:-".*memmove"
}
verifies that NO memmove call is present in the assembly generated for
the copy() line.
The expected number of matches for the regexp can be specified using a
positive number:
func fb(a [4]int) (r [4]int) {
// amd64:2`MOVUPS[^,]+, X0$`,2`MOVUPS\sX0,[^\n]+$`
return a
}
- Architecture specifiers
There are three different ways to specify on which architecture a test
should be run:
* Specify only the architecture (eg: "amd64"). This indicates that the
check should be run on all the supported architecture variants. For
instance, arm checks will be run against all supported GOARM
variations (5,6,7).
* Specify both the architecture and a variant, separated by a slash
(eg: "arm/7"). This means that the check will be run only on that
specific variant.
* Specify the operating system, the architecture and the variant,
separated by slashes (eg: "plan9/386/sse2", "plan9/amd64/"). This is
needed in the rare case that you need to do a codegen test affected
by a specific operating system; by default, tests are compiled only
targeting linux.
- Remarks, and Caveats
-- Write small test functions
As a general guideline, test functions should be small, to avoid
possible interactions between unrelated lines of code that may be
introduced, for example, by the compiler's optimization passes.
Any given line of Go code could get assigned more instructions than it
may appear from reading the source. In particular, matching all MOV
instructions should be avoided; the compiler may add them for
unrelated reasons and this may render the test ineffective.
-- Line matching logic
Regexps are always matched from the start of the instructions line.
This means, for example, that the "MULQ" regexp is equivalent to
"^MULQ" (^ representing the start of the line), and it will NOT match
the following assembly line:
IMULQ $99, AX
To force a match at any point of the line, ".*MULQ" should be used.
For the same reason, a negative regexp like -"memmove" is not enough
to make sure that no memmove call is included in the assembly. A
memmove call looks like this:
CALL runtime.memmove(SB)
To make sure that the "memmove" symbol does not appear anywhere in the
assembly, the negative regexp to be used is -".*memmove".