Stowage/go
2
0
Fork 0
mirror of https://github.com/golang/go.git synced 2025-12-08 06:10:04 +00:00
Code Issues Projects Releases Packages Wiki Activity
go/src/cmd/compile/internal/ssa/_gen/genericOps.go

753 lines
47 KiB
Go
Raw Normal View History

[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
cmd/compile: define high bits of AuxInt Previously if we were only using the low bits of AuxInt, the high bits were ignored and could be junk. This CL changes that behavior to define the high bits to be the sign-extended version of the low bits for all cases. There are 2 main benefits: - Deterministic representation. This helps with CSE. (Const8 [0x1]) and (Const8 [0x101]) used to be the same "value" but CSE couldn't see them as such. - Testability. We can check that all ops leave AuxInt in a state consistent with the new rule. In the old scheme, it was hard to check whether a rule correctly used only the low-order bits. Side benefits: - ==0 and !=0 tests are easier. Drawbacks: - This differs from the runtime representation in registers, where it is important that we allow upper bits to be undefined (so we're not sign/zero-extending all the time). - Ops that treat AuxInt as unsigned (shifts, mostly) need to be a bit more careful. Change-Id: I9a685ff27e36dc03287c9ab1cecd6c0b4045c819 Reviewed-on: https://go-review.googlesource.com/21256 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-29 16:39:53 -07:00
// Generic opcodes typically specify a width. The inputs and outputs
// of that op are the given number of bits wide. There is no notion of
// "sign", so Add32 can be used both for signed and unsigned 32-bit
// addition.
// Signed/unsigned is explicit with the extension ops
// (SignExt*/ZeroExt*) and implicit as the arg to some opcodes
// (e.g. the second argument to shifts is unsigned). If not mentioned,
// all args take signed inputs, or don't care whether their inputs
// are signed or unsigned.
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
var genericOps = []opData{
cmd/compile: implement bits.Mul64 on 32-bit systems This CL implements Mul64uhilo, Hmul64, Hmul64u, and Avg64u on 32-bit systems, with the effect that constant division of both int64s and uint64s can now be emitted directly in all cases, and also that bits.Mul64 can be intrinsified on 32-bit systems. Previously, constant division of uint64s by values 0 ≤ c ≤ 0xFFFF were implemented as uint32 divisions by c and some fixup. After expanding those smaller constant divisions, the code for i/999 required: (386) 7 mul, 10 add, 2 sub, 3 rotate, 3 shift (104 bytes) (arm) 7 mul, 9 add, 3 sub, 2 shift (104 bytes) (mips) 7 mul, 10 add, 5 sub, 6 shift, 3 sgtu (176 bytes) For that much code, we might as well use a full 64x64->128 multiply that can be used for all divisors, not just small ones. Having done that, the same i/999 now generates: (386) 4 mul, 9 add, 2 sub, 2 or, 6 shift (112 bytes) (arm) 4 mul, 8 add, 2 sub, 2 or, 3 shift (92 bytes) (mips) 4 mul, 11 add, 3 sub, 6 shift, 8 sgtu, 4 or (196 bytes) The size increase on 386 is due to a few extra register spills. The size increase on mips is due to add-with-carry being hard. The new approach is more general, letting us delete the old special case and guarantee that all int64 and uint64 divisions by constants are generated directly on 32-bit systems. This especially speeds up code making heavy use of bits.Mul64 with a constant argument, which happens in strconv and various crypto packages. A few examples are benchmarked below. pkg: cmd/compile/internal/test benchmark \ host local linux-amd64 s7 linux-386 s7:GOARCH=386 vs base vs base vs base vs base vs base DivconstI64 ~ ~ ~ -49.66% -21.02% ModconstI64 ~ ~ ~ -13.45% +14.52% DivisiblePow2constI64 ~ ~ ~ +0.97% -1.32% DivisibleconstI64 ~ ~ ~ -20.01% -48.28% DivisibleWDivconstI64 ~ ~ -1.76% -38.59% -42.74% DivconstU64/3 ~ ~ ~ -13.82% -4.09% DivconstU64/5 ~ ~ ~ -14.10% -3.54% DivconstU64/37 -2.07% -4.45% ~ -19.60% -9.55% DivconstU64/1234567 ~ ~ ~ -61.55% -56.93% ModconstU64 ~ ~ ~ -6.25% ~ DivisibleconstU64 ~ ~ ~ -2.78% -7.82% DivisibleWDivconstU64 ~ ~ ~ +4.23% +2.56% pkg: math/bits benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base Add ~ ~ ~ ~ Add32 +1.59% ~ ~ ~ Add64 ~ ~ ~ ~ Add64multiple ~ ~ ~ ~ Sub ~ ~ ~ ~ Sub32 ~ ~ ~ ~ Sub64 ~ ~ -9.20% ~ Sub64multiple ~ ~ ~ ~ Mul ~ ~ ~ ~ Mul32 ~ ~ ~ ~ Mul64 ~ ~ -41.58% -53.21% Div ~ ~ ~ ~ Div32 ~ ~ ~ ~ Div64 ~ ~ ~ ~ pkg: strconv benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base ParseInt/Pos/7bit ~ ~ -11.08% -6.75% ParseInt/Pos/26bit ~ ~ -13.65% -11.02% ParseInt/Pos/31bit ~ ~ -14.65% -9.71% ParseInt/Pos/56bit -1.80% ~ -17.97% -10.78% ParseInt/Pos/63bit ~ ~ -13.85% -9.63% ParseInt/Neg/7bit ~ ~ -12.14% -7.26% ParseInt/Neg/26bit ~ ~ -14.18% -9.81% ParseInt/Neg/31bit ~ ~ -14.51% -9.02% ParseInt/Neg/56bit ~ ~ -15.79% -9.79% ParseInt/Neg/63bit ~ ~ -15.68% -11.07% AppendFloat/Decimal ~ ~ -7.25% -12.26% AppendFloat/Float ~ ~ -15.96% -19.45% AppendFloat/Exp ~ ~ -13.96% -17.76% AppendFloat/NegExp ~ ~ -14.89% -20.27% AppendFloat/LongExp ~ ~ -12.68% -17.97% AppendFloat/Big ~ ~ -11.10% -16.64% AppendFloat/BinaryExp ~ ~ ~ ~ AppendFloat/32Integer ~ ~ -10.05% -10.91% AppendFloat/32ExactFraction ~ ~ -8.93% -13.00% AppendFloat/32Point ~ ~ -10.36% -14.89% AppendFloat/32Exp ~ ~ -9.88% -13.54% AppendFloat/32NegExp ~ ~ -10.16% -14.26% AppendFloat/32Shortest ~ ~ -11.39% -14.96% AppendFloat/32Fixed8Hard ~ ~ ~ -2.31% AppendFloat/32Fixed9Hard ~ ~ ~ -7.01% AppendFloat/64Fixed1 ~ ~ -2.83% -8.23% AppendFloat/64Fixed2 ~ ~ ~ -7.94% AppendFloat/64Fixed3 ~ ~ -4.07% -7.22% AppendFloat/64Fixed4 ~ ~ -7.24% -7.62% AppendFloat/64Fixed12 ~ ~ -6.57% -4.82% AppendFloat/64Fixed16 ~ ~ -4.00% -5.81% AppendFloat/64Fixed12Hard -2.22% ~ -4.07% -6.35% AppendFloat/64Fixed17Hard -2.12% ~ ~ -3.79% AppendFloat/64Fixed18Hard -1.89% ~ +2.48% ~ AppendFloat/Slowpath64 -1.85% ~ -14.49% -18.21% AppendFloat/SlowpathDenormal64 ~ ~ -13.08% -19.41% pkg: crypto/internal/fips140/nistec/fiat benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base Mul/P224 ~ ~ -29.95% -39.60% Mul/P384 ~ ~ -37.11% -63.33% Mul/P521 ~ ~ -26.62% -12.42% Square/P224 +1.46% ~ -40.62% -49.18% Square/P384 ~ ~ -45.51% -69.68% Square/P521 +90.37% ~ -25.26% -11.23% (The +90% is a separate problem and not real; that much variation can be seen on that system by running the same binary from two different files.) pkg: crypto/internal/fips140/edwards25519 benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base EncodingDecoding ~ ~ -34.67% -35.75% ScalarBaseMult ~ ~ -31.25% -30.29% ScalarMult ~ ~ -33.45% -32.54% VarTimeDoubleScalarBaseMult ~ ~ -33.78% -33.68% Change-Id: Id3c91d42cd01def6731b755e99f8f40c6ad1bb65 Reviewed-on: https://go-review.googlesource.com/c/go/+/716061 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
2025-10-27 19:41:39 -04:00
// Pseudo-op.
{name: "Last", argLength: -1}, // return last element of tuple; for "let" bindings
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// 2-input arithmetic
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// Types must be consistent with Go typing. Add, for example, must take two values
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// of the same type and produces that same type.
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Add8", argLength: 2, commutative: true}, // arg0 + arg1
{name: "Add16", argLength: 2, commutative: true},
{name: "Add32", argLength: 2, commutative: true},
{name: "Add64", argLength: 2, commutative: true},
{name: "AddPtr", argLength: 2}, // For address calculations. arg0 is a pointer and arg1 is an int.
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
{name: "Add32F", argLength: 2, commutative: true},
{name: "Add64F", argLength: 2, commutative: true},
[dev.ssa] cmd/compile/internal/ssa: use width and sign specific opcodes Bake the bit width and signedness into opcodes. Pro: Rewrite rules become easier. Less chance for confusion. Con: Lots more opcodes. Let me know what you think. I'm leaning towards this, but I could be convinced otherwise if people think this is too ugly. Update #11467 Change-Id: Icf1b894268cdf73515877bb123839800d97b9df9 Reviewed-on: https://go-review.googlesource.com/12362 Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-07-19 15:48:20 -07:00
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Sub8", argLength: 2}, // arg0 - arg1
{name: "Sub16", argLength: 2},
{name: "Sub32", argLength: 2},
{name: "Sub64", argLength: 2},
{name: "SubPtr", argLength: 2},
{name: "Sub32F", argLength: 2},
{name: "Sub64F", argLength: 2},
{name: "Mul8", argLength: 2, commutative: true}, // arg0 * arg1
{name: "Mul16", argLength: 2, commutative: true},
{name: "Mul32", argLength: 2, commutative: true},
{name: "Mul64", argLength: 2, commutative: true},
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
{name: "Mul32F", argLength: 2, commutative: true},
{name: "Mul64F", argLength: 2, commutative: true},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Div32F", argLength: 2}, // arg0 / arg1
{name: "Div64F", argLength: 2},
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
{name: "Hmul32", argLength: 2, commutative: true},
{name: "Hmul32u", argLength: 2, commutative: true},
{name: "Hmul64", argLength: 2, commutative: true},
{name: "Hmul64u", argLength: 2, commutative: true},
[dev.ssa] cmd/compile: strength-reduce 64-bit constant divides The frontend does this for 32 bits and below, but SSA needs to do it for 64 bits. The algorithms are all copied from cgen.go:cgen_div. Speeds up TimeFormat substantially: ~40% slower to ~10% slower. Change-Id: I023ea2eb6040df98ccd9105e15ca6ea695610a7a Reviewed-on: https://go-review.googlesource.com/19302 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Todd Neal <todd@tneal.org>
2016-02-05 20:26:18 -08:00
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
{name: "Mul32uhilo", argLength: 2, typ: "(UInt32,UInt32)", commutative: true}, // arg0 * arg1, returns (hi, lo)
{name: "Mul64uhilo", argLength: 2, typ: "(UInt64,UInt64)", commutative: true}, // arg0 * arg1, returns (hi, lo)
cmd/compile: intrinsify math/big.mulWW, divWW on AMD64 Change-Id: I59f7afa7a5803d19f8b21fe70fc85ef997bb3a85 Reviewed-on: https://go-review.googlesource.com/30542 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2016-10-06 15:43:47 -04:00
cmd/compile: add intrinsics for runtime/internal/math on 386 and amd64 Add generic, 386 and amd64 specific ops and SSA rules for multiplication with overflow and branching based on overflow flags. Use these to intrinsify runtime/internal/math.MulUinptr. On amd64 mul, overflow := math.MulUintptr(a, b) if overflow { is lowered to two instructions: MULQ SI JO 0x10ee35c No codegen tests as codegen can not currently test unexported internal runtime functions. amd64: name old time/op new time/op delta MulUintptr/small 1.16ns ± 5% 0.88ns ± 6% -24.36% (p=0.000 n=19+20) MulUintptr/large 10.7ns ± 1% 1.1ns ± 1% -89.28% (p=0.000 n=17+19) Change-Id: If60739a86f820e5044d677276c21df90d3c7a86a Reviewed-on: https://go-review.googlesource.com/c/141820 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-01-27 11:55:34 +01:00
{name: "Mul32uover", argLength: 2, typ: "(UInt32,Bool)", commutative: true}, // Let x = arg0*arg1 (full 32x32-> 64 unsigned multiply), returns (uint32(x), (uint32(x) != x))
{name: "Mul64uover", argLength: 2, typ: "(UInt64,Bool)", commutative: true}, // Let x = arg0*arg1 (full 64x64->128 unsigned multiply), returns (uint64(x), (uint64(x) != x))
cmd/compile: move constant divide strength reduction to SSA rules Currently the conversion from constant divides to multiplies is mostly done during the walk pass. This is suboptimal because SSA can determine that the value being divided by is constant more often (e.g. after inlining). Change-Id: If1a9b993edd71be37396b9167f77da271966f85f Reviewed-on: https://go-review.googlesource.com/37015 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-02-13 16:00:09 -08:00
// Weird special instructions for use in the strength reduction of divides.
// These ops compute unsigned (arg0 + arg1) / 2, correct to all
// 32/64 bits, even when the intermediate result of the add has 33/65 bits.
// These ops can assume arg0 >= arg1.
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
// Note: these ops aren't commutative!
cmd/compile: move constant divide strength reduction to SSA rules Currently the conversion from constant divides to multiplies is mostly done during the walk pass. This is suboptimal because SSA can determine that the value being divided by is constant more often (e.g. after inlining). Change-Id: If1a9b993edd71be37396b9167f77da271966f85f Reviewed-on: https://go-review.googlesource.com/37015 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-02-13 16:00:09 -08:00
{name: "Avg32u", argLength: 2, typ: "UInt32"}, // 32-bit platforms only
{name: "Avg64u", argLength: 2, typ: "UInt64"}, // 64-bit platforms only
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
// For Div16, Div32 and Div64, AuxInt non-zero means that the divisor has been proved to be not -1
// or that the dividend is not the most negative value.
cmd/compile: define high bits of AuxInt Previously if we were only using the low bits of AuxInt, the high bits were ignored and could be junk. This CL changes that behavior to define the high bits to be the sign-extended version of the low bits for all cases. There are 2 main benefits: - Deterministic representation. This helps with CSE. (Const8 [0x1]) and (Const8 [0x101]) used to be the same "value" but CSE couldn't see them as such. - Testability. We can check that all ops leave AuxInt in a state consistent with the new rule. In the old scheme, it was hard to check whether a rule correctly used only the low-order bits. Side benefits: - ==0 and !=0 tests are easier. Drawbacks: - This differs from the runtime representation in registers, where it is important that we allow upper bits to be undefined (so we're not sign/zero-extending all the time). - Ops that treat AuxInt as unsigned (shifts, mostly) need to be a bit more careful. Change-Id: I9a685ff27e36dc03287c9ab1cecd6c0b4045c819 Reviewed-on: https://go-review.googlesource.com/21256 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-29 16:39:53 -07:00
{name: "Div8", argLength: 2}, // arg0 / arg1, signed
{name: "Div8u", argLength: 2}, // arg0 / arg1, unsigned
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
{name: "Div16", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Div16u", argLength: 2},
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
{name: "Div32", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Div32u", argLength: 2},
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
{name: "Div64", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Div64u", argLength: 2},
cmd/compile: intrinsify math/big.mulWW, divWW on AMD64 Change-Id: I59f7afa7a5803d19f8b21fe70fc85ef997bb3a85 Reviewed-on: https://go-review.googlesource.com/30542 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2016-10-06 15:43:47 -04:00
{name: "Div128u", argLength: 3}, // arg0:arg1 / arg2 (128-bit divided by 64-bit), returns (q, r)
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
// For Mod16, Mod32 and Mod64, AuxInt non-zero means that the divisor has been proved to be not -1.
cmd/compile: define high bits of AuxInt Previously if we were only using the low bits of AuxInt, the high bits were ignored and could be junk. This CL changes that behavior to define the high bits to be the sign-extended version of the low bits for all cases. There are 2 main benefits: - Deterministic representation. This helps with CSE. (Const8 [0x1]) and (Const8 [0x101]) used to be the same "value" but CSE couldn't see them as such. - Testability. We can check that all ops leave AuxInt in a state consistent with the new rule. In the old scheme, it was hard to check whether a rule correctly used only the low-order bits. Side benefits: - ==0 and !=0 tests are easier. Drawbacks: - This differs from the runtime representation in registers, where it is important that we allow upper bits to be undefined (so we're not sign/zero-extending all the time). - Ops that treat AuxInt as unsigned (shifts, mostly) need to be a bit more careful. Change-Id: I9a685ff27e36dc03287c9ab1cecd6c0b4045c819 Reviewed-on: https://go-review.googlesource.com/21256 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-29 16:39:53 -07:00
{name: "Mod8", argLength: 2}, // arg0 % arg1, signed
{name: "Mod8u", argLength: 2}, // arg0 % arg1, unsigned
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
{name: "Mod16", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Mod16u", argLength: 2},
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
{name: "Mod32", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Mod32u", argLength: 2},
cmd/compile: use proved bounds to remove signed division fix-ups prove is able to find 94 occurrences in std cmd where a divisor can't have the value -1. The change removes the extraneous fix-up code for these cases. Fixes #25239 Change-Id: Ic184de971f47cc57c702eb72805b8e291c14035d Reviewed-on: https://go-review.googlesource.com/c/130215 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-06 19:50:38 +10:00
{name: "Mod64", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Mod64u", argLength: 2},
{name: "And8", argLength: 2, commutative: true}, // arg0 & arg1
{name: "And16", argLength: 2, commutative: true},
{name: "And32", argLength: 2, commutative: true},
{name: "And64", argLength: 2, commutative: true},
{name: "Or8", argLength: 2, commutative: true}, // arg0 | arg1
{name: "Or16", argLength: 2, commutative: true},
{name: "Or32", argLength: 2, commutative: true},
{name: "Or64", argLength: 2, commutative: true},
{name: "Xor8", argLength: 2, commutative: true}, // arg0 ^ arg1
{name: "Xor16", argLength: 2, commutative: true},
{name: "Xor32", argLength: 2, commutative: true},
{name: "Xor64", argLength: 2, commutative: true},
[dev.ssa] cmd/compile/internal/ssa: implement lots of small (<8byte) ops. Lots and lots of ops! Also XOR for good measure. Add a pass to the compiler generator to check that all of the architecture-specific opcodes are handled by genValue. We will catch any missing ones if we come across them during compilation, but probably better to catch them statically. Change-Id: Ic4adfbec55c8257f88117bc732fa664486262868 Reviewed-on: https://go-review.googlesource.com/12813 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-07-28 16:04:50 -07:00
[dev.ssa] cmd/compile/internal/ssa: fix shift operations Convert shift ops to also encode the size of the shift amount. Change signed right shift from using CMOV to using bit twiddles. It is a little bit better (5 instructions instead of 4, but fewer bytes and slightly faster code). It's also a bit faster than the 4-instruction branch version, even with a very predictable branch. As tested on my machine, YMMV. Implement OCOM while we are here. Change-Id: I8ca12dd62fae5d626dc0e6da5d4bbd34fd9640d2 Reviewed-on: https://go-review.googlesource.com/12867 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-07-29 17:07:09 -07:00
// For shifts, AxB means the shifted value has A bits and the shift amount has B bits.
cmd/compile: define high bits of AuxInt Previously if we were only using the low bits of AuxInt, the high bits were ignored and could be junk. This CL changes that behavior to define the high bits to be the sign-extended version of the low bits for all cases. There are 2 main benefits: - Deterministic representation. This helps with CSE. (Const8 [0x1]) and (Const8 [0x101]) used to be the same "value" but CSE couldn't see them as such. - Testability. We can check that all ops leave AuxInt in a state consistent with the new rule. In the old scheme, it was hard to check whether a rule correctly used only the low-order bits. Side benefits: - ==0 and !=0 tests are easier. Drawbacks: - This differs from the runtime representation in registers, where it is important that we allow upper bits to be undefined (so we're not sign/zero-extending all the time). - Ops that treat AuxInt as unsigned (shifts, mostly) need to be a bit more careful. Change-Id: I9a685ff27e36dc03287c9ab1cecd6c0b4045c819 Reviewed-on: https://go-review.googlesource.com/21256 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-29 16:39:53 -07:00
// Shift amounts are considered unsigned.
cmd/compile: ensure constant shift amounts are in range for arm Ensure constant shift amounts are in the range [0-31]. When shift amounts are out of range, bad things happen. Shift amounts out of range occur when lowering 64-bit shifts (we take an in-range shift s in [0-63] and calculate s-32 and 32-s, both of which might be out of [0-31]). The constant shift operations themselves still work, but their shift amounts get copied unmolested to operations like ORshiftLL which use only the low 5 bits. That changes an operation like <<100 which unconditionally produces 0, to <<4, which doesn't. Fixes #48476 Change-Id: I87363ef2b4ceaf3b2e316426064626efdfbb8ee3 Reviewed-on: https://go-review.googlesource.com/c/go/+/350969 Trust: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-09-19 21:09:57 -07:00
// If arg1 is known to be nonnegative and less than the number of bits in arg0,
cmd/compile: mention the AuxInt field for bounded shifts This comment needs updating after moving the "bounded" mark from Aux to AuxInt. Change-Id: I924fb22a81fffcd6944b93f0e3357c3aa2c4c49e Reviewed-on: https://go-review.googlesource.com/111880 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-07 13:42:28 -07:00
// then auxInt may be set to 1.
cmd/compile: simplify shifts using bounds from prove pass The prove pass sometimes has bounds information that later rewrite passes do not. Use this information to mark shifts as bounded, and then use that information to generate better code on amd64. It may prove to be helpful on other architectures, too. While here, coalesce the existing shift lowering rules. This triggers 35 times building std+cmd. The full list is below. Here's an example from runtime.heapBitsSetType: if nb < 8 { b |= uintptr(*p) << nb p = add1(p) } else { nb -= 8 } We now generate better code on amd64 for that left shift. Updates #25087 vendor/golang_org/x/crypto/curve25519/mont25519_amd64.go:48:20: Proved Rsh8Ux64 bounded runtime/mbitmap.go:1252:22: Proved Lsh64x64 bounded runtime/mbitmap.go:1265:16: Proved Lsh64x64 bounded runtime/mbitmap.go:1275:28: Proved Lsh64x64 bounded runtime/mbitmap.go:1645:25: Proved Lsh64x64 bounded runtime/mbitmap.go:1663:25: Proved Lsh64x64 bounded runtime/mbitmap.go:1808:41: Proved Lsh64x64 bounded runtime/mbitmap.go:1831:49: Proved Lsh64x64 bounded syscall/route_bsd.go:227:23: Proved Lsh32x64 bounded syscall/route_bsd.go:295:23: Proved Lsh32x64 bounded syscall/route_darwin.go:40:23: Proved Lsh32x64 bounded compress/bzip2/bzip2.go:384:26: Proved Lsh64x16 bounded vendor/golang_org/x/net/route/address.go:370:14: Proved Lsh64x64 bounded compress/flate/inflate.go:201:54: Proved Lsh64x64 bounded math/big/prime.go:50:25: Proved Lsh64x64 bounded vendor/golang_org/x/crypto/cryptobyte/asn1.go:464:43: Proved Lsh8x8 bounded net/ip.go:87:21: Proved Rsh8Ux64 bounded cmd/internal/goobj/read.go:267:23: Proved Lsh64x64 bounded cmd/vendor/golang.org/x/arch/arm64/arm64asm/decode.go:534:27: Proved Lsh32x32 bounded cmd/vendor/golang.org/x/arch/arm64/arm64asm/decode.go:544:27: Proved Lsh32x32 bounded cmd/internal/obj/arm/asm5.go:1044:16: Proved Lsh32x64 bounded cmd/internal/obj/arm/asm5.go:1065:10: Proved Lsh32x32 bounded cmd/internal/obj/mips/obj0.go:1311:21: Proved Lsh32x64 bounded cmd/compile/internal/syntax/scanner.go:352:23: Proved Lsh64x64 bounded go/types/expr.go:222:36: Proved Lsh64x64 bounded crypto/x509/x509.go:1626:9: Proved Rsh8Ux64 bounded cmd/link/internal/loadelf/ldelf.go:823:22: Proved Lsh8x64 bounded net/http/h2_bundle.go:1470:17: Proved Lsh8x8 bounded net/http/h2_bundle.go:1477:46: Proved Lsh8x8 bounded net/http/h2_bundle.go:1481:31: Proved Lsh64x8 bounded cmd/compile/internal/ssa/rewriteARM64.go:18759:17: Proved Lsh64x64 bounded cmd/compile/internal/ssa/sparsemap.go:70:23: Proved Lsh32x64 bounded cmd/compile/internal/ssa/sparsemap.go:73:45: Proved Lsh32x64 bounded Change-Id: I58bb72f3e6f12f6ac69be633ea7222c245438142 Reviewed-on: https://go-review.googlesource.com/109776 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Giovanni Bajo <rasky@develer.com>
2018-04-26 20:56:03 -07:00
// This enables better code generation on some platforms.
{name: "Lsh8x8", argLength: 2, aux: "Bool"}, // arg0 << arg1
{name: "Lsh8x16", argLength: 2, aux: "Bool"},
{name: "Lsh8x32", argLength: 2, aux: "Bool"},
{name: "Lsh8x64", argLength: 2, aux: "Bool"},
{name: "Lsh16x8", argLength: 2, aux: "Bool"},
{name: "Lsh16x16", argLength: 2, aux: "Bool"},
{name: "Lsh16x32", argLength: 2, aux: "Bool"},
{name: "Lsh16x64", argLength: 2, aux: "Bool"},
{name: "Lsh32x8", argLength: 2, aux: "Bool"},
{name: "Lsh32x16", argLength: 2, aux: "Bool"},
{name: "Lsh32x32", argLength: 2, aux: "Bool"},
{name: "Lsh32x64", argLength: 2, aux: "Bool"},
{name: "Lsh64x8", argLength: 2, aux: "Bool"},
{name: "Lsh64x16", argLength: 2, aux: "Bool"},
{name: "Lsh64x32", argLength: 2, aux: "Bool"},
{name: "Lsh64x64", argLength: 2, aux: "Bool"},
{name: "Rsh8x8", argLength: 2, aux: "Bool"}, // arg0 >> arg1, signed
{name: "Rsh8x16", argLength: 2, aux: "Bool"},
{name: "Rsh8x32", argLength: 2, aux: "Bool"},
{name: "Rsh8x64", argLength: 2, aux: "Bool"},
{name: "Rsh16x8", argLength: 2, aux: "Bool"},
{name: "Rsh16x16", argLength: 2, aux: "Bool"},
{name: "Rsh16x32", argLength: 2, aux: "Bool"},
{name: "Rsh16x64", argLength: 2, aux: "Bool"},
{name: "Rsh32x8", argLength: 2, aux: "Bool"},
{name: "Rsh32x16", argLength: 2, aux: "Bool"},
{name: "Rsh32x32", argLength: 2, aux: "Bool"},
{name: "Rsh32x64", argLength: 2, aux: "Bool"},
{name: "Rsh64x8", argLength: 2, aux: "Bool"},
{name: "Rsh64x16", argLength: 2, aux: "Bool"},
{name: "Rsh64x32", argLength: 2, aux: "Bool"},
{name: "Rsh64x64", argLength: 2, aux: "Bool"},
{name: "Rsh8Ux8", argLength: 2, aux: "Bool"}, // arg0 >> arg1, unsigned
{name: "Rsh8Ux16", argLength: 2, aux: "Bool"},
{name: "Rsh8Ux32", argLength: 2, aux: "Bool"},
{name: "Rsh8Ux64", argLength: 2, aux: "Bool"},
{name: "Rsh16Ux8", argLength: 2, aux: "Bool"},
{name: "Rsh16Ux16", argLength: 2, aux: "Bool"},
{name: "Rsh16Ux32", argLength: 2, aux: "Bool"},
{name: "Rsh16Ux64", argLength: 2, aux: "Bool"},
{name: "Rsh32Ux8", argLength: 2, aux: "Bool"},
{name: "Rsh32Ux16", argLength: 2, aux: "Bool"},
{name: "Rsh32Ux32", argLength: 2, aux: "Bool"},
{name: "Rsh32Ux64", argLength: 2, aux: "Bool"},
{name: "Rsh64Ux8", argLength: 2, aux: "Bool"},
{name: "Rsh64Ux16", argLength: 2, aux: "Bool"},
{name: "Rsh64Ux32", argLength: 2, aux: "Bool"},
{name: "Rsh64Ux64", argLength: 2, aux: "Bool"},
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// 2-input comparisons
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
{name: "Eq8", argLength: 2, commutative: true, typ: "Bool"}, // arg0 == arg1
{name: "Eq16", argLength: 2, commutative: true, typ: "Bool"},
{name: "Eq32", argLength: 2, commutative: true, typ: "Bool"},
{name: "Eq64", argLength: 2, commutative: true, typ: "Bool"},
{name: "EqPtr", argLength: 2, commutative: true, typ: "Bool"},
{name: "EqInter", argLength: 2, typ: "Bool"}, // arg0 or arg1 is nil; other cases handled by frontend
{name: "EqSlice", argLength: 2, typ: "Bool"}, // arg0 or arg1 is nil; other cases handled by frontend
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
{name: "Eq32F", argLength: 2, commutative: true, typ: "Bool"},
{name: "Eq64F", argLength: 2, commutative: true, typ: "Bool"},
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
{name: "Neq8", argLength: 2, commutative: true, typ: "Bool"}, // arg0 != arg1
{name: "Neq16", argLength: 2, commutative: true, typ: "Bool"},
{name: "Neq32", argLength: 2, commutative: true, typ: "Bool"},
{name: "Neq64", argLength: 2, commutative: true, typ: "Bool"},
{name: "NeqPtr", argLength: 2, commutative: true, typ: "Bool"},
{name: "NeqInter", argLength: 2, typ: "Bool"}, // arg0 or arg1 is nil; other cases handled by frontend
{name: "NeqSlice", argLength: 2, typ: "Bool"}, // arg0 or arg1 is nil; other cases handled by frontend
cmd/compile: automatically handle commuting ops in rewrite rules Note that this is a redo of an undo of the original buggy CL 38666. We have lots of rewrite rules that vary only in the fact that we have 2 versions for the 2 different orderings of various commuting ops. For example: (ADDL x (MOVLconst [c])) -> (ADDLconst [c] x) (ADDL (MOVLconst [c]) x) -> (ADDLconst [c] x) It can get unwieldly quickly, especially when there is more than one commuting op in a rule. Our existing "fix" for this problem is to have rules that canonicalize the operations first. For example: (Eq64 x (Const64 <t> [c])) && x.Op != OpConst64 -> (Eq64 (Const64 <t> [c]) x) Subsequent rules can then assume if there is a constant arg to Eq64, it will be the first one. This fix kinda works, but it is fragile and only works when we remember to include the required extra rules. The fundamental problem is that the rule matcher doesn't know anything about commuting ops. This CL fixes that fact. We already have information about which ops commute. (The register allocator takes advantage of commutivity.) The rule generator now automatically generates multiple rules for a single source rule when there are commutative ops in the rule. We can now drop all of our almost-duplicate source-level rules and the canonicalization rules. I have some CLs in progress that will be a lot less verbose when the rule generator handles commutivity for me. I had to reorganize the load-combining rules a bit. The 8-way OR rules generated 128 different reorderings, which was causing the generator to put too much code in the rewrite*.go files (the big ones were going from 25K lines to 132K lines). Instead I reorganized the rules to combine pairs of loads at a time. The generated rule files are now actually a bit (5%) smaller. Make.bash times are ~unchanged. Compiler benchmarks are not observably different. Probably because we don't spend much compiler time in rule matching anyway. I've also done a pass over all of our ops adding commutative markings for ops which hadn't had them previously. Fixes #18292 Change-Id: Ic1c0e43fbf579539f459971625f69690c9ab8805 Reviewed-on: https://go-review.googlesource.com/38801 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2017-03-30 03:30:22 +00:00
{name: "Neq32F", argLength: 2, commutative: true, typ: "Bool"},
{name: "Neq64F", argLength: 2, commutative: true, typ: "Bool"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
{name: "Less8", argLength: 2, typ: "Bool"}, // arg0 < arg1, signed
{name: "Less8U", argLength: 2, typ: "Bool"}, // arg0 < arg1, unsigned
{name: "Less16", argLength: 2, typ: "Bool"},
{name: "Less16U", argLength: 2, typ: "Bool"},
{name: "Less32", argLength: 2, typ: "Bool"},
{name: "Less32U", argLength: 2, typ: "Bool"},
{name: "Less64", argLength: 2, typ: "Bool"},
{name: "Less64U", argLength: 2, typ: "Bool"},
{name: "Less32F", argLength: 2, typ: "Bool"},
{name: "Less64F", argLength: 2, typ: "Bool"},
{name: "Leq8", argLength: 2, typ: "Bool"}, // arg0 <= arg1, signed
{name: "Leq8U", argLength: 2, typ: "Bool"}, // arg0 <= arg1, unsigned
{name: "Leq16", argLength: 2, typ: "Bool"},
{name: "Leq16U", argLength: 2, typ: "Bool"},
{name: "Leq32", argLength: 2, typ: "Bool"},
{name: "Leq32U", argLength: 2, typ: "Bool"},
{name: "Leq64", argLength: 2, typ: "Bool"},
{name: "Leq64U", argLength: 2, typ: "Bool"},
{name: "Leq32F", argLength: 2, typ: "Bool"},
{name: "Leq64F", argLength: 2, typ: "Bool"},
cmd/compile/internal/ssa: emit csel on arm64 Introduce a new SSA pass to generate CondSelect intstrutions, and add CondSelect lowering rules for arm64. In order to make the CSEL instruction easier to optimize, and to simplify the introduction of CSNEG, CSINC, and CSINV in the future, modify the CSEL instruction to accept a condition code in the aux field. Notably, this change makes the go1 Gzip benchmark more than 10% faster. Benchmarks on a Cavium ThunderX: name old time/op new time/op delta BinaryTree17-96 15.9s ± 6% 16.0s ± 4% ~ (p=0.968 n=10+9) Fannkuch11-96 7.17s ± 0% 7.00s ± 0% -2.43% (p=0.000 n=8+9) FmtFprintfEmpty-96 208ns ± 1% 207ns ± 0% ~ (p=0.152 n=10+8) FmtFprintfString-96 379ns ± 0% 375ns ± 0% -0.95% (p=0.000 n=10+9) FmtFprintfInt-96 385ns ± 0% 383ns ± 0% -0.52% (p=0.000 n=9+10) FmtFprintfIntInt-96 591ns ± 0% 586ns ± 0% -0.85% (p=0.006 n=7+9) FmtFprintfPrefixedInt-96 656ns ± 0% 667ns ± 0% +1.71% (p=0.000 n=10+10) FmtFprintfFloat-96 967ns ± 0% 984ns ± 0% +1.78% (p=0.000 n=10+10) FmtManyArgs-96 2.35µs ± 0% 2.25µs ± 0% -4.63% (p=0.000 n=9+8) GobDecode-96 31.0ms ± 0% 30.8ms ± 0% -0.36% (p=0.006 n=9+9) GobEncode-96 24.4ms ± 0% 24.5ms ± 0% +0.30% (p=0.000 n=9+9) Gzip-96 1.60s ± 0% 1.43s ± 0% -10.58% (p=0.000 n=9+10) Gunzip-96 167ms ± 0% 169ms ± 0% +0.83% (p=0.000 n=8+9) HTTPClientServer-96 311µs ± 1% 308µs ± 0% -0.75% (p=0.000 n=10+10) JSONEncode-96 65.0ms ± 0% 64.8ms ± 0% -0.25% (p=0.000 n=9+8) JSONDecode-96 262ms ± 1% 261ms ± 1% ~ (p=0.579 n=10+10) Mandelbrot200-96 18.0ms ± 0% 18.1ms ± 0% +0.17% (p=0.000 n=8+10) GoParse-96 14.0ms ± 0% 14.1ms ± 1% +0.42% (p=0.003 n=9+10) RegexpMatchEasy0_32-96 644ns ± 2% 645ns ± 2% ~ (p=0.836 n=10+10) RegexpMatchEasy0_1K-96 3.70µs ± 0% 3.49µs ± 0% -5.58% (p=0.000 n=10+10) RegexpMatchEasy1_32-96 662ns ± 2% 657ns ± 2% ~ (p=0.137 n=10+10) RegexpMatchEasy1_1K-96 4.47µs ± 0% 4.31µs ± 0% -3.48% (p=0.000 n=10+10) RegexpMatchMedium_32-96 844ns ± 2% 849ns ± 1% ~ (p=0.208 n=10+10) RegexpMatchMedium_1K-96 179µs ± 0% 182µs ± 0% +1.20% (p=0.000 n=10+10) RegexpMatchHard_32-96 10.0µs ± 0% 10.1µs ± 0% +0.48% (p=0.000 n=10+9) RegexpMatchHard_1K-96 297µs ± 0% 297µs ± 0% -0.14% (p=0.000 n=10+10) Revcomp-96 3.08s ± 0% 3.13s ± 0% +1.56% (p=0.000 n=9+9) Template-96 276ms ± 2% 275ms ± 1% ~ (p=0.393 n=10+10) TimeParse-96 1.37µs ± 0% 1.36µs ± 0% -0.53% (p=0.000 n=10+7) TimeFormat-96 1.40µs ± 0% 1.42µs ± 0% +0.97% (p=0.000 n=10+10) [Geo mean] 264µs 262µs -0.77% Change-Id: Ie54eee4b3092af53e6da3baa6d1755098f57f3a2 Reviewed-on: https://go-review.googlesource.com/55670 Run-TryBot: Philip Hofer <phofer@umich.edu> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2017-08-13 22:36:47 +00:00
// the type of a CondSelect is the same as the type of its first
// two arguments, which should be register-width scalars; the third
// argument should be a boolean
{name: "CondSelect", argLength: 3}, // arg2 ? arg0 : arg1
cmd/compile: introduce bool operations. Introduce OrB, EqB, NeqB, AndB to handle bool operations. Change-Id: I53e4d5125a8090d5eeb4576db619103f19fff58d Reviewed-on: https://go-review.googlesource.com/22412 Reviewed-by: Keith Randall <khr@golang.org>
2016-04-24 21:21:07 +02:00
// boolean ops
cmd/compile/internal/ssa: mark boolean instructions commutative Mark AndB, OrB, EqB, and NeqB as commutative. Change-Id: Ife7cfcb9780cc5dd669617cb52339ab336667da4 Reviewed-on: https://go-review.googlesource.com/42515 Reviewed-by: Giovanni Bajo <rasky@develer.com> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-05-03 13:33:14 +02:00
{name: "AndB", argLength: 2, commutative: true, typ: "Bool"}, // arg0 && arg1 (not shortcircuited)
{name: "OrB", argLength: 2, commutative: true, typ: "Bool"}, // arg0 || arg1 (not shortcircuited)
{name: "EqB", argLength: 2, commutative: true, typ: "Bool"}, // arg0 == arg1
{name: "NeqB", argLength: 2, commutative: true, typ: "Bool"}, // arg0 != arg1
{name: "Not", argLength: 1, typ: "Bool"}, // !arg0, boolean
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
cmd/compile: introduce bool operations. Introduce OrB, EqB, NeqB, AndB to handle bool operations. Change-Id: I53e4d5125a8090d5eeb4576db619103f19fff58d Reviewed-on: https://go-review.googlesource.com/22412 Reviewed-by: Keith Randall <khr@golang.org>
2016-04-24 21:21:07 +02:00
// 1-input ops
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Neg8", argLength: 1}, // -arg0
{name: "Neg16", argLength: 1},
{name: "Neg32", argLength: 1},
{name: "Neg64", argLength: 1},
{name: "Neg32F", argLength: 1},
{name: "Neg64F", argLength: 1},
{name: "Com8", argLength: 1}, // ^arg0
{name: "Com16", argLength: 1},
{name: "Com32", argLength: 1},
{name: "Com64", argLength: 1},
cmd/compile: use prove pass to detect Ctz of non-zero values On amd64, Ctz must include special handling of zeros. But the prove pass has enough information to detect whether the input is non-zero, allowing a more efficient lowering. Introduce new CtzNonZero ops to capture and use this information. Benchmark code: func BenchmarkVisitBits(b *testing.B) { b.Run("8", func(b *testing.B) { for i := 0; i < b.N; i++ { x := uint8(0xff) for x != 0 { sink = bits.TrailingZeros8(x) x &= x - 1 } } }) // and similarly so for 16, 32, 64 } name old time/op new time/op delta VisitBits/8-8 7.27ns ± 4% 5.58ns ± 4% -23.35% (p=0.000 n=28+26) VisitBits/16-8 14.7ns ± 7% 10.5ns ± 4% -28.43% (p=0.000 n=30+28) VisitBits/32-8 27.6ns ± 8% 19.3ns ± 3% -30.14% (p=0.000 n=30+26) VisitBits/64-8 44.0ns ±11% 38.0ns ± 5% -13.48% (p=0.000 n=30+30) Fixes #25077 Change-Id: Ie6e5bd86baf39ee8a4ca7cadcf56d934e047f957 Reviewed-on: https://go-review.googlesource.com/109358 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-04-25 11:52:06 -07:00
{name: "Ctz8", argLength: 1}, // Count trailing (low order) zeroes (returns 0-8)
{name: "Ctz16", argLength: 1}, // Count trailing (low order) zeroes (returns 0-16)
{name: "Ctz32", argLength: 1}, // Count trailing (low order) zeroes (returns 0-32)
{name: "Ctz64", argLength: 1}, // Count trailing (low order) zeroes (returns 0-64)
cmd/compile: optimize Ctz64 on 386 Compared with the version generated by dec64.rules based on Ctz32, the number of assembly instructions is reduced by half. SwissMap uses TrailingZeros64 to find the first match in its control group and may benefit from this CL on 386 architectures. goos: linux goarch: 386 cpu: 13th Gen Intel(R) Core(TM) i7-13700H │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ TrailingZeros64-20 0.8828n ± 1% 0.6299n ± 1% -28.65% (p=0.000 n=20) Change-Id: Iba08a3f4e13efd3349715dfb7fcd5fd470286cd3 Reviewed-on: https://go-review.googlesource.com/c/go/+/624376 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Keith Randall <khr@golang.org>
2024-10-22 17:18:11 +08:00
{name: "Ctz64On32", argLength: 2}, // Count trailing (low order) zeroes (returns 0-64) in arg[1]<<32+arg[0]
cmd/compile: use prove pass to detect Ctz of non-zero values On amd64, Ctz must include special handling of zeros. But the prove pass has enough information to detect whether the input is non-zero, allowing a more efficient lowering. Introduce new CtzNonZero ops to capture and use this information. Benchmark code: func BenchmarkVisitBits(b *testing.B) { b.Run("8", func(b *testing.B) { for i := 0; i < b.N; i++ { x := uint8(0xff) for x != 0 { sink = bits.TrailingZeros8(x) x &= x - 1 } } }) // and similarly so for 16, 32, 64 } name old time/op new time/op delta VisitBits/8-8 7.27ns ± 4% 5.58ns ± 4% -23.35% (p=0.000 n=28+26) VisitBits/16-8 14.7ns ± 7% 10.5ns ± 4% -28.43% (p=0.000 n=30+28) VisitBits/32-8 27.6ns ± 8% 19.3ns ± 3% -30.14% (p=0.000 n=30+26) VisitBits/64-8 44.0ns ±11% 38.0ns ± 5% -13.48% (p=0.000 n=30+30) Fixes #25077 Change-Id: Ie6e5bd86baf39ee8a4ca7cadcf56d934e047f957 Reviewed-on: https://go-review.googlesource.com/109358 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-04-25 11:52:06 -07:00
{name: "Ctz8NonZero", argLength: 1}, // same as above, but arg[0] known to be non-zero, returns 0-7
{name: "Ctz16NonZero", argLength: 1}, // same as above, but arg[0] known to be non-zero, returns 0-15
{name: "Ctz32NonZero", argLength: 1}, // same as above, but arg[0] known to be non-zero, returns 0-31
{name: "Ctz64NonZero", argLength: 1}, // same as above, but arg[0] known to be non-zero, returns 0-63
{name: "BitLen8", argLength: 1}, // Number of bits in arg[0] (returns 0-8)
{name: "BitLen16", argLength: 1}, // Number of bits in arg[0] (returns 0-16)
{name: "BitLen32", argLength: 1}, // Number of bits in arg[0] (returns 0-32)
{name: "BitLen64", argLength: 1}, // Number of bits in arg[0] (returns 0-64)
cmd/compile: added some intrinsics to SSA back end One intrinsic was needed to help get the very best performance out of a future GC; as long as that one was being added, I also added Bswap since that is sometimes a handy thing to have. I had intended to fill out the bit-scan intrinsic family, but the mismatch between the "scan forward" instruction and "count leading zeroes" was large enough to cause me to leave it out -- it poses a dilemma that I'd rather dodge right now. These intrinsics are not exposed for general use. That's a separate issue requiring an API proposal change ( https://github.com/golang/proposal ) All intrinsics are tested, both that they are substituted on the appropriate architecture, and that they produce the expected result. Change-Id: I5848037cfd97de4f75bdc33bdd89bba00af4a8ee Reviewed-on: https://go-review.googlesource.com/20564 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-11 00:10:52 -05:00
cmd/compile: intrinsify math/bits/ReverseBytes{16|32|64} for ppc64/power10 This change intrinsifies ReverseBytes{16|32|64} by generating the corresponding new instructions in Power10: brh, brd and brw and adds a verification test for the same. On Power 9 and 8, the .go code performs optimally as it is. Performance improvement seen on Power10: ReverseBytes32 1.38ns ± 0% 1.18ns ± 0% -14.2 ReverseBytes64 1.52ns ± 0% 1.11ns ± 0% -26.87 ReverseBytes16 1.41ns ± 1% 1.18ns ± 0% -16.47 Change-Id: I88f127f3ab9ba24a772becc21ad90acfba324b37 Reviewed-on: https://go-review.googlesource.com/c/go/+/446675 Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-10-31 11:47:17 -05:00
{name: "Bswap16", argLength: 1}, // Swap bytes
cmd/compile: added some intrinsics to SSA back end One intrinsic was needed to help get the very best performance out of a future GC; as long as that one was being added, I also added Bswap since that is sometimes a handy thing to have. I had intended to fill out the bit-scan intrinsic family, but the mismatch between the "scan forward" instruction and "count leading zeroes" was large enough to cause me to leave it out -- it poses a dilemma that I'd rather dodge right now. These intrinsics are not exposed for general use. That's a separate issue requiring an API proposal change ( https://github.com/golang/proposal ) All intrinsics are tested, both that they are substituted on the appropriate architecture, and that they produce the expected result. Change-Id: I5848037cfd97de4f75bdc33bdd89bba00af4a8ee Reviewed-on: https://go-review.googlesource.com/20564 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-11 00:10:52 -05:00
{name: "Bswap32", argLength: 1}, // Swap bytes
{name: "Bswap64", argLength: 1}, // Swap bytes
cmd/compile: intrinsic for math/bits.Reverse on ARM64 I don't know that it exists for any other architectures. Update #18616 Change-Id: Idfe5dee251764d32787915889ec0be4bebc5be24 Reviewed-on: https://go-review.googlesource.com/38323 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-03-16 22:34:38 -07:00
{name: "BitRev8", argLength: 1}, // Reverse the bits in arg[0]
{name: "BitRev16", argLength: 1}, // Reverse the bits in arg[0]
{name: "BitRev32", argLength: 1}, // Reverse the bits in arg[0]
{name: "BitRev64", argLength: 1}, // Reverse the bits in arg[0]
cmd/compile: move SSA rotate instruction detection to arch-independent rules Detect rotate instructions while still in architecture-independent form. It's easier to do here, and we don't need to repeat it in each architecture file. Change-Id: I9396954b3f3b3bfb96c160d064a02002309935bb Reviewed-on: https://go-review.googlesource.com/c/go/+/421195 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Eric Fang <eric.fang@arm.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Joedian Reid <joedian@golang.org> Reviewed-by: Ruinan Sun <Ruinan.Sun@arm.com> Run-TryBot: Keith Randall <khr@golang.org>
2022-08-03 22:58:30 -07:00
{name: "PopCount8", argLength: 1}, // Count bits in arg[0]
{name: "PopCount16", argLength: 1}, // Count bits in arg[0]
{name: "PopCount32", argLength: 1}, // Count bits in arg[0]
{name: "PopCount64", argLength: 1}, // Count bits in arg[0]
// RotateLeftX instructions rotate the X bits of arg[0] to the left
// by the low lg_2(X) bits of arg[1], interpreted as an unsigned value.
// Note that this works out regardless of the bit width or signedness of
// arg[1]. In particular, RotateLeft by x is the same as RotateRight by -x.
{name: "RotateLeft64", argLength: 2},
{name: "RotateLeft32", argLength: 2},
{name: "RotateLeft16", argLength: 2},
{name: "RotateLeft8", argLength: 2},
cmd/compile: intrinsics for math/bits.OnesCount Popcount instructions on amd64 are not guaranteed to be present, so we must guard their call. Rewrite rules can't generate control flow at the moment, so the intrinsifier needs to generate that code. name old time/op new time/op delta OnesCount-8 2.47ns ± 5% 1.04ns ± 2% -57.70% (p=0.000 n=10+10) OnesCount16-8 1.05ns ± 1% 0.78ns ± 0% -25.56% (p=0.000 n=9+8) OnesCount32-8 1.63ns ± 5% 1.04ns ± 2% -35.96% (p=0.000 n=10+10) OnesCount64-8 2.45ns ± 0% 1.04ns ± 1% -57.55% (p=0.000 n=6+10) Update #18616 Change-Id: I4aff2cc9aa93787898d7b22055fe272a7cf95673 Reviewed-on: https://go-review.googlesource.com/38320 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-03-16 21:33:03 -07:00
cmd/compile: optimize single-precision floating point square root Add generic rule to rewrite the single-precision square root expression with one single-precision instruction. The optimization will reduce two times of precision converting between double-precision and single-precision. On arm64 flatform. previous: FCVTSD F0, F0 FSQRTD F0, F0 FCVTDS F0, F0 optimized: FSQRTS S0, S0 And this patch adds the test case to check the correctness. This patch refers to CL 241877, contributed by Alice Xu (dianhong.xu@arm.com) Change-Id: I6de5d02281c693017ac4bd4c10963dd55989bd7e Reviewed-on: https://go-review.googlesource.com/c/go/+/276873 Trust: fannie zhang <Fannie.Zhang@arm.com> Run-TryBot: fannie zhang <Fannie.Zhang@arm.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-12-07 19:15:15 +08:00
// Square root.
cmd/compile: add s390x intrinsics for Ceil, Floor, Round and Trunc Ceil, Floor and Trunc are pre-existing intrinsics. Round is a new function and has been added as an intrinsic in this CL. All of the functions can be implemented as a single 'LOAD FP INTEGER' instruction, FIDBR, on s390x. name old time/op new time/op delta Ceil 2.34ns ± 0% 0.85ns ± 0% -63.74% (p=0.000 n=5+4) Floor 2.33ns ± 0% 0.85ns ± 1% -63.35% (p=0.008 n=5+5) Round 4.23ns ± 0% 0.85ns ± 0% -79.89% (p=0.000 n=5+4) Trunc 2.35ns ± 0% 0.85ns ± 0% -63.83% (p=0.029 n=4+4) Change-Id: Idee7ba24a2899d12bf9afee4eedd6b4aaad3c510 Reviewed-on: https://go-review.googlesource.com/63890 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-09-14 20:00:02 +01:00
// Special cases:
// +∞ → +∞
// ±0 → ±0 (sign preserved)
// x<0 → NaN
// NaN → NaN
cmd/compile: fix whitespace in comment The whitespace was there to align with the following comment, but the extra whitespace was unnecessary; it wasn't gofmt'd. Then the file got gofmt'd, but the whitespace didn't get fixed. Change-Id: I45aad9605b99d83545e4e611ae3ea1b2ff9e6bf6 Reviewed-on: https://go-review.googlesource.com/c/go/+/301649 Trust: Josh Bleecher Snyder <josharian@gmail.com> Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-14 14:27:06 -07:00
{name: "Sqrt", argLength: 1}, // √arg0 (floating point, double precision)
cmd/compile: optimize single-precision floating point square root Add generic rule to rewrite the single-precision square root expression with one single-precision instruction. The optimization will reduce two times of precision converting between double-precision and single-precision. On arm64 flatform. previous: FCVTSD F0, F0 FSQRTD F0, F0 FCVTDS F0, F0 optimized: FSQRTS S0, S0 And this patch adds the test case to check the correctness. This patch refers to CL 241877, contributed by Alice Xu (dianhong.xu@arm.com) Change-Id: I6de5d02281c693017ac4bd4c10963dd55989bd7e Reviewed-on: https://go-review.googlesource.com/c/go/+/276873 Trust: fannie zhang <Fannie.Zhang@arm.com> Run-TryBot: fannie zhang <Fannie.Zhang@arm.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-12-07 19:15:15 +08:00
{name: "Sqrt32", argLength: 1}, // √arg0 (floating point, single precision)
cmd/compile: add s390x intrinsics for Ceil, Floor, Round and Trunc Ceil, Floor and Trunc are pre-existing intrinsics. Round is a new function and has been added as an intrinsic in this CL. All of the functions can be implemented as a single 'LOAD FP INTEGER' instruction, FIDBR, on s390x. name old time/op new time/op delta Ceil 2.34ns ± 0% 0.85ns ± 0% -63.74% (p=0.000 n=5+4) Floor 2.33ns ± 0% 0.85ns ± 1% -63.35% (p=0.008 n=5+5) Round 4.23ns ± 0% 0.85ns ± 0% -79.89% (p=0.000 n=5+4) Trunc 2.35ns ± 0% 0.85ns ± 0% -63.83% (p=0.029 n=4+4) Change-Id: Idee7ba24a2899d12bf9afee4eedd6b4aaad3c510 Reviewed-on: https://go-review.googlesource.com/63890 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-09-14 20:00:02 +01:00
// Round to integer, float64 only.
// Special cases:
// ±∞ → ±∞ (sign preserved)
// ±0 → ±0 (sign preserved)
// NaN → NaN
cmd/compile: intrinsify math.RoundToEven on s390x The new RoundToEven function can be implemented as a single FIDBR instruction on s390x. name old time/op new time/op delta RoundToEven 5.32ns ± 1% 0.86ns ± 1% -83.86% (p=0.000 n=10+10) Change-Id: Iaf597e57a0d1085961701e3c75ff4f6f6dcebb5f Reviewed-on: https://go-review.googlesource.com/74350 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-10-30 09:02:44 -04:00
{name: "Floor", argLength: 1}, // round arg0 toward -∞
{name: "Ceil", argLength: 1}, // round arg0 toward +∞
{name: "Trunc", argLength: 1}, // round arg0 toward 0
{name: "Round", argLength: 1}, // round arg0 to nearest, ties away from 0
{name: "RoundToEven", argLength: 1}, // round arg0 to nearest, ties to even
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
cmd/compile,cmd/internal/obj/ppc64: make math.Abs,math.Copysign instrinsics on ppc64x This adds support for math Abs, Copysign to be instrinsics on ppc64x. New instruction FCPSGN is added to generate fcpsgn. Some new rules are added to improve the int<->float conversions that are generated mainly due to the Float64bits and Float64frombits in the math package. PPC64.rules is also modified as suggested in the review for CL 63290. Improvements: benchmark old ns/op new ns/op delta BenchmarkAbs-16 1.12 0.69 -38.39% BenchmarkCopysign-16 1.30 0.93 -28.46% BenchmarkNextafter32-16 9.34 8.05 -13.81% BenchmarkFrexp-16 8.81 7.60 -13.73% Others that used Copysign also saw smaller improvements. I attempted to make this work using rules since that seems to be preferred, but due to the use of Float64bits and Float64frombits in these functions, several rules had to be added and even then not all cases were matched. Using rules became too complicated and seemed too fragile for these. Updates #21390 Change-Id: Ia265da9a18355e08000818a4fba1a40e9e031995 Reviewed-on: https://go-review.googlesource.com/67130 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Keith Randall <khr@golang.org>
2017-09-28 17:11:31 -04:00
// Modify the sign bit
{name: "Abs", argLength: 1}, // absolute value arg0
{name: "Copysign", argLength: 2}, // copy sign from arg0 to arg1
cmd/compile: use integer min/max instructions on riscv64 When GORISCV64 enables rva22u64, make use of integer MIN/MINU/MAX/MAXU instructions in compiler rewrite rules. Change-Id: I4e7c514516acad03f2869d4c8936f06582cf7ea9 Reviewed-on: https://go-review.googlesource.com/c/go/+/559660 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
2024-02-01 00:39:30 +11:00
// Integer min/max implementation, if hardware is available.
{name: "Min64", argLength: 2}, // min(arg0,arg1), signed
{name: "Max64", argLength: 2}, // max(arg0,arg1), signed
{name: "Min64u", argLength: 2}, // min(arg0,arg1), unsigned
{name: "Max64u", argLength: 2}, // max(arg0,arg1), unsigned
cmd/compile: implement float min/max in hardware for amd64 and arm64 Update #59488 Change-Id: I89f5ea494cbcc887f6fae8560e57bcbd8749be86 Reviewed-on: https://go-review.googlesource.com/c/go/+/514596 Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-07-31 14:08:42 -07:00
// Float min/max implementation, if hardware is available.
{name: "Min64F", argLength: 2}, // min(arg0,arg1)
{name: "Min32F", argLength: 2}, // min(arg0,arg1)
{name: "Max64F", argLength: 2}, // max(arg0,arg1)
{name: "Max32F", argLength: 2}, // max(arg0,arg1)
cmd/compile: introduce generic ssa intrinsic for fused-multiply-add In order to make math.FMA a compiler intrinsic for ISAs like ARM64, PPC64[le], and S390X, a generic 3-argument opcode "Fma" is provided and rewritten as ARM64: (Fma x y z) -> (FMADDD z x y) PPC64: (Fma x y z) -> (FMADD x y z) S390X: (Fma x y z) -> (FMADD z x y) Updates #25819. Change-Id: Ie5bc628311e6feeb28ddf9adaa6e702c8c291efa Reviewed-on: https://go-review.googlesource.com/c/go/+/131959 Run-TryBot: Akhil Indurti <aindurti@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-29 20:57:33 -04:00
// 3-input opcode.
// Fused-multiply-add, float64 only.
// When a*b+c is exactly zero (before rounding), then the result is +0 or -0.
// The 0's sign is determined according to the standard rules for the
// addition (-0 if both a*b and c are -0, +0 otherwise).
//
// Otherwise, when a*b+c rounds to zero, then the resulting 0's sign is
// determined by the sign of the exact result a*b+c.
// See section 6.3 in ieee754.
//
// When the multiply is an infinity times a zero, the result is NaN.
// See section 7.2 in ieee754.
math, cmd/compile: rename Fma to FMA This API was added for #25819, where it was discussed as math.FMA. The commit adding it used math.Fma, presumably for consistency with the rest of the unusual names in package math (Sincos, Acosh, Erfcinv, Float32bits, etc). I believe that using an idiomatic Go name is more important here than consistency with these other names, most of which are historical baggage from C's standard library. Early additions like Float32frombits happened before "uppercase for export" (so they were originally like "float32frombits") and they were not properly reconsidered when we uppercased the symbols to export them. That's a mistake we live with. The names of functions we have added since then, and even a few that were legacy, are more properly Go-cased, such as IsNaN, IsInf, and RoundToEven, rather than Isnan, Isinf, and Roundtoeven. And also constants like MaxFloat32. For new API, we should keep using proper Go-cased symbols instead of minimally-upper-cased-C symbols. So math.FMA, not math.Fma. This API has not yet been released, so this change does not break the compatibility promise. This CL also modifies cmd/compile, since the compiler knows the name of the function. I could have stopped at changing the string constants, but it seemed to make more sense to use a consistent casing everywhere. Change-Id: I0f6f3407f41e99bfa8239467345c33945088896e Reviewed-on: https://go-review.googlesource.com/c/go/+/205317 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-04 19:43:45 -05:00
{name: "FMA", argLength: 3}, // compute (a*b)+c without intermediate rounding
cmd/compile: introduce generic ssa intrinsic for fused-multiply-add In order to make math.FMA a compiler intrinsic for ISAs like ARM64, PPC64[le], and S390X, a generic 3-argument opcode "Fma" is provided and rewritten as ARM64: (Fma x y z) -> (FMADDD z x y) PPC64: (Fma x y z) -> (FMADD x y z) S390X: (Fma x y z) -> (FMADD z x y) Updates #25819. Change-Id: Ie5bc628311e6feeb28ddf9adaa6e702c8c291efa Reviewed-on: https://go-review.googlesource.com/c/go/+/131959 Run-TryBot: Akhil Indurti <aindurti@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-29 20:57:33 -04:00
cmd/compile: update comment about OpPhi argLength We use -1 for Phi argLength, not "a really large number". Change-Id: I5d4fa364c22c3cf40213f2368a9a20de7e989aac Reviewed-on: https://go-review.googlesource.com/c/go/+/203883 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-28 08:32:06 -07:00
// Data movement. Max argument length for Phi is indefinite.
cmd/compile/internal/ssa: note zero-width Ops Add a bool to opInfo to indicate if an Op never results in any instructions. This is a conservative approximation: some operations, like Copy, may or may not generate code depending on their arguments. I built the list by reading each arch's ssaGenValue function. Hopefully I got them all. Change-Id: I130b251b65f18208294e129bb7ddc3f91d57d31d Reviewed-on: https://go-review.googlesource.com/97957 Reviewed-by: Keith Randall <khr@golang.org>
2018-02-28 16:30:07 -05:00
{name: "Phi", argLength: -1, zeroWidth: true}, // select an argument based on which predecessor block we came from
{name: "Copy", argLength: 1}, // output = arg0
[dev.ssa] cmd/compile: be safer about uintptr/unsafe.Pointer conversions Make sure that when a pointer value is live across a function call, we save it as a pointer. (And similarly a uintptr live across a function call should not be saved as a pointer.) Add a nasty test case. This is probably what is preventing the merge from master to dev.ssa. Signs point to something like this bug happening in mallocgc. Change-Id: Ib23fa1251b8d1c50d82c6a448cb4a4fc28219029 Reviewed-on: https://go-review.googlesource.com/16830 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2015-11-10 15:35:36 -08:00
// Convert converts between pointers and integers.
// We have a special op for this so as to not confuse GC
// (particularly stack maps). It takes a memory arg so it
// gets correctly ordered with respect to GC safepoints.
cmd/compile: don't lower OpConvert Currently, each architecture lowers OpConvert to an arch-specific OpXXXconvert. This is silly because OpConvert means the same thing on all architectures and is logically a no-op that exists only to keep track of conversions to and from unsafe.Pointer. Furthermore, lowering it makes it harder to recognize in other analyses, particularly liveness analysis. This CL eliminates the lowering of OpConvert, leaving it as the generic op until code generation time. The main complexity here is that we still need to register-allocate OpConvert operations. Currently, each arch's lowered OpConvert specifies all GP registers in its register mask. Ideally, OpConvert wouldn't affect value homing at all, and we could just copy the home of OpConvert's source, but this can potentially home an OpConvert in a LocalSlot, which neither regalloc nor stackalloc expect. Rather than try to disentangle this assumption from regalloc and stackalloc, we continue to register-allocate OpConvert, but teach regalloc that OpConvert can be allocated to any allocatable GP register. For #24543. Change-Id: I795a6aee5fd94d4444a7bafac3838a400c9f7bb6 Reviewed-on: https://go-review.googlesource.com/108496 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-04-02 16:08:09 -04:00
// It gets compiled to nothing, so its result must in the same
// register as its argument. regalloc knows it can use any
// allocatable integer register for OpConvert.
[dev.ssa] cmd/compile: be safer about uintptr/unsafe.Pointer conversions Make sure that when a pointer value is live across a function call, we save it as a pointer. (And similarly a uintptr live across a function call should not be saved as a pointer.) Add a nasty test case. This is probably what is preventing the merge from master to dev.ssa. Signs point to something like this bug happening in mallocgc. Change-Id: Ib23fa1251b8d1c50d82c6a448cb4a4fc28219029 Reviewed-on: https://go-review.googlesource.com/16830 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2015-11-10 15:35:36 -08:00
// arg0=ptr/int arg1=mem, output=int/ptr
cmd/compile: don't lower OpConvert Currently, each architecture lowers OpConvert to an arch-specific OpXXXconvert. This is silly because OpConvert means the same thing on all architectures and is logically a no-op that exists only to keep track of conversions to and from unsafe.Pointer. Furthermore, lowering it makes it harder to recognize in other analyses, particularly liveness analysis. This CL eliminates the lowering of OpConvert, leaving it as the generic op until code generation time. The main complexity here is that we still need to register-allocate OpConvert operations. Currently, each arch's lowered OpConvert specifies all GP registers in its register mask. Ideally, OpConvert wouldn't affect value homing at all, and we could just copy the home of OpConvert's source, but this can potentially home an OpConvert in a LocalSlot, which neither regalloc nor stackalloc expect. Rather than try to disentangle this assumption from regalloc and stackalloc, we continue to register-allocate OpConvert, but teach regalloc that OpConvert can be allocated to any allocatable GP register. For #24543. Change-Id: I795a6aee5fd94d4444a7bafac3838a400c9f7bb6 Reviewed-on: https://go-review.googlesource.com/108496 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-04-02 16:08:09 -04:00
{name: "Convert", argLength: 2, zeroWidth: true, resultInArg0: true},
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// constants. Constant values are stored in the aux or
[dev.ssa] cmd/compile: optimization for && and || expressions Compiling && and || expressions often leads to control flow of the following form: p: If a goto b else c b: <- p ... x = phi(a, ...) If x goto t else u Note that if we take the edge p->b, then we are guaranteed to take the edge b->t also. So in this situation, we might as well go directly from p to t. Change-Id: I6974f1e6367119a2ddf2014f9741fdb490edcc12 Reviewed-on: https://go-review.googlesource.com/18910 Reviewed-by: David Chase <drchase@google.com>
2016-01-25 09:21:17 -08:00
// auxint fields.
[dev.ssa] cmd/compile: add aux typing, flags to ops Add the aux type to opcodes. Add rematerializeable as a flag. Change-Id: I906e19281498f3ee51bb136299bf26e13a54b2ec Reviewed-on: https://go-review.googlesource.com/19088 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Todd Neal <todd@tneal.org>
2016-01-31 11:39:39 -08:00
{name: "ConstBool", aux: "Bool"}, // auxint is 0 for false and 1 for true
{name: "ConstString", aux: "String"}, // value is aux.(string)
{name: "ConstNil", typ: "BytePtr"}, // nil pointer
cmd/compile: define high bits of AuxInt Previously if we were only using the low bits of AuxInt, the high bits were ignored and could be junk. This CL changes that behavior to define the high bits to be the sign-extended version of the low bits for all cases. There are 2 main benefits: - Deterministic representation. This helps with CSE. (Const8 [0x1]) and (Const8 [0x101]) used to be the same "value" but CSE couldn't see them as such. - Testability. We can check that all ops leave AuxInt in a state consistent with the new rule. In the old scheme, it was hard to check whether a rule correctly used only the low-order bits. Side benefits: - ==0 and !=0 tests are easier. Drawbacks: - This differs from the runtime representation in registers, where it is important that we allow upper bits to be undefined (so we're not sign/zero-extending all the time). - Ops that treat AuxInt as unsigned (shifts, mostly) need to be a bit more careful. Change-Id: I9a685ff27e36dc03287c9ab1cecd6c0b4045c819 Reviewed-on: https://go-review.googlesource.com/21256 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-29 16:39:53 -07:00
{name: "Const8", aux: "Int8"}, // auxint is sign-extended 8 bits
{name: "Const16", aux: "Int16"}, // auxint is sign-extended 16 bits
{name: "Const32", aux: "Int32"}, // auxint is sign-extended 32 bits
cmd/compile: move constant divide strength reduction to SSA rules Currently the conversion from constant divides to multiplies is mostly done during the walk pass. This is suboptimal because SSA can determine that the value being divided by is constant more often (e.g. after inlining). Change-Id: If1a9b993edd71be37396b9167f77da271966f85f Reviewed-on: https://go-review.googlesource.com/37015 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-02-13 16:00:09 -08:00
// Note: ConstX are sign-extended even when the type of the value is unsigned.
// For instance, uint8(0xaa) is stored as auxint=0xffffffffffffffaa.
cmd/compile: don't allow NaNs in floating-point constant ops Trying this CL again, with a fixed test that allows platforms to disagree on the exact behavior of converting NaNs. We store 32-bit floating point constants in a 64-bit field, by converting that 32-bit float to 64-bit float to store it, and convert it back to use it. That works for *almost* all floating-point constants. The exception is signaling NaNs. The round trip described above means we can't represent a 32-bit signaling NaN, because conversions strip the signaling bit. To fix this issue, just forbid NaNs as floating-point constants in SSA form. This shouldn't affect any real-world code, as people seldom constant-propagate NaNs (except in test code). Additionally, NaNs are somewhat underspecified (which of the many NaNs do you get when dividing 0/0?), so when cross-compiling there's a danger of using the compiler machine's NaN regime for some math, and the target machine's NaN regime for other math. Better to use the target machine's NaN regime always. Update #36400 Change-Id: Idf203b688a15abceabbd66ba290d4e9f63619ecb Reviewed-on: https://go-review.googlesource.com/c/go/+/221790 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2020-03-03 17:56:20 +00:00
{name: "Const64", aux: "Int64"}, // value is auxint
// Note: for both Const32F and Const64F, we disallow encoding NaNs.
// Signaling NaNs are tricky because if you do anything with them, they become quiet.
// Particularly, converting a 32 bit sNaN to 64 bit and back converts it to a qNaN.
// See issue 36399 and 36400.
// Encodings of +inf, -inf, and -0 are fine.
cmd/compile: fix typo in comment Change-Id: I25fbd63f10ea9892589ad44cc45761926aff0648 Reviewed-on: https://go-review.googlesource.com/c/go/+/212841 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-12-30 08:35:40 -08:00
{name: "Const32F", aux: "Float32"}, // value is math.Float64frombits(uint64(auxint)) and is exactly representable as float 32
cmd/compile: move constant divide strength reduction to SSA rules Currently the conversion from constant divides to multiplies is mostly done during the walk pass. This is suboptimal because SSA can determine that the value being divided by is constant more often (e.g. after inlining). Change-Id: If1a9b993edd71be37396b9167f77da271966f85f Reviewed-on: https://go-review.googlesource.com/37015 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-02-13 16:00:09 -08:00
{name: "Const64F", aux: "Float64"}, // value is math.Float64frombits(uint64(auxint))
{name: "ConstInterface"}, // nil interface
{name: "ConstSlice"}, // nil slice
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// Constant-like things
cmd/compile/internal/ssa: note zero-width Ops Add a bool to opInfo to indicate if an Op never results in any instructions. This is a conservative approximation: some operations, like Copy, may or may not generate code depending on their arguments. I built the list by reading each arch's ssaGenValue function. Hopefully I got them all. Change-Id: I130b251b65f18208294e129bb7ddc3f91d57d31d Reviewed-on: https://go-review.googlesource.com/97957 Reviewed-by: Keith Randall <khr@golang.org>
2018-02-28 16:30:07 -05:00
{name: "InitMem", zeroWidth: true}, // memory input to the function.
{name: "Arg", aux: "SymOff", symEffect: "Read", zeroWidth: true}, // argument to the function. aux=GCNode of arg, off = offset in that arg.
[dev.ssa] cmd/compile/internal/ssa: Handle variables correctly Use *Node of type ONAME instead of string as the key for variable maps. This will prevent aliasing between two identically named but differently scoped variables. Introduce an Aux value that encodes the offset of a variable from a base pointer (either global base pointer or stack pointer). Allow LEAQ and derivatives (MOVQ, etc.) to also have such an Aux field. Allocate space for AUTO variables in stackalloc. Change-Id: Ibdccdaea4bbc63a1f4882959ac374f2b467e3acd Reviewed-on: https://go-review.googlesource.com/11238 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-06-19 21:02:28 -07:00
cmd/compile: add AMD64 parameter register defs, Arg ops, plumb to ssa.Config This is partial plumbing recycled from the original register abi test work; these are the parts that translate easily. Some other bits are deferred till later when they are ready to be used. For #40724. Change-Id: Ica8c55a4526793446189725a2bc3839124feb38f Reviewed-on: https://go-review.googlesource.com/c/go/+/260539 Trust: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-07 09:44:16 -04:00
// Like Arg, these are generic ops that survive lowering. AuxInt is a register index, and the actual output register for each index is defined by the architecture.
// AuxInt = integer argument index (not a register number). ABI-specified spill loc obtained from function
cmd/compile: make modified Aux type for OpArgXXXX pass ssa/check For #40724. Change-Id: I7d1e76139d187cd15a6e0df9d19542b7200589f6 Reviewed-on: https://go-review.googlesource.com/c/go/+/297911 Trust: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-01 11:02:48 -05:00
{name: "ArgIntReg", aux: "NameOffsetInt8", zeroWidth: true}, // argument to the function in an int reg.
{name: "ArgFloatReg", aux: "NameOffsetInt8", zeroWidth: true}, // argument to the function in a float reg.
cmd/compile: add AMD64 parameter register defs, Arg ops, plumb to ssa.Config This is partial plumbing recycled from the original register abi test work; these are the parts that translate easily. Some other bits are deferred till later when they are ready to be used. For #40724. Change-Id: Ica8c55a4526793446189725a2bc3839124feb38f Reviewed-on: https://go-review.googlesource.com/c/go/+/260539 Trust: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-07 09:44:16 -04:00
cmd/compile: remove Symbol wrappers from Aux fields We used to have {Arg,Auto,Extern}Symbol structs with which we wrapped a *gc.Node or *obj.LSym before storing them in the Aux field of an ssa.Value. This let the SSA part of the compiler distinguish between autos and args, for example. We no longer need the wrappers as we can query the underlying objects directly. There was also some sloppy usage, where VarDef had a *gc.Node directly in its Aux field, whereas the use of that variable had that *gc.Node wrapped in an AutoSymbol. Thus the Aux fields didn't match (using ==) when they probably should. This sloppy usage cleanup is the only thing in the CL that changes the generated code - we can get rid of some more unused auto variables if the matching happens reliably. Removing this wrapper also lets us get rid of the varsyms cache (which was used to prevent wrapping the same *gc.Node twice). Change-Id: I0dedf8f82f84bfee413d310342b777316bd1d478 Reviewed-on: https://go-review.googlesource.com/64452 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-09-18 14:53:56 -07:00
// The address of a variable. arg0 is the base pointer.
// If the variable is a global, the base pointer will be SB and
// the Aux field will be a *obj.LSym.
// If the variable is a local, the base pointer will be SP and
cmd/compile: remove references to *gc.Node in docs ssa.Sym is only implemented by *ir.Name or *obj.LSym. Change-Id: Ia171db618abd8b438fcc2cf402f40f3fe3ec6833 Reviewed-on: https://go-review.googlesource.com/c/go/+/660995 Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-03-26 15:46:30 +01:00
// the Aux field will be a *ir.Name
cmd/compile: add LocalAddr that takes SP,mem operands Lack of a well-defined order between VarDef and related address operations sometimes causes problems with store order and write barrier transformations; glitches in the order are made irreparable (by later optimizations) if the two parts of the glitch straddle a split in the original block caused by insertion of a write barrier diamond. Fix this by creating a LocalAddr for addresses of locals (what VarDef matters for) that takes a memory input to help make the order explicit. Addr is modified to only be legal for SB operand, so there is no overlap between Addr and LocalAddr uses (there may be some downstream cleanup from this). Changes to generic.rules and rewrite.go ensure that codegen tests continue to pass; CSE of LocalAddr is impaired, not quite sure of the cost. Fixes #26105. Change-Id: Id4192b4440aa4e9d7ba54a465c456df9b530b515 Reviewed-on: https://go-review.googlesource.com/122483 Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2018-07-03 11:34:38 -04:00
{name: "Addr", argLength: 1, aux: "Sym", symEffect: "Addr"}, // Address of a variable. Arg0=SB. Aux identifies the variable.
{name: "LocalAddr", argLength: 2, aux: "Sym", symEffect: "Addr"}, // Address of a variable. Arg0=SP. Arg1=mem. Aux identifies the variable.
[dev.ssa] cmd/compile/internal/ssa: Handle variables correctly Use *Node of type ONAME instead of string as the key for variable maps. This will prevent aliasing between two identically named but differently scoped variables. Introduce an Aux value that encodes the offset of a variable from a base pointer (either global base pointer or stack pointer). Allow LEAQ and derivatives (MOVQ, etc.) to also have such an Aux field. Allocate space for AUTO variables in stackalloc. Change-Id: Ibdccdaea4bbc63a1f4882959ac374f2b467e3acd Reviewed-on: https://go-review.googlesource.com/11238 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-06-19 21:02:28 -07:00
cmd/compile: use zero register instead of specialized *zero instructions This lets us get rid of lots of specialized opcodes for storing zero. Instead, use regular store opcodes that just happen to use the zero register as one of their inputs. Change-Id: I2902a6f9b0831cb598df45189ca6bb57221bef72 Reviewed-on: https://go-review.googlesource.com/c/go/+/633075 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
2024-11-24 15:29:56 -08:00
{name: "SP", zeroWidth: true, fixedReg: true}, // stack pointer
{name: "SB", typ: "Uintptr", zeroWidth: true, fixedReg: true}, // static base pointer (a.k.a. globals pointer)
{name: "Invalid"}, // unused value
cmd/compile: add anchored version of SP The SPanchored opcode is identical to SP, except that it takes a memory argument so that it (and more importantly, anything that uses it) must be scheduled at or after that memory argument. This opcode ensures that a LEAQ of a variable gets scheduled after the corresponding VARDEF for that variable. This may lead to less CSE of LEAQ operations. The effect is very small. The go binary is only 80 bytes bigger after this CL. Usually LEAQs get folded into load/store operations, so the effect is only for pointerful types, large enough to need a duffzero, and have their address passed somewhere. Even then, usually the CSEd LEAQs will be un-CSEd because the two uses are on different sides of a function call and the LEAQ ends up being rematerialized at the second use anyway. Change-Id: Ib893562cd05369b91dd563b48fb83f5250950293 Reviewed-on: https://go-review.googlesource.com/c/go/+/452916 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Martin Möhrmann <moehrmann@google.com> Reviewed-by: Martin Möhrmann <martin@golang.org> Reviewed-by: Keith Randall <khr@google.com>
2022-11-21 22:22:36 -08:00
{name: "SPanchored", typ: "Uintptr", argLength: 2, zeroWidth: true}, // arg0 = SP, arg1 = mem. Result is identical to arg0, but cannot be scheduled before memory state arg1.
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// Memory operations
cmd/compile/internal/ssa: inline memmove with known size Replace calls to memmove with known (constant) size, with OpMove. Do it only if it is safe from aliasing point of view. Helps with code like this: append(buf,"const str"...) In strconv this provides nice benefit: Quote-6 731ns ± 2% 647ns ± 3% -11.41% (p=0.000 n=10+10) QuoteRune-6 117ns ± 5% 111ns ± 1% -4.54% (p=0.000 n=10+10) AppendQuote-6 475ns ± 0% 396ns ± 0% -16.59% (p=0.000 n=9+10) AppendQuoteRune-6 32.0ns ± 0% 27.4ns ± 0% -14.41% (p=0.000 n=8+9) Change-Id: I7704f5c51b46aed2d8f033de74c75140fc35036c Reviewed-on: https://go-review.googlesource.com/54394 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-09 14:00:38 -05:00
{name: "Load", argLength: 2}, // Load from arg0. arg1=memory
cmd/compile: add new ops for experiment with late call expansion Added Dereference, StaticLECall, SelectN, SelectNAddr Change-Id: I5426ae488e83956539aa07f7415b8acc26c933e6 Reviewed-on: https://go-review.googlesource.com/c/go/+/239082 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-06-15 14:20:36 -04:00
{name: "Dereference", argLength: 2}, // Load from arg0. arg1=memory. Helper op for arg/result passing, result is an otherwise not-SSA-able "value".
cmd/compile/internal/ssa: inline memmove with known size Replace calls to memmove with known (constant) size, with OpMove. Do it only if it is safe from aliasing point of view. Helps with code like this: append(buf,"const str"...) In strconv this provides nice benefit: Quote-6 731ns ± 2% 647ns ± 3% -11.41% (p=0.000 n=10+10) QuoteRune-6 117ns ± 5% 111ns ± 1% -4.54% (p=0.000 n=10+10) AppendQuote-6 475ns ± 0% 396ns ± 0% -16.59% (p=0.000 n=9+10) AppendQuoteRune-6 32.0ns ± 0% 27.4ns ± 0% -14.41% (p=0.000 n=8+9) Change-Id: I7704f5c51b46aed2d8f033de74c75140fc35036c Reviewed-on: https://go-review.googlesource.com/54394 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-09 14:00:38 -05:00
{name: "Store", argLength: 3, typ: "Mem", aux: "Typ"}, // Store arg1 to arg0. arg2=memory, aux=type. Returns memory.
[dev.simd] cmd/compile: changes for AVX2 SIMD masked load/store This is "glue" changes and hand work for the AVX2 masked loads/stores. Does not include generated function/method declarations or intrinsic registration. Change-Id: Ic95f90b117d0c471f174407ce3f729f1f517b23c Reviewed-on: https://go-review.googlesource.com/c/go/+/689295 Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-16 13:29:14 -04:00
// masked memory operations.
// TODO add 16 and 8
[dev.simd] cmd/compile: opcodes and rules and code generation to enable AVX512 masked loads/stores Change-Id: I9e05fc5031420f60a2e6bac7b9f86365f0f4c0f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/690335 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-07-23 21:04:38 -04:00
{name: "LoadMasked8", argLength: 3}, // Load from arg0, arg1 = mask of 8-bits, arg2 = memory
{name: "LoadMasked16", argLength: 3}, // Load from arg0, arg1 = mask of 16-bits, arg2 = memory
[dev.simd] cmd/compile: changes for AVX2 SIMD masked load/store This is "glue" changes and hand work for the AVX2 masked loads/stores. Does not include generated function/method declarations or intrinsic registration. Change-Id: Ic95f90b117d0c471f174407ce3f729f1f517b23c Reviewed-on: https://go-review.googlesource.com/c/go/+/689295 Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-16 13:29:14 -04:00
{name: "LoadMasked32", argLength: 3}, // Load from arg0, arg1 = mask of 32-bits, arg2 = memory
{name: "LoadMasked64", argLength: 3}, // Load from arg0, arg1 = mask of 64-bits, arg2 = memory
[dev.simd] cmd/compile: opcodes and rules and code generation to enable AVX512 masked loads/stores Change-Id: I9e05fc5031420f60a2e6bac7b9f86365f0f4c0f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/690335 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-07-23 21:04:38 -04:00
{name: "StoreMasked8", argLength: 4, typ: "Mem", aux: "Typ"}, // Store arg2 to arg0, arg1=mask of 8-bits, arg3 = memory
{name: "StoreMasked16", argLength: 4, typ: "Mem", aux: "Typ"}, // Store arg2 to arg0, arg1=mask of 16-bits, arg3 = memory
[dev.simd] cmd/compile: changes for AVX2 SIMD masked load/store This is "glue" changes and hand work for the AVX2 masked loads/stores. Does not include generated function/method declarations or intrinsic registration. Change-Id: Ic95f90b117d0c471f174407ce3f729f1f517b23c Reviewed-on: https://go-review.googlesource.com/c/go/+/689295 Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-16 13:29:14 -04:00
{name: "StoreMasked32", argLength: 4, typ: "Mem", aux: "Typ"}, // Store arg2 to arg0, arg1=mask of 32-bits, arg3 = memory
{name: "StoreMasked64", argLength: 4, typ: "Mem", aux: "Typ"}, // Store arg2 to arg0, arg1=mask of 64-bits, arg3 = memory
cmd/compile: handle partially overlapping assignments Normally, when moving Go values of type T from one location to another, we don't need to worry about partial overlaps. The two Ts must either be in disjoint (nonoverlapping) memory or in exactly the same location. There are 2 cases where this isn't true: 1) Using unsafe you can arrange partial overlaps. 2) Since Go 1.17, you can use a cast from a slice to a ptr-to-array. https://go.dev/ref/spec#Conversions_from_slice_to_array_pointer This feature can be used to construct partial overlaps of array types. var a [3]int p := (*[2]int)(a[:]) q := (*[2]int)(a[1:]) *p = *q We don't care about solving 1. Or at least, we haven't historically and no one has complained. For 2, we need to ensure that if there might be partial overlap, then we can't use OpMove; we must use memmove instead. (memmove handles partial overlap by copying in the correct direction. OpMove does not.) Note that we have to be careful here not to introduce a call when we're marshaling arguments to a call or unmarshaling results from a call. Fixes #54467 Change-Id: I1ca6aba8041576849c1d85f1fa33ae61b80a373d Reviewed-on: https://go-review.googlesource.com/c/go/+/425076 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Keith Randall <khr@golang.org>
2022-08-22 10:26:50 -07:00
// Normally we require that the source and destination of Move do not overlap.
// There is an exception when we know all the loads will happen before all
// the stores. In that case, overlap is ok. See
cmd/compile/internal/ssa: inline memmove with known size Replace calls to memmove with known (constant) size, with OpMove. Do it only if it is safe from aliasing point of view. Helps with code like this: append(buf,"const str"...) In strconv this provides nice benefit: Quote-6 731ns ± 2% 647ns ± 3% -11.41% (p=0.000 n=10+10) QuoteRune-6 117ns ± 5% 111ns ± 1% -4.54% (p=0.000 n=10+10) AppendQuote-6 475ns ± 0% 396ns ± 0% -16.59% (p=0.000 n=9+10) AppendQuoteRune-6 32.0ns ± 0% 27.4ns ± 0% -14.41% (p=0.000 n=8+9) Change-Id: I7704f5c51b46aed2d8f033de74c75140fc35036c Reviewed-on: https://go-review.googlesource.com/54394 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-09 14:00:38 -05:00
// memmove inlining in generic.rules. When inlineablememmovesize (in ../rewrite.go)
// returns true, we must do all loads before all stores, when lowering Move.
cmd/compile: document Move's type Fixes #37381 Change-Id: I8abf07d6342c10fc8d52e11c6a70fb0ec09220d2 Reviewed-on: https://go-review.googlesource.com/c/go/+/220683 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-24 16:16:27 -08:00
// The type of Move is used for the write barrier pass to insert write barriers
// and for alignment on some architectures.
// For pointerless types, it is possible for the type to be inaccurate.
// For type alignment and pointer information, use the type in Aux;
// for type size, use the size in AuxInt.
// The "inline runtime.memmove" rewrite rule generates Moves with inaccurate types,
// such as type byte instead of the more accurate type [8]byte.
cmd/compile: use type information in Aux for Store size Remove size AuxInt in Store, and alignment in Move/Zero. We still pass size AuxInt to Move/Zero, as it is used for partial Move/Zero lowering (e.g. cmd/compile/internal/ssa/gen/386.rules:288). SizeAndAlign is gone. Passes "toolstash -cmp" on std. Change-Id: I1ca34652b65dd30de886940e789fcf41d521475d Reviewed-on: https://go-review.googlesource.com/38150 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-03-13 21:51:08 -04:00
{name: "Move", argLength: 3, typ: "Mem", aux: "TypSize"}, // arg0=destptr, arg1=srcptr, arg2=mem, auxint=size, aux=type. Returns memory.
{name: "Zero", argLength: 2, typ: "Mem", aux: "TypSize"}, // arg0=destptr, arg1=mem, auxint=size, aux=type. Returns memory.
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
cmd/compile: add a writebarrier phase in SSA When the compiler insert write barriers, the frontend makes conservative decisions at an early stage. This may have false positives which result in write barriers for stack writes. A new phase, writebarrier, is added to the SSA backend, to delay the decision and eliminate false positives. The frontend still makes conservative decisions. When building SSA, instead of emitting runtime calls directly, it emits WB ops (StoreWB, MoveWB, etc.), which will be expanded to branches and runtime calls in writebarrier phase. Writes to static locations on stack are detected and write barriers are removed. All write barriers of stack writes found by the script from issue #17330 are eliminated (except two false positives). Fixes #17330. Change-Id: I9bd66333da9d0ceb64dcaa3c6f33502798d1a0f8 Reviewed-on: https://go-review.googlesource.com/31131 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2016-10-13 06:57:00 -04:00
// Memory operations with write barriers.
// Expand to runtime calls. Write barrier will be removed if write on stack.
cmd/compile: use type information in Aux for Store size Remove size AuxInt in Store, and alignment in Move/Zero. We still pass size AuxInt to Move/Zero, as it is used for partial Move/Zero lowering (e.g. cmd/compile/internal/ssa/gen/386.rules:288). SizeAndAlign is gone. Passes "toolstash -cmp" on std. Change-Id: I1ca34652b65dd30de886940e789fcf41d521475d Reviewed-on: https://go-review.googlesource.com/38150 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-03-13 21:51:08 -04:00
{name: "StoreWB", argLength: 3, typ: "Mem", aux: "Typ"}, // Store arg1 to arg0. arg2=memory, aux=type. Returns memory.
{name: "MoveWB", argLength: 3, typ: "Mem", aux: "TypSize"}, // arg0=destptr, arg1=srcptr, arg2=mem, auxint=size, aux=type. Returns memory.
{name: "ZeroWB", argLength: 2, typ: "Mem", aux: "TypSize"}, // arg0=destptr, arg1=mem, auxint=size, aux=type. Returns memory.
cmd/compile: detect write barrier completion differently Instead of keeping track of in which blocks write barriers complete, introduce a new op that marks the exact memory state where the write barrier completes. For future use. This allows us to move some of the write barrier code to between the start of the merging block and the WBend marker. Change-Id: If3809b260292667d91bf0ee18d7b4d0eb1e929f0 Reviewed-on: https://go-review.googlesource.com/c/go/+/447777 Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Keith Randall <khr@golang.org>
2022-10-09 19:06:23 -07:00
{name: "WBend", argLength: 1, typ: "Mem"}, // Write barrier code is done, interrupting is now allowed.
cmd/compile: add a writebarrier phase in SSA When the compiler insert write barriers, the frontend makes conservative decisions at an early stage. This may have false positives which result in write barriers for stack writes. A new phase, writebarrier, is added to the SSA backend, to delay the decision and eliminate false positives. The frontend still makes conservative decisions. When building SSA, instead of emitting runtime calls directly, it emits WB ops (StoreWB, MoveWB, etc.), which will be expanded to branches and runtime calls in writebarrier phase. Writes to static locations on stack are detected and write barriers are removed. All write barriers of stack writes found by the script from issue #17330 are eliminated (except two false positives). Fixes #17330. Change-Id: I9bd66333da9d0ceb64dcaa3c6f33502798d1a0f8 Reviewed-on: https://go-review.googlesource.com/31131 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org>
2016-10-13 06:57:00 -04:00
cmd/compile: batch write barrier calls Have the write barrier call return a pointer to a buffer into which the generated code records pointers that need write barrier treatment. Change-Id: I7871764298e0aa1513de417010c8d46b296b199e Reviewed-on: https://go-review.googlesource.com/c/go/+/447781 Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Bypass: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-01 16:46:43 -07:00
// WB invokes runtime.gcWriteBarrier. This is not a normal
cmd/compile: compiler support for buffered write barrier This CL implements the compiler support for calling the buffered write barrier added by the previous CL. Since the buffered write barrier is only implemented on amd64 right now, this still supports the old, eager write barrier as well. There's little overhead to supporting both and this way a few tests in test/fixedbugs that expect to have liveness maps at write barrier calls can easily opt-in to the old, eager barrier. This significantly improves the performance of the write barrier: name old time/op new time/op delta WriteBarrier-12 73.5ns ±20% 19.2ns ±27% -73.90% (p=0.000 n=19+18) It also reduces the size of binaries because the write barrier call is more compact: name old object-bytes new object-bytes delta Template 398k ± 0% 393k ± 0% -1.14% (p=0.008 n=5+5) Unicode 208k ± 0% 206k ± 0% -1.00% (p=0.008 n=5+5) GoTypes 1.18M ± 0% 1.15M ± 0% -2.00% (p=0.008 n=5+5) Compiler 4.05M ± 0% 3.88M ± 0% -4.26% (p=0.008 n=5+5) SSA 8.25M ± 0% 8.11M ± 0% -1.59% (p=0.008 n=5+5) Flate 228k ± 0% 224k ± 0% -1.83% (p=0.008 n=5+5) GoParser 295k ± 0% 284k ± 0% -3.62% (p=0.008 n=5+5) Reflect 1.00M ± 0% 0.99M ± 0% -0.70% (p=0.008 n=5+5) Tar 339k ± 0% 333k ± 0% -1.67% (p=0.008 n=5+5) XML 404k ± 0% 395k ± 0% -2.10% (p=0.008 n=5+5) [Geo mean] 704k 690k -2.00% name old exe-bytes new exe-bytes delta HelloSize 1.05M ± 0% 1.04M ± 0% -1.55% (p=0.008 n=5+5) https://perf.golang.org/search?q=upload:20171027.1 (Amusingly, this also reduces compiler allocations by 0.75%, which, combined with the better write barrier, speeds up the compiler overall by 2.10%. See the perf link.) It slightly improves the performance of most of the go1 benchmarks and improves the performance of the x/benchmarks: name old time/op new time/op delta BinaryTree17-12 2.40s ± 1% 2.47s ± 1% +2.69% (p=0.000 n=19+19) Fannkuch11-12 2.95s ± 0% 2.95s ± 0% +0.21% (p=0.000 n=20+19) FmtFprintfEmpty-12 41.8ns ± 4% 41.4ns ± 2% -1.03% (p=0.014 n=20+20) FmtFprintfString-12 68.7ns ± 2% 67.5ns ± 1% -1.75% (p=0.000 n=20+17) FmtFprintfInt-12 79.0ns ± 3% 77.1ns ± 1% -2.40% (p=0.000 n=19+17) FmtFprintfIntInt-12 127ns ± 1% 123ns ± 3% -3.42% (p=0.000 n=20+20) FmtFprintfPrefixedInt-12 152ns ± 1% 150ns ± 1% -1.02% (p=0.000 n=18+17) FmtFprintfFloat-12 211ns ± 1% 209ns ± 0% -0.99% (p=0.000 n=20+16) FmtManyArgs-12 500ns ± 0% 496ns ± 0% -0.73% (p=0.000 n=17+20) GobDecode-12 6.44ms ± 1% 6.53ms ± 0% +1.28% (p=0.000 n=20+19) GobEncode-12 5.46ms ± 0% 5.46ms ± 1% ~ (p=0.550 n=19+20) Gzip-12 220ms ± 1% 216ms ± 0% -1.75% (p=0.000 n=19+19) Gunzip-12 38.8ms ± 0% 38.6ms ± 0% -0.30% (p=0.000 n=18+19) HTTPClientServer-12 79.0µs ± 1% 78.2µs ± 1% -1.01% (p=0.000 n=20+20) JSONEncode-12 11.9ms ± 0% 11.9ms ± 0% -0.29% (p=0.000 n=20+19) JSONDecode-12 52.6ms ± 0% 52.2ms ± 0% -0.68% (p=0.000 n=19+20) Mandelbrot200-12 3.69ms ± 0% 3.68ms ± 0% -0.36% (p=0.000 n=20+20) GoParse-12 3.13ms ± 1% 3.18ms ± 1% +1.67% (p=0.000 n=19+20) RegexpMatchEasy0_32-12 73.2ns ± 1% 72.3ns ± 1% -1.19% (p=0.000 n=19+18) RegexpMatchEasy0_1K-12 241ns ± 0% 239ns ± 0% -0.83% (p=0.000 n=17+16) RegexpMatchEasy1_32-12 68.6ns ± 1% 69.0ns ± 1% +0.47% (p=0.015 n=18+16) RegexpMatchEasy1_1K-12 364ns ± 0% 361ns ± 0% -0.67% (p=0.000 n=16+17) RegexpMatchMedium_32-12 104ns ± 1% 103ns ± 1% -0.79% (p=0.001 n=20+15) RegexpMatchMedium_1K-12 33.8µs ± 3% 34.0µs ± 2% ~ (p=0.267 n=20+19) RegexpMatchHard_32-12 1.64µs ± 1% 1.62µs ± 2% -1.25% (p=0.000 n=19+18) RegexpMatchHard_1K-12 49.2µs ± 0% 48.7µs ± 1% -0.93% (p=0.000 n=19+18) Revcomp-12 391ms ± 5% 396ms ± 7% ~ (p=0.154 n=19+19) Template-12 63.1ms ± 0% 59.5ms ± 0% -5.76% (p=0.000 n=18+19) TimeParse-12 307ns ± 0% 306ns ± 0% -0.39% (p=0.000 n=19+17) TimeFormat-12 325ns ± 0% 323ns ± 0% -0.50% (p=0.000 n=19+19) [Geo mean] 47.3µs 46.9µs -0.67% https://perf.golang.org/search?q=upload:20171026.1 name old time/op new time/op delta Garbage/benchmem-MB=64-12 2.25ms ± 1% 2.20ms ± 1% -2.31% (p=0.000 n=18+18) HTTP-12 12.6µs ± 0% 12.6µs ± 0% -0.72% (p=0.000 n=18+17) JSON-12 11.0ms ± 0% 11.0ms ± 1% -0.68% (p=0.000 n=17+19) https://perf.golang.org/search?q=upload:20171026.2 Updates #14951. Updates #22460. Change-Id: Id4c0932890a1d41020071bec73b8522b1367d3e7 Reviewed-on: https://go-review.googlesource.com/73712 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-10-26 12:33:04 -04:00
// call: it takes arguments in registers, doesn't clobber
// general-purpose registers (the exact clobber set is
// arch-dependent), and is not a safe-point.
cmd/compile: batch write barrier calls Have the write barrier call return a pointer to a buffer into which the generated code records pointers that need write barrier treatment. Change-Id: I7871764298e0aa1513de417010c8d46b296b199e Reviewed-on: https://go-review.googlesource.com/c/go/+/447781 Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Bypass: Keith Randall <khr@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-01 16:46:43 -07:00
{name: "WB", argLength: 1, typ: "(BytePtr,Mem)", aux: "Int64"}, // arg0=mem, auxint=# of buffer entries needed. Returns buffer pointer and memory.
cmd/compile: compiler support for buffered write barrier This CL implements the compiler support for calling the buffered write barrier added by the previous CL. Since the buffered write barrier is only implemented on amd64 right now, this still supports the old, eager write barrier as well. There's little overhead to supporting both and this way a few tests in test/fixedbugs that expect to have liveness maps at write barrier calls can easily opt-in to the old, eager barrier. This significantly improves the performance of the write barrier: name old time/op new time/op delta WriteBarrier-12 73.5ns ±20% 19.2ns ±27% -73.90% (p=0.000 n=19+18) It also reduces the size of binaries because the write barrier call is more compact: name old object-bytes new object-bytes delta Template 398k ± 0% 393k ± 0% -1.14% (p=0.008 n=5+5) Unicode 208k ± 0% 206k ± 0% -1.00% (p=0.008 n=5+5) GoTypes 1.18M ± 0% 1.15M ± 0% -2.00% (p=0.008 n=5+5) Compiler 4.05M ± 0% 3.88M ± 0% -4.26% (p=0.008 n=5+5) SSA 8.25M ± 0% 8.11M ± 0% -1.59% (p=0.008 n=5+5) Flate 228k ± 0% 224k ± 0% -1.83% (p=0.008 n=5+5) GoParser 295k ± 0% 284k ± 0% -3.62% (p=0.008 n=5+5) Reflect 1.00M ± 0% 0.99M ± 0% -0.70% (p=0.008 n=5+5) Tar 339k ± 0% 333k ± 0% -1.67% (p=0.008 n=5+5) XML 404k ± 0% 395k ± 0% -2.10% (p=0.008 n=5+5) [Geo mean] 704k 690k -2.00% name old exe-bytes new exe-bytes delta HelloSize 1.05M ± 0% 1.04M ± 0% -1.55% (p=0.008 n=5+5) https://perf.golang.org/search?q=upload:20171027.1 (Amusingly, this also reduces compiler allocations by 0.75%, which, combined with the better write barrier, speeds up the compiler overall by 2.10%. See the perf link.) It slightly improves the performance of most of the go1 benchmarks and improves the performance of the x/benchmarks: name old time/op new time/op delta BinaryTree17-12 2.40s ± 1% 2.47s ± 1% +2.69% (p=0.000 n=19+19) Fannkuch11-12 2.95s ± 0% 2.95s ± 0% +0.21% (p=0.000 n=20+19) FmtFprintfEmpty-12 41.8ns ± 4% 41.4ns ± 2% -1.03% (p=0.014 n=20+20) FmtFprintfString-12 68.7ns ± 2% 67.5ns ± 1% -1.75% (p=0.000 n=20+17) FmtFprintfInt-12 79.0ns ± 3% 77.1ns ± 1% -2.40% (p=0.000 n=19+17) FmtFprintfIntInt-12 127ns ± 1% 123ns ± 3% -3.42% (p=0.000 n=20+20) FmtFprintfPrefixedInt-12 152ns ± 1% 150ns ± 1% -1.02% (p=0.000 n=18+17) FmtFprintfFloat-12 211ns ± 1% 209ns ± 0% -0.99% (p=0.000 n=20+16) FmtManyArgs-12 500ns ± 0% 496ns ± 0% -0.73% (p=0.000 n=17+20) GobDecode-12 6.44ms ± 1% 6.53ms ± 0% +1.28% (p=0.000 n=20+19) GobEncode-12 5.46ms ± 0% 5.46ms ± 1% ~ (p=0.550 n=19+20) Gzip-12 220ms ± 1% 216ms ± 0% -1.75% (p=0.000 n=19+19) Gunzip-12 38.8ms ± 0% 38.6ms ± 0% -0.30% (p=0.000 n=18+19) HTTPClientServer-12 79.0µs ± 1% 78.2µs ± 1% -1.01% (p=0.000 n=20+20) JSONEncode-12 11.9ms ± 0% 11.9ms ± 0% -0.29% (p=0.000 n=20+19) JSONDecode-12 52.6ms ± 0% 52.2ms ± 0% -0.68% (p=0.000 n=19+20) Mandelbrot200-12 3.69ms ± 0% 3.68ms ± 0% -0.36% (p=0.000 n=20+20) GoParse-12 3.13ms ± 1% 3.18ms ± 1% +1.67% (p=0.000 n=19+20) RegexpMatchEasy0_32-12 73.2ns ± 1% 72.3ns ± 1% -1.19% (p=0.000 n=19+18) RegexpMatchEasy0_1K-12 241ns ± 0% 239ns ± 0% -0.83% (p=0.000 n=17+16) RegexpMatchEasy1_32-12 68.6ns ± 1% 69.0ns ± 1% +0.47% (p=0.015 n=18+16) RegexpMatchEasy1_1K-12 364ns ± 0% 361ns ± 0% -0.67% (p=0.000 n=16+17) RegexpMatchMedium_32-12 104ns ± 1% 103ns ± 1% -0.79% (p=0.001 n=20+15) RegexpMatchMedium_1K-12 33.8µs ± 3% 34.0µs ± 2% ~ (p=0.267 n=20+19) RegexpMatchHard_32-12 1.64µs ± 1% 1.62µs ± 2% -1.25% (p=0.000 n=19+18) RegexpMatchHard_1K-12 49.2µs ± 0% 48.7µs ± 1% -0.93% (p=0.000 n=19+18) Revcomp-12 391ms ± 5% 396ms ± 7% ~ (p=0.154 n=19+19) Template-12 63.1ms ± 0% 59.5ms ± 0% -5.76% (p=0.000 n=18+19) TimeParse-12 307ns ± 0% 306ns ± 0% -0.39% (p=0.000 n=19+17) TimeFormat-12 325ns ± 0% 323ns ± 0% -0.50% (p=0.000 n=19+19) [Geo mean] 47.3µs 46.9µs -0.67% https://perf.golang.org/search?q=upload:20171026.1 name old time/op new time/op delta Garbage/benchmem-MB=64-12 2.25ms ± 1% 2.20ms ± 1% -2.31% (p=0.000 n=18+18) HTTP-12 12.6µs ± 0% 12.6µs ± 0% -0.72% (p=0.000 n=18+17) JSON-12 11.0ms ± 0% 11.0ms ± 1% -0.68% (p=0.000 n=17+19) https://perf.golang.org/search?q=upload:20171026.2 Updates #14951. Updates #22460. Change-Id: Id4c0932890a1d41020071bec73b8522b1367d3e7 Reviewed-on: https://go-review.googlesource.com/73712 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-10-26 12:33:04 -04:00
cmd/compile: add intrinsic HasCPUFeature for checking cpu features Before using some CPU instructions, we must check for their presence. We use global variables in the runtime package to record features. Prior to this CL, we issued a regular memory load for these features. The downside to this is that, because it is a regular memory load, it cannot be hoisted out of loops or otherwise reordered with other loads. This CL introduces a new intrinsic just for checking cpu features. It still ends up resulting in a memory load, but that memory load can now be floated to the entry block and rematerialized as needed. One downside is that the regular load could be combined with the comparison into a CMPBconstload+NE. This new intrinsic cannot; it generates MOVB+TESTB+NE. (It is possible that MOVBQZX+TESTQ+NE would be better.) This CL does only amd64. It is easy to extend to other architectures. For the benchmark in #36196, on my machine, this offers a mild speedup. name old time/op new time/op delta FMA-8 1.39ns ± 6% 1.29ns ± 9% -7.19% (p=0.000 n=97+96) NonFMA-8 2.03ns ±11% 2.04ns ±12% ~ (p=0.618 n=99+98) Updates #15808 Updates #36196 Change-Id: I75e2fcfcf5a6df1bdb80657a7143bed69fca6deb Reviewed-on: https://go-review.googlesource.com/c/go/+/212360 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Giovanni Bajo <rasky@develer.com>
2019-12-19 10:58:28 -08:00
{name: "HasCPUFeature", argLength: 0, typ: "bool", aux: "Sym", symEffect: "None"}, // aux=place that this feature flag can be loaded from
cmd/compile,runtime: provide index information on bounds check failure A few examples (for accessing a slice of length 3): s[-1] runtime error: index out of range [-1] s[3] runtime error: index out of range [3] with length 3 s[-1:0] runtime error: slice bounds out of range [-1:] s[3:0] runtime error: slice bounds out of range [3:0] s[3:-1] runtime error: slice bounds out of range [:-1] s[3:4] runtime error: slice bounds out of range [:4] with capacity 3 s[0:3:4] runtime error: slice bounds out of range [::4] with capacity 3 Note that in cases where there are multiple things wrong with the indexes (e.g. s[3:-1]), we report one of those errors kind of arbitrarily, currently the rightmost one. An exhaustive set of examples is in issue30116[u].out in the CL. The message text has the same prefix as the old message text. That leads to slightly awkward phrasing but hopefully minimizes the chance that code depending on the error text will break. Increases the size of the go binary by 0.5% (amd64). The panic functions take arguments in registers in order to keep the size of the compiled code as small as possible. Fixes #30116 Change-Id: Idb99a827b7888822ca34c240eca87b7e44a04fdd Reviewed-on: https://go-review.googlesource.com/c/go/+/161477 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-02-06 14:12:36 -08:00
// PanicBounds and PanicExtend generate a runtime panic.
// Their arguments provide index values to use in panic messages.
// Both PanicBounds and PanicExtend have an AuxInt value from the BoundsKind type (in ../op.go).
// PanicBounds' index is int sized.
// PanicExtend's index is int64 sized. (PanicExtend is only used on 32-bit archs.)
cmd/compile: mark PanicBounds/Extend as calls PanicBounds and PanicExtend are lowered to runtime calls (with a non-Go ABI), but are not currently marked as calls. Since liveness analysis only emits stack maps at calls in the runtime, this means these panic call sites in the runtime won't get a stack map. These almost immediately turn into throws in the runtime, but there's still a chance they'll try to grow the stack first, which would lead to a different panic. To fix this, mark these operations as calls. Outside the runtime, we currently emit stack maps for everything that isn't an unsafe-point, so these panic calls get stack maps by default. However, we're about to move to emitting stack maps only at call sites, at which point this will start to matter outside the runtime as well. I confirmed that this has no effect on anything but PCDATA/FUNCDATA in runtime and net/http. For #36365. Change-Id: Ic5bb463fd152cc320c815dc04cf62005261ae169 Reviewed-on: https://go-review.googlesource.com/c/go/+/230539 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-27 15:58:16 -04:00
{name: "PanicBounds", argLength: 3, aux: "Int64", typ: "Mem", call: true}, // arg0=idx, arg1=len, arg2=mem, returns memory.
{name: "PanicExtend", argLength: 4, aux: "Int64", typ: "Mem", call: true}, // arg0=idxHi, arg1=idxLo, arg2=len, arg3=mem, returns memory.
cmd/compile,runtime: provide index information on bounds check failure A few examples (for accessing a slice of length 3): s[-1] runtime error: index out of range [-1] s[3] runtime error: index out of range [3] with length 3 s[-1:0] runtime error: slice bounds out of range [-1:] s[3:0] runtime error: slice bounds out of range [3:0] s[3:-1] runtime error: slice bounds out of range [:-1] s[3:4] runtime error: slice bounds out of range [:4] with capacity 3 s[0:3:4] runtime error: slice bounds out of range [::4] with capacity 3 Note that in cases where there are multiple things wrong with the indexes (e.g. s[3:-1]), we report one of those errors kind of arbitrarily, currently the rightmost one. An exhaustive set of examples is in issue30116[u].out in the CL. The message text has the same prefix as the old message text. That leads to slightly awkward phrasing but hopefully minimizes the chance that code depending on the error text will break. Increases the size of the go binary by 0.5% (amd64). The panic functions take arguments in registers in order to keep the size of the compiled code as small as possible. Fixes #30116 Change-Id: Idb99a827b7888822ca34c240eca87b7e44a04fdd Reviewed-on: https://go-review.googlesource.com/c/go/+/161477 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-02-06 14:12:36 -08:00
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// Function calls. Arguments to the call have already been written to the stack.
// Return values appear on the stack. The method receiver, if any, is treated
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// as a phantom first argument.
cmd/compile: convert devirtualization rule to typed aux Passes toolstash-check. Change-Id: Iebcdc35f1a2112d5384c70eb3fdbd92ebb3d248e Reviewed-on: https://go-review.googlesource.com/c/go/+/229689 Reviewed-by: Keith Randall <khr@golang.org>
2020-04-23 22:46:11 -07:00
// TODO(josharian): ClosureCall and InterCall should have Int32 aux
// to match StaticCall's 32 bit arg size limit.
cmd/compile: add new ops for experiment with late call expansion Added Dereference, StaticLECall, SelectN, SelectNAddr Change-Id: I5426ae488e83956539aa07f7415b8acc26c933e6 Reviewed-on: https://go-review.googlesource.com/c/go/+/239082 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-06-15 14:20:36 -04:00
// TODO(drchase,josharian): could the arg size limit be bundled into the rules for CallOff?
cmd/compile: test register ABI for method, interface, closure calls This is enabled with a ridiculous magic name for method, or for last input type passed, that needs to be changed to something inutterable before actual release. Ridiculous method name: MagicMethodNameForTestingRegisterABI Ridiculous last (input) type name: MagicLastTypeNameForTestingRegisterABI RLTN is tested with strings.Contains, so you can have MagicLastTypeNameForTestingRegisterABI1 and MagicLastTypeNameForTestingRegisterABI2 if that is helpful Includes test test/abi/fibish2.go Updates #44816. Change-Id: I592a6edc71ca9bebdd1d00e24edee1ceebb3e43f Reviewed-on: https://go-review.googlesource.com/c/go/+/299410 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-05 14:24:41 -05:00
// Before lowering, LECalls receive their fixed inputs (first), memory (last),
// and a variable number of input values in the middle.
// They produce a variable number of result values.
// These values are not necessarily "SSA-able"; they can be too large,
// but in that case inputs are loaded immediately before with OpDereference,
// and outputs are stored immediately with OpStore.
//
// After call expansion, Calls have the same fixed-middle-memory arrangement of inputs,
// with the difference that the "middle" is only the register-resident inputs,
// and the non-register inputs are instead stored at ABI-defined offsets from SP
// (and the stores thread through the memory that is ultimately an input to the call).
// Outputs follow a similar pattern; register-resident outputs are the leading elements
// of a Result-typed output, with memory last, and any memory-resident outputs have been
// stored to ABI-defined locations. Each non-memory input or output fits in a register.
//
// Subsequent architecture-specific lowering only changes the opcode.
{name: "ClosureCall", argLength: -1, aux: "CallOff", call: true}, // arg0=code pointer, arg1=context ptr, arg2..argN-1 are register inputs, argN=memory. auxint=arg size. Returns Result of register results, plus memory.
{name: "StaticCall", argLength: -1, aux: "CallOff", call: true}, // call function aux.(*obj.LSym), arg0..argN-1 are register inputs, argN=memory. auxint=arg size. Returns Result of register results, plus memory.
{name: "InterCall", argLength: -1, aux: "CallOff", call: true}, // interface call. arg0=code pointer, arg1..argN-1 are register inputs, argN=memory, auxint=arg size. Returns Result of register results, plus memory.
cmd/compile: restore tail call for method wrappers For certain type of method wrappers we used to generate a tail call. That was disabled in CL 307234 when register ABI is used, because with the current IR it was difficult to generate a tail call with the arguments in the right places. The problem was that the IR does not contain a CALL-like node with arguments; instead, it contains an OAS node that adjusts the receiver, than an OTAILCALL node that just contains the target, but no argument (with the assumption that the OAS node will put the adjusted receiver in the right place). With register ABI, putting arguments in registers are done in SSA. The assignment (OAS) doesn't put the receiver in register. This CL changes the IR of a tail call to take an actual OCALL node. Specifically, a tail call is represented as OTAILCALL (OCALL target args...) This way, the call target and args are connected through the OCALL node. So the call can be analyzed in SSA and the args can be passed in the right places. (Alternatively, we could have OTAILCALL node directly take the target and the args, without the OCALL node. Using an OCALL node is convenient as there are existing code that processes OCALL nodes which do not need to be changed. Also, a tail call is similar to ORETURN (OCALL target args...), except it doesn't preserve the frame. I did the former but I'm open to change.) The SSA representation is similar. Previously, the IR lowers to a Store the receiver then a BlockRetJmp which jumps to the target (without putting the arg in register). Now we use a TailCall op, which takes the target and the args. The call expansion pass and the register allocator handles TailCall pretty much like a StaticCall, and it will do the right ABI analysis and put the args in the right places. (Args other than the receiver are already in the right places. For register args it generates no code for them. For stack args currently it generates a self copy. I'll work on optimize that out.) BlockRetJmp is still used, signaling it is a tail call. The actual call is made in the TailCall op so BlockRetJmp generates no code (we could use BlockExit if we like). This slightly reduces binary size: old new cmd/go 14003088 13953936 cmd/link 6275552 6271456 Change-Id: I2d16d8d419fe1f17554916d317427383e17e27f0 Reviewed-on: https://go-review.googlesource.com/c/go/+/350145 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: David Chase <drchase@google.com>
2021-09-10 22:05:55 -04:00
{name: "TailCall", argLength: -1, aux: "CallOff", call: true}, // tail call function aux.(*obj.LSym), arg0..argN-1 are register inputs, argN=memory. auxint=arg size. Returns Result of register results, plus memory.
cmd/compile: test register ABI for method, interface, closure calls This is enabled with a ridiculous magic name for method, or for last input type passed, that needs to be changed to something inutterable before actual release. Ridiculous method name: MagicMethodNameForTestingRegisterABI Ridiculous last (input) type name: MagicLastTypeNameForTestingRegisterABI RLTN is tested with strings.Contains, so you can have MagicLastTypeNameForTestingRegisterABI1 and MagicLastTypeNameForTestingRegisterABI2 if that is helpful Includes test test/abi/fibish2.go Updates #44816. Change-Id: I592a6edc71ca9bebdd1d00e24edee1ceebb3e43f Reviewed-on: https://go-review.googlesource.com/c/go/+/299410 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-05 14:24:41 -05:00
cmd/compile: enable late expansion for interface calls Includes a few tweaks to Value.copyOf(a) (make it a no-op for a self-copy) and new pattern hack "___" (3 underscores) is like ellipsis, except the replacement doesn't need to have matching ellipsis/underscores. Moved the arg-length check in generated pattern-matching code BEFORE the args are probed, because not all instances of variable length OpFoo will have all the args mentioned in some rule for OpFoo, and when that happens, the compiler panics without the early check. Change-Id: I66de40672b3794a6427890ff96c805a488d783f4 Reviewed-on: https://go-review.googlesource.com/c/go/+/247537 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-08-07 22:46:43 -04:00
{name: "ClosureLECall", argLength: -1, aux: "CallOff", call: true}, // late-expanded closure call. arg0=code pointer, arg1=context ptr, arg2..argN-1 are inputs, argN is mem. auxint = arg size. Result is tuple of result(s), plus mem.
{name: "StaticLECall", argLength: -1, aux: "CallOff", call: true}, // late-expanded static call function aux.(*ssa.AuxCall.Fn). arg0..argN-1 are inputs, argN is mem. auxint = arg size. Result is tuple of result(s), plus mem.
{name: "InterLECall", argLength: -1, aux: "CallOff", call: true}, // late-expanded interface call. arg0=code pointer, arg1..argN-1 are inputs, argN is mem. auxint = arg size. Result is tuple of result(s), plus mem.
cmd/compile: restore tail call for method wrappers For certain type of method wrappers we used to generate a tail call. That was disabled in CL 307234 when register ABI is used, because with the current IR it was difficult to generate a tail call with the arguments in the right places. The problem was that the IR does not contain a CALL-like node with arguments; instead, it contains an OAS node that adjusts the receiver, than an OTAILCALL node that just contains the target, but no argument (with the assumption that the OAS node will put the adjusted receiver in the right place). With register ABI, putting arguments in registers are done in SSA. The assignment (OAS) doesn't put the receiver in register. This CL changes the IR of a tail call to take an actual OCALL node. Specifically, a tail call is represented as OTAILCALL (OCALL target args...) This way, the call target and args are connected through the OCALL node. So the call can be analyzed in SSA and the args can be passed in the right places. (Alternatively, we could have OTAILCALL node directly take the target and the args, without the OCALL node. Using an OCALL node is convenient as there are existing code that processes OCALL nodes which do not need to be changed. Also, a tail call is similar to ORETURN (OCALL target args...), except it doesn't preserve the frame. I did the former but I'm open to change.) The SSA representation is similar. Previously, the IR lowers to a Store the receiver then a BlockRetJmp which jumps to the target (without putting the arg in register). Now we use a TailCall op, which takes the target and the args. The call expansion pass and the register allocator handles TailCall pretty much like a StaticCall, and it will do the right ABI analysis and put the args in the right places. (Args other than the receiver are already in the right places. For register args it generates no code for them. For stack args currently it generates a self copy. I'll work on optimize that out.) BlockRetJmp is still used, signaling it is a tail call. The actual call is made in the TailCall op so BlockRetJmp generates no code (we could use BlockExit if we like). This slightly reduces binary size: old new cmd/go 14003088 13953936 cmd/link 6275552 6271456 Change-Id: I2d16d8d419fe1f17554916d317427383e17e27f0 Reviewed-on: https://go-review.googlesource.com/c/go/+/350145 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: David Chase <drchase@google.com>
2021-09-10 22:05:55 -04:00
{name: "TailLECall", argLength: -1, aux: "CallOff", call: true}, // late-expanded static tail call function aux.(*ssa.AuxCall.Fn). arg0..argN-1 are inputs, argN is mem. auxint = arg size. Result is tuple of result(s), plus mem.
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
[dev.ssa] cmd/compile: use Copy instead of ConvNop The existing backend simply elides OCONVNOP. There's no reason for us to do any differently. Rather than insert ConvNops and then rewrite them away, stop creating them in the first place. Change-Id: I4bcbe2229fcebd189ae18df24f2c612feb6e215e Reviewed-on: https://go-review.googlesource.com/12810 Reviewed-by: Keith Randall <khr@golang.org>
2015-07-28 14:31:25 -07:00
// Conversions: signed extensions, zero (unsigned) extensions, truncations
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "SignExt8to16", argLength: 1, typ: "Int16"},
[dev.ssa] cmd/compile: implement the following for SSA on ARM - generic Ops: Phi, CALL variants, NilCheck - generic Blocks: Plain, Check - 32-bit arithmetics - CMP and conditional branches - load/store - zero/sign-extensions (8 to 16, 8 to 32, 16 to 32) Progress on SSA backend for ARM. Still not complete. Now "errors" package compiles and tests passed. Updates #15365. Change-Id: If126fd17f8695cbf55d64085bb3f1a4a53205701 Reviewed-on: https://go-review.googlesource.com/22856 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-06 10:13:31 -07:00
{name: "SignExt8to32", argLength: 1, typ: "Int32"},
[dev.ssa] cmd/compile: provide default types for all extension ops Change-Id: I655327818297cc6792c81912f2cebdc321381561 Reviewed-on: https://go-review.googlesource.com/24465 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2016-06-25 16:07:56 -07:00
{name: "SignExt8to64", argLength: 1, typ: "Int64"},
[dev.ssa] cmd/compile: implement the following for SSA on ARM - generic Ops: Phi, CALL variants, NilCheck - generic Blocks: Plain, Check - 32-bit arithmetics - CMP and conditional branches - load/store - zero/sign-extensions (8 to 16, 8 to 32, 16 to 32) Progress on SSA backend for ARM. Still not complete. Now "errors" package compiles and tests passed. Updates #15365. Change-Id: If126fd17f8695cbf55d64085bb3f1a4a53205701 Reviewed-on: https://go-review.googlesource.com/22856 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-06 10:13:31 -07:00
{name: "SignExt16to32", argLength: 1, typ: "Int32"},
[dev.ssa] cmd/compile: provide default types for all extension ops Change-Id: I655327818297cc6792c81912f2cebdc321381561 Reviewed-on: https://go-review.googlesource.com/24465 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2016-06-25 16:07:56 -07:00
{name: "SignExt16to64", argLength: 1, typ: "Int64"},
{name: "SignExt32to64", argLength: 1, typ: "Int64"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "ZeroExt8to16", argLength: 1, typ: "UInt16"},
[dev.ssa] cmd/compile: implement the following for SSA on ARM - generic Ops: Phi, CALL variants, NilCheck - generic Blocks: Plain, Check - 32-bit arithmetics - CMP and conditional branches - load/store - zero/sign-extensions (8 to 16, 8 to 32, 16 to 32) Progress on SSA backend for ARM. Still not complete. Now "errors" package compiles and tests passed. Updates #15365. Change-Id: If126fd17f8695cbf55d64085bb3f1a4a53205701 Reviewed-on: https://go-review.googlesource.com/22856 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-06 10:13:31 -07:00
{name: "ZeroExt8to32", argLength: 1, typ: "UInt32"},
[dev.ssa] cmd/compile: provide default types for all extension ops Change-Id: I655327818297cc6792c81912f2cebdc321381561 Reviewed-on: https://go-review.googlesource.com/24465 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2016-06-25 16:07:56 -07:00
{name: "ZeroExt8to64", argLength: 1, typ: "UInt64"},
[dev.ssa] cmd/compile: implement the following for SSA on ARM - generic Ops: Phi, CALL variants, NilCheck - generic Blocks: Plain, Check - 32-bit arithmetics - CMP and conditional branches - load/store - zero/sign-extensions (8 to 16, 8 to 32, 16 to 32) Progress on SSA backend for ARM. Still not complete. Now "errors" package compiles and tests passed. Updates #15365. Change-Id: If126fd17f8695cbf55d64085bb3f1a4a53205701 Reviewed-on: https://go-review.googlesource.com/22856 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-06 10:13:31 -07:00
{name: "ZeroExt16to32", argLength: 1, typ: "UInt32"},
[dev.ssa] cmd/compile: provide default types for all extension ops Change-Id: I655327818297cc6792c81912f2cebdc321381561 Reviewed-on: https://go-review.googlesource.com/24465 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2016-06-25 16:07:56 -07:00
{name: "ZeroExt16to64", argLength: 1, typ: "UInt64"},
{name: "ZeroExt32to64", argLength: 1, typ: "UInt64"},
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "Trunc16to8", argLength: 1},
{name: "Trunc32to8", argLength: 1},
{name: "Trunc32to16", argLength: 1},
{name: "Trunc64to8", argLength: 1},
{name: "Trunc64to16", argLength: 1},
{name: "Trunc64to32", argLength: 1},
{name: "Cvt32to32F", argLength: 1},
{name: "Cvt32to64F", argLength: 1},
{name: "Cvt64to32F", argLength: 1},
{name: "Cvt64to64F", argLength: 1},
{name: "Cvt32Fto32", argLength: 1},
{name: "Cvt32Fto64", argLength: 1},
{name: "Cvt64Fto32", argLength: 1},
{name: "Cvt64Fto64", argLength: 1},
{name: "Cvt32Fto64F", argLength: 1},
{name: "Cvt64Fto32F", argLength: 1},
cmd/compile: use correct types in phiopt We try to preserve type correctness of generic ops. phiopt modified a bool to be an int without a conversion. Add a conversion. There are a few random fluctations in the generated code as a result, but nothing noteworthy or systematic. no binary size changes file before after Δ % math.s 35966 35961 -5 -0.014% debug/dwarf.s 108141 108147 +6 +0.006% crypto/dsa.s 6047 6044 -3 -0.050% image/png.s 42882 42885 +3 +0.007% go/parser.s 80281 80278 -3 -0.004% cmd/internal/obj.s 115116 115113 -3 -0.003% go/types.s 322130 322118 -12 -0.004% cmd/internal/obj/arm64.s 151679 151685 +6 +0.004% go/internal/gccgoimporter.s 56487 56493 +6 +0.011% cmd/test2json.s 1650 1647 -3 -0.182% cmd/link/internal/loadelf.s 35442 35443 +1 +0.003% cmd/go/internal/work.s 305039 305035 -4 -0.001% cmd/link/internal/ld.s 544835 544834 -1 -0.000% net/http.s 558777 558774 -3 -0.001% cmd/compile/internal/ssa.s 3926551 3926994 +443 +0.011% cmd/compile/internal/gc.s 1552320 1552321 +1 +0.000% total 18862241 18862670 +429 +0.002% Change-Id: I4289e773be6be534ea3f907d68f614441b8f9b46 Reviewed-on: https://go-review.googlesource.com/c/go/+/221607 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2020-02-28 17:04:16 -08:00
{name: "CvtBoolToUint8", argLength: 1},
[dev.ssa] cmd/compile: add [u]int and FP to FP conversions Change-Id: I8c17f706a3e0f1fa2d754bfb4ccd1f7a027cb3db Reviewed-on: https://go-review.googlesource.com/13744 Reviewed-by: Keith Randall <khr@golang.org>
2015-08-20 15:14:20 -04:00
cmd/compile: emit fused multiply-{add,subtract} instructions on s390x Explcitly block fused multiply-add pattern matching when a cast is used after the multiplication, for example: - (a * b) + c // can emit fused multiply-add - float64(a * b) + c // cannot emit fused multiply-add float{32,64} and complex{64,128} casts of matching types are now kept as OCONV operations rather than being replaced with OCONVNOP operations because they now imply a rounding operation (and therefore aren't a no-op anymore). Operations (for example, multiplication) on complex types may utilize fused multiply-add and -subtract instructions internally. There is no way to disable this behavior at the moment. Improves the performance of the floating point implementation of poly1305: name old speed new speed delta 64 246MB/s ± 0% 275MB/s ± 0% +11.48% (p=0.000 n=10+8) 1K 312MB/s ± 0% 357MB/s ± 0% +14.41% (p=0.000 n=10+10) 64Unaligned 246MB/s ± 0% 274MB/s ± 0% +11.43% (p=0.000 n=10+10) 1KUnaligned 312MB/s ± 0% 357MB/s ± 0% +14.39% (p=0.000 n=10+8) Updates #17895. Change-Id: Ia771d275bb9150d1a598f8cc773444663de5ce16 Reviewed-on: https://go-review.googlesource.com/36963 Run-TryBot: Michael Munday <munday@ca.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-02-12 22:12:12 -05:00
// Force rounding to precision of type.
{name: "Round32F", argLength: 1},
{name: "Round64F", argLength: 1},
[dev.ssa] cmd/compile: minor cleanup Cull dead code. Add TODOs. Change-Id: I81d24371de47f5a27d3a3a0ec0ef5baaf6814c06 Reviewed-on: https://go-review.googlesource.com/12659 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-07-24 11:55:52 -07:00
// Automatically inserted safety checks
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "IsNonNil", argLength: 1, typ: "Bool"}, // arg0 != nil
cmd/compile: define high bits of AuxInt Previously if we were only using the low bits of AuxInt, the high bits were ignored and could be junk. This CL changes that behavior to define the high bits to be the sign-extended version of the low bits for all cases. There are 2 main benefits: - Deterministic representation. This helps with CSE. (Const8 [0x1]) and (Const8 [0x101]) used to be the same "value" but CSE couldn't see them as such. - Testability. We can check that all ops leave AuxInt in a state consistent with the new rule. In the old scheme, it was hard to check whether a rule correctly used only the low-order bits. Side benefits: - ==0 and !=0 tests are easier. Drawbacks: - This differs from the runtime representation in registers, where it is important that we allow upper bits to be undefined (so we're not sign/zero-extending all the time). - Ops that treat AuxInt as unsigned (shifts, mostly) need to be a bit more careful. Change-Id: I9a685ff27e36dc03287c9ab1cecd6c0b4045c819 Reviewed-on: https://go-review.googlesource.com/21256 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-29 16:39:53 -07:00
{name: "IsInBounds", argLength: 2, typ: "Bool"}, // 0 <= arg0 < arg1. arg1 is guaranteed >= 0.
{name: "IsSliceInBounds", argLength: 2, typ: "Bool"}, // 0 <= arg0 <= arg1. arg1 is guaranteed >= 0.
cmd/compile: ensure pointer arithmetic happens after the nil check Have nil checks return a pointer that is known non-nil. Users of that pointer can use the result, ensuring that they are ordered after the nil check itself. The order dependence goes away after scheduling, when we've fixed an order. At that point we move uses back to the original pointer so it doesn't change regalloc any. This prevents pointer arithmetic on nil from being spilled to the stack and then observed by a stack scan. Fixes #63657 Change-Id: I1a5fa4f2e6d9000d672792b4f90dfc1b7b67f6ea Reviewed-on: https://go-review.googlesource.com/c/go/+/537775 Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com>
2023-10-25 13:35:13 -07:00
{name: "NilCheck", argLength: 2, nilCheck: true}, // arg0=ptr, arg1=mem. Panics if arg0 is nil. Returns the ptr unmodified.
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
[dev.ssa] cmd/compile: implement OGETG Change-Id: I7ecf62cf399c710b4a617803c43e83fce09b8a7d Reviewed-on: https://go-review.googlesource.com/13585 Reviewed-by: Keith Randall <khr@golang.org>
2015-08-12 11:22:16 -07:00
// Pseudo-ops
cmd/compile/internal/ssa: note zero-width Ops Add a bool to opInfo to indicate if an Op never results in any instructions. This is a conservative approximation: some operations, like Copy, may or may not generate code depending on their arguments. I built the list by reading each arch's ssaGenValue function. Hopefully I got them all. Change-Id: I130b251b65f18208294e129bb7ddc3f91d57d31d Reviewed-on: https://go-review.googlesource.com/97957 Reviewed-by: Keith Randall <khr@golang.org>
2018-02-28 16:30:07 -05:00
{name: "GetG", argLength: 1, zeroWidth: true}, // runtime.getg() (read g pointer). arg0=mem
{name: "GetClosurePtr"}, // get closure pointer from dedicated register
runtime: move getcallerpc to internal/runtime/sys Moving these intrinsics to a base package enables other internal/runtime packages to use them. For #54766. Change-Id: I0b3eded3bb45af53e3eb5bab93e3792e6a8beb46 Reviewed-on: https://go-review.googlesource.com/c/go/+/613260 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-09-16 14:07:43 -04:00
{name: "GetCallerPC"}, // for GetCallerPC intrinsic
runtime: move getcallersp to internal/runtime/sys Moving these intrinsics to a base package enables other internal/runtime packages to use them. For #54766. Change-Id: I45a530422207dd94b5ad4eee51216c9410a84040 Reviewed-on: https://go-review.googlesource.com/c/go/+/613261 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-09-16 15:58:36 -04:00
{name: "GetCallerSP", argLength: 1}, // for GetCallerSP intrinsic. arg0=mem.
[dev.ssa] cmd/compile: make failed nil checks panic Introduce pseudo-ops PanicMem and LoweredPanicMem. PanicMem could be rewritten directly into MOVL during lowering, but then we couldn't log nil checks. With this change, runnable nil check tests pass: GOSSAPKG=main go run run.go -- nil*.go Compiler output nil check tests fail: GOSSAPKG=p go run run.go -- nil*.go This is due to several factors: * SSA has improved elimination of unnecessary nil checks. * SSA is missing elimination of implicit nil checks. * SSA is missing extra logging about why nil checks were removed. I'm not sure how best to resolve these failures, particularly in a world in which the two backends will live side by side for some time. For now, punt on the problem. Change-Id: Ib2ca6824551671f92e0e1800b036f5ca0905e2a3 Reviewed-on: https://go-review.googlesource.com/13474 Reviewed-by: Keith Randall <khr@golang.org>
2015-08-11 09:47:45 -07:00
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// Indexing operations
cmd/compile: make [0]T and [1]T SSAable types We used to have to keep on-stack copies of these types. Now they can be registerized. [0]T is kind of trivial but might as well handle it. This change enables another change I'm working on to improve how x.(T) expressions are handled (#17405). This CL helps because now all types that are direct interface types are registerizeable (e.g. [1]*byte). No higher-degree arrays for now because non-constant indexes are hard. Update #17405 Change-Id: I2399940965d17b3969ae66f6fe447a8cefdd6edd Reviewed-on: https://go-review.googlesource.com/32416 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2016-10-30 21:10:03 -07:00
{name: "PtrIndex", argLength: 2}, // arg0=ptr, arg1=index. Computes ptr+sizeof(*v.type)*index, where index is extended to ptrwidth type
{name: "OffPtr", argLength: 1, aux: "Int64"}, // arg0 + auxint (arg0 and result are pointers)
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// Slices
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "SliceMake", argLength: 3}, // arg0=ptr, arg1=len, arg2=cap
{name: "SlicePtr", argLength: 1, typ: "BytePtr"}, // ptr(arg0)
{name: "SliceLen", argLength: 1}, // len(arg0)
{name: "SliceCap", argLength: 1}, // cap(arg0)
cmd/compile: allow conversion from slice to array ptr Panic if the slice is too short. Updates #395 Change-Id: I90f4bff2da5d8f3148ba06d2482084f32b25c29a Reviewed-on: https://go-review.googlesource.com/c/go/+/301650 Trust: Josh Bleecher Snyder <josharian@gmail.com> Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-03-14 14:24:47 -07:00
// SlicePtrUnchecked, like SlicePtr, extracts the pointer from a slice.
// SlicePtr values are assumed non-nil, because they are guarded by bounds checks.
// SlicePtrUnchecked values can be nil.
{name: "SlicePtrUnchecked", argLength: 1},
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
[dev.ssa] cmd/compile: add compose/decompose for complex, phi, constants Still to do: arithmetic Change-Id: I31fd23b34980c9ed4b4e304b8597134b2ba6ca5c Reviewed-on: https://go-review.googlesource.com/14024 Reviewed-by: Keith Randall <khr@golang.org>
2015-08-28 14:24:10 -04:00
// Complex (part/whole)
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "ComplexMake", argLength: 2}, // arg0=real, arg1=imag
{name: "ComplexReal", argLength: 1}, // real(arg0)
{name: "ComplexImag", argLength: 1}, // imag(arg0)
[dev.ssa] cmd/compile: add compose/decompose for complex, phi, constants Still to do: arithmetic Change-Id: I31fd23b34980c9ed4b4e304b8597134b2ba6ca5c Reviewed-on: https://go-review.googlesource.com/14024 Reviewed-by: Keith Randall <khr@golang.org>
2015-08-28 14:24:10 -04:00
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// Strings
cmd/compile: keep pointer input arguments live throughout function Introduce a KeepAlive op which makes sure that its argument is kept live until the KeepAlive. Use KeepAlive to mark pointer input arguments as live after each function call and at each return. We do this change only for pointer arguments. Those are the critical ones to handle because they might have finalizers. Doing compound arguments (slices, structs, ...) is more complicated because we would need to track field liveness individually (we do that for auto variables now, but inputs requires extra trickery). Turn off the automatic marking of args as live. That way, when args are explicitly nulled, plive will know that the original argument is dead. The KeepAlive op will be the eventual implementation of runtime.KeepAlive. Fixes #15277 Change-Id: I5f223e65d99c9f8342c03fbb1512c4d363e903e5 Reviewed-on: https://go-review.googlesource.com/22365 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Russ Cox <rsc@golang.org>
2016-04-21 19:28:28 -07:00
{name: "StringMake", argLength: 2}, // arg0=ptr, arg1=len
{name: "StringPtr", argLength: 1, typ: "BytePtr"}, // ptr(arg0)
{name: "StringLen", argLength: 1, typ: "Int"}, // len(arg0)
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
[dev.ssa] cmd/compile/internal/ssa: implement ITAB Implement ITAB, selecting the itable field of an interface. Soften the lowering check to allow lowerings that leave generic but dead ops behind. (The ITAB lowering does this.) Change-Id: Icc84961dd4060d143602f001311aa1d8be0d7fc0 Reviewed-on: https://go-review.googlesource.com/13144 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-08-04 15:47:22 -07:00
// Interfaces
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "IMake", argLength: 2}, // arg0=itab, arg1=data
cmd/compile: mark the first word of an interface as a uintptr The first word of an interface is a pointer, but for the purposes of GC we don't need to treat it as such. 1. If it is a non-empty interface, the pointer points to an itab which is always in persistentalloc space. 2. If it is an empty interface, the pointer points to a _type. a. If it is a compile-time-allocated type, it points into the read-only data section. b. If it is a reflect-allocated type, it points into the Go heap. Reflect is responsible for keeping a reference to the underlying type so it won't be GCd. If we ever have a moving GC, we need to change this for 2b (as well as scan itabs to update their itab._type fields). Write barriers on the first word of interfaces have already been removed. Change-Id: I643e91d7ac4de980ac2717436eff94097c65d959 Reviewed-on: https://go-review.googlesource.com/97518 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-02-27 13:46:03 -08:00
{name: "ITab", argLength: 1, typ: "Uintptr"}, // arg0=interface, returns itable field
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "IData", argLength: 1}, // arg0=interface, returns data field
[dev.ssa] cmd/compile/internal/ssa: implement ITAB Implement ITAB, selecting the itable field of an interface. Soften the lowering check to allow lowerings that leave generic but dead ops behind. (The ITAB lowering does this.) Change-Id: Icc84961dd4060d143602f001311aa1d8be0d7fc0 Reviewed-on: https://go-review.googlesource.com/13144 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-08-04 15:47:22 -07:00
[dev.ssa] cmd/compile: Allow structs to be SSAd Break small structs up into their components so they can be registerized. Change StructSelect to use field indexes instead of field offsets, as field offsets aren't unique in the presence of zero-sized fields. Change-Id: I2f1dc89f7fa58e1cf58aa1a32b238959d53f62e4 Reviewed-on: https://go-review.googlesource.com/18570 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
2016-01-11 21:05:33 -08:00
// Structs
cmd/compile: generalize struct load/store The SSA backend currently only handle struct with up to 4 fields. Thus, there are different operations corresponding to number fields of the struct. This CL generalizes these with just one OpStructMake, allow struct types with arbitrary number of fields. However, the ssa.MaxStruct is still kept as-is, and future CL will increase this value to optimize large structs. Updates #24416 Change-Id: I192ffbea881186693584476b5639394e79be45c5 Reviewed-on: https://go-review.googlesource.com/c/go/+/611075 Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: David Chase <drchase@google.com>
2024-09-05 14:56:43 +07:00
{name: "StructMake", argLength: -1}, // args...=field0..n-1. Returns struct with n fields.
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "StructSelect", argLength: 1, aux: "Int64"}, // arg0=struct, auxint=field index. Returns the auxint'th field.
[dev.ssa] cmd/compile: Allow structs to be SSAd Break small structs up into their components so they can be registerized. Change StructSelect to use field indexes instead of field offsets, as field offsets aren't unique in the presence of zero-sized fields. Change-Id: I2f1dc89f7fa58e1cf58aa1a32b238959d53f62e4 Reviewed-on: https://go-review.googlesource.com/18570 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
2016-01-11 21:05:33 -08:00
cmd/compile: make [0]T and [1]T SSAable types We used to have to keep on-stack copies of these types. Now they can be registerized. [0]T is kind of trivial but might as well handle it. This change enables another change I'm working on to improve how x.(T) expressions are handled (#17405). This CL helps because now all types that are direct interface types are registerizeable (e.g. [1]*byte). No higher-degree arrays for now because non-constant indexes are hard. Update #17405 Change-Id: I2399940965d17b3969ae66f6fe447a8cefdd6edd Reviewed-on: https://go-review.googlesource.com/32416 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2016-10-30 21:10:03 -07:00
// Arrays
{name: "ArrayMake0"}, // Returns array with 0 elements
{name: "ArrayMake1", argLength: 1}, // Returns array with 1 element
{name: "ArraySelect", argLength: 1, aux: "Int64"}, // arg0=array, auxint=index. Returns a[i].
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// Spill&restore ops for the register allocator. These are
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// semantically identical to OpCopy; they do not take/return
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// stores like regular memory ops do. We can get away without memory
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
// args because we know there is no aliasing of spill slots on the stack.
[dev.ssa] cmd/compile: add max arg length to opcodes Add the max arg length to opcodes and use it in zcse. Doesn't affect speed, but allows better checking in checkFunc and removes the need to keep a list of zero arg opcodes up to date. Change-Id: I157c6587154604119720ec6228b767b6e52bb5c7 Reviewed-on: https://go-review.googlesource.com/19994 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Todd Neal <todd@tneal.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-02-27 08:04:48 -06:00
{name: "StoreReg", argLength: 1},
{name: "LoadReg", argLength: 1},
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// Used during ssa construction. Like Copy, but the arg has not been specified yet.
cmd/compile/internal/ssa: populate SymEffects for SSA Ops Changes to ${GOARCH}Ops.go files were mechanically produced using github.com/mdempsky/ssa-symops, a one-off tool that inserts "SymEffect: X" elements by pattern matching against the Op names. Change-Id: Ibf3e481ffd588647f2a31662d72114b740ccbfcf Reviewed-on: https://go-review.googlesource.com/38084 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-03-09 14:46:43 -08:00
{name: "FwdRef", aux: "Sym", symEffect: "None"},
[dev.ssa] cmd/runtime: generate gc bitmaps for SSA-compiled code This change is all about leveraging the gc bitmap generation that is already done by the current compiler. We rearrange how stack allocation is done so that we generate a variable declaration for each spill. We also reorganize how args/locals are recorded during SSA. Then we can use the existing allocauto/defframe to allocate the stack frame and liveness to make the gc bitmaps. With this change, stack copying works correctly and we no longer need hacks in runtime/stack*.go to make tests work. GC is close to working, it just needs write barriers. Change-Id: I990fb4e3fbe98850c6be35c3185a1c85d9e1a6ba Reviewed-on: https://go-review.googlesource.com/13894 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-08-24 02:16:19 -07:00
all: single space after period. The tree's pretty inconsistent about single space vs double space after a period in documentation. Make it consistently a single space, per earlier decisions. This means contributors won't be confused by misleading precedence. This CL doesn't use go/doc to parse. It only addresses // comments. It was generated with: $ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])') $ go test go/doc -update Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7 Reviewed-on: https://go-review.googlesource.com/20022 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Dave Day <djd@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:21:55 +00:00
// Unknown value. Used for Values whose values don't matter because they are dead code.
[dev.ssa] cmd/compile: optimize phi ops Redo how we keep track of forward references when building SSA. When the forward reference is resolved, update the Value node in place. Improve the phi elimination pass so it can simplify phis of phis. Give SSA package access to decoded line numbers. Fix line numbers for constant booleans. Change-Id: I3dc9896148d260be2f3dd14cbe5db639ec9fa6b7 Reviewed-on: https://go-review.googlesource.com/18674 Reviewed-by: David Chase <drchase@google.com> Run-TryBot: Keith Randall <khr@golang.org>
2016-01-14 16:02:23 -08:00
{name: "Unknown"},
cmd/compile: remove references to *gc.Node in docs ssa.Sym is only implemented by *ir.Name or *obj.LSym. Change-Id: Ia171db618abd8b438fcc2cf402f40f3fe3ec6833 Reviewed-on: https://go-review.googlesource.com/c/go/+/660995 Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-03-26 15:46:30 +01:00
{name: "VarDef", argLength: 1, aux: "Sym", typ: "Mem", symEffect: "None", zeroWidth: true}, // aux is a *ir.Name of a variable that is about to be initialized. arg0=mem, returns mem
all: fix typos Use the following (suboptimal) script to obtain a list of possible typos: #!/usr/bin/env sh set -x git ls-files |\ grep -e '\.\(c\|cc\|go\)$' |\ xargs -n 1\ awk\ '/\/\// { gsub(/.*\/\//, ""); print; } /\/\*/, /\*\// { gsub(/.*\/\*/, ""); gsub(/\*\/.*/, ""); }' |\ hunspell -d en_US -l |\ grep '^[[:upper:]]\{0,1\}[[:lower:]]\{1,\}$' |\ grep -v -e '^.\{1,4\}$' -e '^.\{16,\}$' |\ sort -f |\ uniq -c |\ awk '$1 == 1 { print $2; }' Then, go through the results manually and fix the most obvious typos in the non-vendored code. Change-Id: I3cb5830a176850e1a0584b8a40b47bde7b260eae Reviewed-on: https://go-review.googlesource.com/c/go/+/193848 Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-08 19:36:13 +03:00
// TODO: what's the difference between VarLive and KeepAlive?
cmd/compile: remove references to *gc.Node in docs ssa.Sym is only implemented by *ir.Name or *obj.LSym. Change-Id: Ia171db618abd8b438fcc2cf402f40f3fe3ec6833 Reviewed-on: https://go-review.googlesource.com/c/go/+/660995 Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-03-26 15:46:30 +01:00
{name: "VarLive", argLength: 1, aux: "Sym", symEffect: "Read", zeroWidth: true}, // aux is a *ir.Name of a variable that must be kept live. arg0=mem, returns mem
cmd/compile,runtime: remove ambiguously live logic The previous CL introduced stack objects. This CL removes the old ambiguously live liveness analysis. After this CL we're relying on stack objects exclusively. Update a bunch of liveness tests to reflect the new world. Fixes #22350 Change-Id: I739b26e015882231011ce6bc1a7f426049e59f31 Reviewed-on: https://go-review.googlesource.com/c/134156 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-09-07 14:55:09 -07:00
{name: "KeepAlive", argLength: 2, typ: "Mem", zeroWidth: true}, // arg[0] is a value that must be kept alive until this mark. arg[1]=mem, returns mem
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
cmd/compile,runtime: redo mid-stack inlining tracebacks Work involved in getting a stack trace is divided between runtime.Callers and runtime.CallersFrames. Before this CL, runtime.Callers returns a pc per runtime frame. runtime.CallersFrames is responsible for expanding a runtime frame into potentially multiple user frames. After this CL, runtime.Callers returns a pc per user frame. runtime.CallersFrames just maps those to user frame info. Entries in the result of runtime.Callers are now pcs of the calls (or of the inline marks), not of the instruction just after the call. Fixes #29007 Fixes #28640 Update #26320 Change-Id: I1c9567596ff73dc73271311005097a9188c3406f Reviewed-on: https://go-review.googlesource.com/c/152537 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-12-04 07:58:18 -08:00
// InlMark marks the start of an inlined function body. Its AuxInt field
// distinguishes which entry in the local inline tree it is marking.
{name: "InlMark", argLength: 1, aux: "Int32", typ: "Void"}, // arg[0]=mem, returns void.
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
// Ops for breaking 64-bit operations on 32-bit architectures
{name: "Int64Make", argLength: 2, typ: "UInt64"}, // arg0=hi, arg1=lo
{name: "Int64Hi", argLength: 1, typ: "UInt32"}, // high 32-bit of arg0
{name: "Int64Lo", argLength: 1, typ: "UInt32"}, // low 32-bit of arg0
cmd/compile: implement bits.Mul64 on 32-bit systems This CL implements Mul64uhilo, Hmul64, Hmul64u, and Avg64u on 32-bit systems, with the effect that constant division of both int64s and uint64s can now be emitted directly in all cases, and also that bits.Mul64 can be intrinsified on 32-bit systems. Previously, constant division of uint64s by values 0 ≤ c ≤ 0xFFFF were implemented as uint32 divisions by c and some fixup. After expanding those smaller constant divisions, the code for i/999 required: (386) 7 mul, 10 add, 2 sub, 3 rotate, 3 shift (104 bytes) (arm) 7 mul, 9 add, 3 sub, 2 shift (104 bytes) (mips) 7 mul, 10 add, 5 sub, 6 shift, 3 sgtu (176 bytes) For that much code, we might as well use a full 64x64->128 multiply that can be used for all divisors, not just small ones. Having done that, the same i/999 now generates: (386) 4 mul, 9 add, 2 sub, 2 or, 6 shift (112 bytes) (arm) 4 mul, 8 add, 2 sub, 2 or, 3 shift (92 bytes) (mips) 4 mul, 11 add, 3 sub, 6 shift, 8 sgtu, 4 or (196 bytes) The size increase on 386 is due to a few extra register spills. The size increase on mips is due to add-with-carry being hard. The new approach is more general, letting us delete the old special case and guarantee that all int64 and uint64 divisions by constants are generated directly on 32-bit systems. This especially speeds up code making heavy use of bits.Mul64 with a constant argument, which happens in strconv and various crypto packages. A few examples are benchmarked below. pkg: cmd/compile/internal/test benchmark \ host local linux-amd64 s7 linux-386 s7:GOARCH=386 vs base vs base vs base vs base vs base DivconstI64 ~ ~ ~ -49.66% -21.02% ModconstI64 ~ ~ ~ -13.45% +14.52% DivisiblePow2constI64 ~ ~ ~ +0.97% -1.32% DivisibleconstI64 ~ ~ ~ -20.01% -48.28% DivisibleWDivconstI64 ~ ~ -1.76% -38.59% -42.74% DivconstU64/3 ~ ~ ~ -13.82% -4.09% DivconstU64/5 ~ ~ ~ -14.10% -3.54% DivconstU64/37 -2.07% -4.45% ~ -19.60% -9.55% DivconstU64/1234567 ~ ~ ~ -61.55% -56.93% ModconstU64 ~ ~ ~ -6.25% ~ DivisibleconstU64 ~ ~ ~ -2.78% -7.82% DivisibleWDivconstU64 ~ ~ ~ +4.23% +2.56% pkg: math/bits benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base Add ~ ~ ~ ~ Add32 +1.59% ~ ~ ~ Add64 ~ ~ ~ ~ Add64multiple ~ ~ ~ ~ Sub ~ ~ ~ ~ Sub32 ~ ~ ~ ~ Sub64 ~ ~ -9.20% ~ Sub64multiple ~ ~ ~ ~ Mul ~ ~ ~ ~ Mul32 ~ ~ ~ ~ Mul64 ~ ~ -41.58% -53.21% Div ~ ~ ~ ~ Div32 ~ ~ ~ ~ Div64 ~ ~ ~ ~ pkg: strconv benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base ParseInt/Pos/7bit ~ ~ -11.08% -6.75% ParseInt/Pos/26bit ~ ~ -13.65% -11.02% ParseInt/Pos/31bit ~ ~ -14.65% -9.71% ParseInt/Pos/56bit -1.80% ~ -17.97% -10.78% ParseInt/Pos/63bit ~ ~ -13.85% -9.63% ParseInt/Neg/7bit ~ ~ -12.14% -7.26% ParseInt/Neg/26bit ~ ~ -14.18% -9.81% ParseInt/Neg/31bit ~ ~ -14.51% -9.02% ParseInt/Neg/56bit ~ ~ -15.79% -9.79% ParseInt/Neg/63bit ~ ~ -15.68% -11.07% AppendFloat/Decimal ~ ~ -7.25% -12.26% AppendFloat/Float ~ ~ -15.96% -19.45% AppendFloat/Exp ~ ~ -13.96% -17.76% AppendFloat/NegExp ~ ~ -14.89% -20.27% AppendFloat/LongExp ~ ~ -12.68% -17.97% AppendFloat/Big ~ ~ -11.10% -16.64% AppendFloat/BinaryExp ~ ~ ~ ~ AppendFloat/32Integer ~ ~ -10.05% -10.91% AppendFloat/32ExactFraction ~ ~ -8.93% -13.00% AppendFloat/32Point ~ ~ -10.36% -14.89% AppendFloat/32Exp ~ ~ -9.88% -13.54% AppendFloat/32NegExp ~ ~ -10.16% -14.26% AppendFloat/32Shortest ~ ~ -11.39% -14.96% AppendFloat/32Fixed8Hard ~ ~ ~ -2.31% AppendFloat/32Fixed9Hard ~ ~ ~ -7.01% AppendFloat/64Fixed1 ~ ~ -2.83% -8.23% AppendFloat/64Fixed2 ~ ~ ~ -7.94% AppendFloat/64Fixed3 ~ ~ -4.07% -7.22% AppendFloat/64Fixed4 ~ ~ -7.24% -7.62% AppendFloat/64Fixed12 ~ ~ -6.57% -4.82% AppendFloat/64Fixed16 ~ ~ -4.00% -5.81% AppendFloat/64Fixed12Hard -2.22% ~ -4.07% -6.35% AppendFloat/64Fixed17Hard -2.12% ~ ~ -3.79% AppendFloat/64Fixed18Hard -1.89% ~ +2.48% ~ AppendFloat/Slowpath64 -1.85% ~ -14.49% -18.21% AppendFloat/SlowpathDenormal64 ~ ~ -13.08% -19.41% pkg: crypto/internal/fips140/nistec/fiat benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base Mul/P224 ~ ~ -29.95% -39.60% Mul/P384 ~ ~ -37.11% -63.33% Mul/P521 ~ ~ -26.62% -12.42% Square/P224 +1.46% ~ -40.62% -49.18% Square/P384 ~ ~ -45.51% -69.68% Square/P521 +90.37% ~ -25.26% -11.23% (The +90% is a separate problem and not real; that much variation can be seen on that system by running the same binary from two different files.) pkg: crypto/internal/fips140/edwards25519 benchmark \ host s7 linux-amd64 linux-386 s7:GOARCH=386 vs base vs base vs base vs base EncodingDecoding ~ ~ -34.67% -35.75% ScalarBaseMult ~ ~ -31.25% -30.29% ScalarMult ~ ~ -33.45% -32.54% VarTimeDoubleScalarBaseMult ~ ~ -33.78% -33.68% Change-Id: Id3c91d42cd01def6731b755e99f8f40c6ad1bb65 Reviewed-on: https://go-review.googlesource.com/c/go/+/716061 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com>
2025-10-27 19:41:39 -04:00
{name: "Add32carry", argLength: 2, commutative: true, typ: "(UInt32,Flags)"}, // arg0 + arg1, returns (value, carry)
{name: "Add32withcarry", argLength: 3, commutative: true}, // arg0 + arg1 + arg2, arg2=carry (0 or 1)
{name: "Add32carrywithcarry", argLength: 3, commutative: true, typ: "(UInt32,Flags)"}, // arg0 + arg1 + arg2, arg2=carry, returns (value, carry)
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
cmd/compile: inline atomics from runtime/internal/atomic on amd64 Inline atomic reads and writes on amd64. There's no reason to pay the overhead of a call for these. To keep atomic loads from being reordered, we make them return a <value,memory> tuple. Change the meaning of resultInArg0 for tuple-generating ops to mean the first part of the result tuple, not the second. This means we can always put the store part of the tuple last, matching how arguments are laid out. This requires reordering the outputs of add32carry and sub32carry and their descendents in various architectures. benchmark old ns/op new ns/op delta BenchmarkAtomicLoad64-8 2.09 0.26 -87.56% BenchmarkAtomicStore64-8 7.54 5.72 -24.14% TBD (in a different CL): Cas, Or8, ... Change-Id: I713ea88e7da3026c44ea5bdb56ed094b20bc5207 Reviewed-on: https://go-review.googlesource.com/27641 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-08-23 16:49:28 -07:00
{name: "Sub32carry", argLength: 2, typ: "(UInt32,Flags)"}, // arg0 - arg1, returns (value, carry)
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
{name: "Sub32withcarry", argLength: 3}, // arg0 - arg1 - arg2, arg2=carry (0 or 1)
cmd/compile: intrinsify math/bits.Add on amd64 name old time/op new time/op delta Add-8 1.11ns ± 0% 1.18ns ± 0% +6.31% (p=0.029 n=4+4) Add32-8 1.02ns ± 0% 1.02ns ± 1% ~ (p=0.333 n=4+5) Add64-8 1.11ns ± 1% 1.17ns ± 0% +5.79% (p=0.008 n=5+5) Add64multiple-8 4.35ns ± 1% 0.86ns ± 0% -80.22% (p=0.000 n=5+4) The individual ops are a bit slower (but still very fast). Using the ops in carry chains is very fast. Update #28273 Change-Id: Id975f76df2b930abf0e412911d327b6c5b1befe5 Reviewed-on: https://go-review.googlesource.com/c/144257 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-10-23 14:05:38 -07:00
{name: "Add64carry", argLength: 3, commutative: true, typ: "(UInt64,UInt64)"}, // arg0 + arg1 + arg2, arg2 must be 0 or 1. returns (value, value>>64)
cmd/compile: intrinsify math/bits.Sub on amd64 name old time/op new time/op delta Sub-8 1.12ns ± 1% 1.17ns ± 1% +5.20% (p=0.008 n=5+5) Sub32-8 1.11ns ± 0% 1.11ns ± 0% ~ (all samples are equal) Sub64-8 1.12ns ± 0% 1.18ns ± 1% +5.00% (p=0.016 n=4+5) Sub64multiple-8 4.10ns ± 1% 0.86ns ± 1% -78.93% (p=0.008 n=5+5) Fixes #28273 Change-Id: Ibcb6f2fd32d987c3bcbae4f4cd9d335a3de98548 Reviewed-on: https://go-review.googlesource.com/c/144258 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-10-23 14:38:22 -07:00
{name: "Sub64borrow", argLength: 3, typ: "(UInt64,UInt64)"}, // arg0 - (arg1 + arg2), arg2 must be 0 or 1. returns (value, value>>64&1)
cmd/compile: intrinsify math/bits.Add on amd64 name old time/op new time/op delta Add-8 1.11ns ± 0% 1.18ns ± 0% +6.31% (p=0.029 n=4+4) Add32-8 1.02ns ± 0% 1.02ns ± 1% ~ (p=0.333 n=4+5) Add64-8 1.11ns ± 1% 1.17ns ± 0% +5.79% (p=0.008 n=5+5) Add64multiple-8 4.35ns ± 1% 0.86ns ± 0% -80.22% (p=0.000 n=5+4) The individual ops are a bit slower (but still very fast). Using the ops in carry chains is very fast. Update #28273 Change-Id: Id975f76df2b930abf0e412911d327b6c5b1befe5 Reviewed-on: https://go-review.googlesource.com/c/144257 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-10-23 14:05:38 -07:00
[dev.ssa] cmd/compile: handle 64-bit shifts on ARM Also fix a mistake in previous CL about x8 and x16 shifts: the shift needs ZeroExt. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: Ibc352760023d38bc6b9c5251e929fe26e016637a Reviewed-on: https://go-review.googlesource.com/23486 Reviewed-by: David Chase <drchase@google.com>
2016-05-25 23:17:42 -04:00
{name: "Signmask", argLength: 1, typ: "Int32"}, // 0 if arg0 >= 0, -1 if arg0 < 0
{name: "Zeromask", argLength: 1, typ: "UInt32"}, // 0 if arg0 == 0, 0xffffffff if arg0 != 0
cmd/compile: use masks instead of branches for slicing When we do var x []byte = ... y := x[i:] We can't just use y.ptr = x.ptr + i, as the new pointer may point to the next object in memory after the backing array. We used to fix this by doing: y.cap = x.cap - i delta := i if y.cap == 0 { delta = 0 } y.ptr = x.ptr + delta That generates a branch in what is otherwise straight-line code. Better to do: y.cap = x.cap - i mask := (y.cap - 1) >> 63 // -1 if y.cap==0, 0 otherwise y.ptr = x.ptr + i &^ mask It's about the same number of instructions (~4, depending on what parts are constant, and the target architecture), but it is all inline. It plays nicely with CSE, and the mask can be computed in parallel with the index (in cases where a multiply is required). It is a minor win in both speed and space. Change-Id: Ied60465a0b8abb683c02208402e5bb7ac0e8370f Reviewed-on: https://go-review.googlesource.com/32022 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-10-25 15:49:52 -07:00
{name: "Slicemask", argLength: 1}, // 0 if arg0 == 0, -1 if arg0 > 0, undef if arg0<0. Type is native int size.
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
cmd/compile: add spectre mitigation mode enabled by -spectre This commit adds a new cmd/compile flag -spectre, which accepts a comma-separated list of possible Spectre mitigations to apply, or the empty string (none), or "all". The only known mitigation right now is "index", which uses conditional moves to ensure that x86-64 CPUs do not speculate past index bounds checks. Speculating past index bounds checks may be problematic on systems running privileged servers that accept requests from untrusted users who can execute their own programs on the same machine. (And some more constraints that make it even more unlikely in practice.) The cases this protects against are analogous to the ones Microsoft explains in the "Array out of bounds load/store feeding ..." sections here: https://docs.microsoft.com/en-us/cpp/security/developer-guidance-speculative-execution?view=vs-2019#array-out-of-bounds-load-feeding-an-indirect-branch Change-Id: Ib7532d7e12466b17e04c4e2075c2a456dc98f610 Reviewed-on: https://go-review.googlesource.com/c/go/+/222660 Reviewed-by: Keith Randall <khr@golang.org>
2020-01-06 10:31:39 -05:00
{name: "SpectreIndex", argLength: 2}, // arg0 if 0 <= arg0 < arg1, 0 otherwise. Type is native int size.
{name: "SpectreSliceIndex", argLength: 2}, // arg0 if 0 <= arg0 <= arg1, 0 otherwise. Type is native int size.
[dev.ssa] cmd/compile: handle floating point on ARM Machine supports (or the runtime simulates in soft float mode) (u)int32<->float conversions. The frontend rewrites int64<->float conversions to call to runtime function. For int64->float32 conversion, the frontend generates . . AS u(100) l(10) tc(1) . . . NAME-main.~r1 u(1) a(true) g(1) l(9) x(8+0) class(PPARAMOUT) f(1) float32 . . . CALLFUNC u(100) l(10) tc(1) float32 . . . . NAME-runtime.int64tofloat64 u(1) a(true) x(0+0) class(PFUNC) tc(1) used(true) FUNC-func(int64) float64 The CALLFUNC node has type float32, whereas runtime.int64tofloat64 returns float64. The legacy backend implicitly makes a float64->float32 conversion. The SSA backend does not do implicit conversion, so we insert an explicit CONV here. All cmd/compile/internal/gc/testdata/*_ssa.go tests passed. Progress on SSA for ARM. Still not complete. Update #15365. Change-Id: I30937c8ff977271246b068f48224693776804339 Reviewed-on: https://go-review.googlesource.com/23652 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-31 11:27:16 -04:00
{name: "Cvt32Uto32F", argLength: 1}, // uint32 -> float32, only used on 32-bit arch
{name: "Cvt32Uto64F", argLength: 1}, // uint32 -> float64, only used on 32-bit arch
{name: "Cvt32Fto32U", argLength: 1}, // float32 -> uint32, only used on 32-bit arch
{name: "Cvt64Fto32U", argLength: 1}, // float64 -> uint32, only used on 32-bit arch
cmd/compile: add more ARM64 optimizations - Use machine instructions for uint64<->float conversions - Do not enforce alignment on Zero/Move ARM64 supports unaligned load/stores, but only aligned offset or small offset can be encoded into instructions. - Do combined loads Change-Id: Iffca7dd0f13070b17b784861ce5a30af584680eb Reviewed-on: https://go-review.googlesource.com/27086 Reviewed-by: David Chase <drchase@google.com>
2016-08-16 14:17:33 -04:00
{name: "Cvt64Uto32F", argLength: 1}, // uint64 -> float32, only used on archs that has the instruction
{name: "Cvt64Uto64F", argLength: 1}, // uint64 -> float64, only used on archs that has the instruction
{name: "Cvt32Fto64U", argLength: 1}, // float32 -> uint64, only used on archs that has the instruction
{name: "Cvt64Fto64U", argLength: 1}, // float64 -> uint64, only used on archs that has the instruction
[dev.ssa] cmd/compile: handle floating point on ARM Machine supports (or the runtime simulates in soft float mode) (u)int32<->float conversions. The frontend rewrites int64<->float conversions to call to runtime function. For int64->float32 conversion, the frontend generates . . AS u(100) l(10) tc(1) . . . NAME-main.~r1 u(1) a(true) g(1) l(9) x(8+0) class(PPARAMOUT) f(1) float32 . . . CALLFUNC u(100) l(10) tc(1) float32 . . . . NAME-runtime.int64tofloat64 u(1) a(true) x(0+0) class(PFUNC) tc(1) used(true) FUNC-func(int64) float64 The CALLFUNC node has type float32, whereas runtime.int64tofloat64 returns float64. The legacy backend implicitly makes a float64->float32 conversion. The SSA backend does not do implicit conversion, so we insert an explicit CONV here. All cmd/compile/internal/gc/testdata/*_ssa.go tests passed. Progress on SSA for ARM. Still not complete. Update #15365. Change-Id: I30937c8ff977271246b068f48224693776804339 Reviewed-on: https://go-review.googlesource.com/23652 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-31 11:27:16 -04:00
[dev.ssa] cmd/compile: decompose 64-bit integer on ARM Introduce dec64 rules to (generically) decompose 64-bit integer on 32-bit architectures. 64-bit integer is composed/decomposed with Int64Make/Hi/Lo ops, as for complex types. The idea of dealing with Add64 is the following: (Add64 (Int64Make xh xl) (Int64Make yh yl)) -> (Int64Make (Add32withcarry xh yh (Select0 (Add32carry xl yl))) (Select1 (Add32carry xl yl))) where Add32carry returns a tuple (flags,uint32). Select0 and Select1 read the first and the second component of the tuple, respectively. The two Add32carry will be CSE'd. Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo). Also add support of KeepAlive, to fix build after merge. Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go, cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in cmd/compile/internal/gc/testdata passed. Progress on SSA for ARM. Still not complete. Updates #15365. Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec Reviewed-on: https://go-review.googlesource.com/23213 Reviewed-by: Keith Randall <khr@golang.org>
2016-05-18 18:14:36 -04:00
// pseudo-ops for breaking Tuple
cmd/compile: add new ops for experiment with late call expansion Added Dereference, StaticLECall, SelectN, SelectNAddr Change-Id: I5426ae488e83956539aa07f7415b8acc26c933e6 Reviewed-on: https://go-review.googlesource.com/c/go/+/239082 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-06-15 14:20:36 -04:00
{name: "Select0", argLength: 1, zeroWidth: true}, // the first component of a tuple
{name: "Select1", argLength: 1, zeroWidth: true}, // the second component of a tuple
cmd/compile: add MakeTuple generic SSA op to remove duplicate Select[01] rules Change-Id: Id94a5e503f02aa29dc1e334b521770107d4261db Reviewed-on: https://go-review.googlesource.com/c/go/+/656615 Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Jorropo <jorropo.pgm@gmail.com>
2025-03-11 09:52:10 +01:00
{name: "MakeTuple", argLength: 2}, // arg0 arg1 are components of a "Tuple" (like the result from a 128bits op).
cmd/compile: using new calls, optimize runtime.memequal(x,constant,1) Proof of concept; also an actual optimization that fires 180 times in the Go source base. Change-Id: I5cb87474be764264cde6e4cbcb471ef109306f08 Reviewed-on: https://go-review.googlesource.com/c/go/+/248404 Trust: David Chase <drchase@google.com> Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-08-13 20:43:39 -04:00
{name: "SelectN", argLength: 1, aux: "Int64"}, // arg0=result, auxint=field index. Returns the auxint'th member.
{name: "SelectNAddr", argLength: 1, aux: "Int64"}, // arg0=result, auxint=field index. Returns the address of auxint'th member. Used for un-SSA-able result types.
{name: "MakeResult", argLength: -1}, // arg0 .. are components of a "Result" (like the result from a Call). The last arg should be memory (like the result from a call).
cmd/compile: inline atomics from runtime/internal/atomic on amd64 Inline atomic reads and writes on amd64. There's no reason to pay the overhead of a call for these. To keep atomic loads from being reordered, we make them return a <value,memory> tuple. Change the meaning of resultInArg0 for tuple-generating ops to mean the first part of the result tuple, not the second. This means we can always put the store part of the tuple last, matching how arguments are laid out. This requires reordering the outputs of add32carry and sub32carry and their descendents in various architectures. benchmark old ns/op new ns/op delta BenchmarkAtomicLoad64-8 2.09 0.26 -87.56% BenchmarkAtomicStore64-8 7.54 5.72 -24.14% TBD (in a different CL): Cas, Or8, ... Change-Id: I713ea88e7da3026c44ea5bdb56ed094b20bc5207 Reviewed-on: https://go-review.googlesource.com/27641 Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-08-23 16:49:28 -07:00
cmd/compile: drop SSA atomics TODO These ops have been used for sync/atomic since golang.org/cl/28076. Change-Id: Ic1445c073273f6b191b97018e8eb6f7ad9c48922 Reviewed-on: https://go-review.googlesource.com/c/go/+/261077 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-10-09 12:44:07 -04:00
// Atomic operations used for semantically inlining sync/atomic and
runtime: migrate internal/atomic to internal/runtime For #65355 Change-Id: I65dd090fb99de9b231af2112c5ccb0eb635db2be Reviewed-on: https://go-review.googlesource.com/c/go/+/560155 Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ibrahim Bazoka <ibrahimbazoka729@gmail.com> Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
2024-02-01 10:21:14 +08:00
// internal/runtime/atomic. Atomic loads return a new memory so that
cmd/compile: drop SSA atomics TODO These ops have been used for sync/atomic since golang.org/cl/28076. Change-Id: Ic1445c073273f6b191b97018e8eb6f7ad9c48922 Reviewed-on: https://go-review.googlesource.com/c/go/+/261077 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-10-09 12:44:07 -04:00
// the loads are properly ordered with respect to other loads and
// stores.
cmd/compile,runtime/internal/atomic: add Load8 Change-Id: Id52a5730cf9207ee7ccebac4ef12791dc5720e7c Reviewed-on: https://go-review.googlesource.com/c/go/+/172283 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-03-28 14:58:06 -04:00
{name: "AtomicLoad8", argLength: 2, typ: "(UInt8,Mem)"}, // Load from arg0. arg1=memory. Returns loaded value and new memory.
cmd/compile, runtime: add new lightweight atomics for ppc64x This change creates the infrastructure for new lightweight atomics primitives in runtime/internal/atomic: - LoadAcq, for load-acquire - StoreRel, for store-release - CasRel, for Compare-and-Swap-release and implements them for ppc64x. There is visible performance improvement in producer-consumer scenarios, like BenchmarkChanProdCons*: benchmark old ns/op new ns/op delta BenchmarkChanProdCons0-48 2034 2034 +0.00% BenchmarkChanProdCons10-48 1798 1608 -10.57% BenchmarkChanProdCons100-48 1596 1585 -0.69% BenchmarkChanProdConsWork0-48 2084 2046 -1.82% BenchmarkChanProdConsWork10-48 1829 1668 -8.80% BenchmarkChanProdConsWork100-48 1650 1650 +0.00% Fixes #21348 Change-Id: I1f6ce377e4a0fe4bd7f5f775e8036f50070ad8db Reviewed-on: https://go-review.googlesource.com/c/142277 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-08-06 15:36:16 -05:00
{name: "AtomicLoad32", argLength: 2, typ: "(UInt32,Mem)"}, // Load from arg0. arg1=memory. Returns loaded value and new memory.
{name: "AtomicLoad64", argLength: 2, typ: "(UInt64,Mem)"}, // Load from arg0. arg1=memory. Returns loaded value and new memory.
{name: "AtomicLoadPtr", argLength: 2, typ: "(BytePtr,Mem)"}, // Load from arg0. arg1=memory. Returns loaded value and new memory.
{name: "AtomicLoadAcq32", argLength: 2, typ: "(UInt32,Mem)"}, // Load from arg0. arg1=memory. Lock acquisition, returns loaded value and new memory.
cmd/compiler,cmd/go,sync: add internal {LoadAcq,StoreRel}64 on ppc64 Add an internal atomic intrinsic for load with acquire semantics (extending LoadAcq to 64b) and add LoadAcquintptr for internal use within the sync package. For other arches, this remaps to the appropriate atomic.Load{,64} intrinsic which should not alter code generation. Similarly, add StoreRel{uintptr,64} for consistency, and inline. Finally, add an exception to allow sync to directly use the runtime/internal/atomic package which avoids more convoluted workarounds (contributed by Lynn Boger). In an extreme example, sync.(*Pool).pin consumes 20% of wall time during fmt tests. This is reduced to 5% on ppc64le/power9. From the fmt benchmarks on ppc64le: name old time/op new time/op delta SprintfPadding 468ns ± 0% 451ns ± 0% -3.63% SprintfEmpty 73.3ns ± 0% 51.9ns ± 0% -29.20% SprintfString 135ns ± 0% 122ns ± 0% -9.63% SprintfTruncateString 232ns ± 0% 214ns ± 0% -7.76% SprintfTruncateBytes 216ns ± 0% 202ns ± 0% -6.48% SprintfSlowParsingPath 162ns ± 0% 142ns ± 0% -12.35% SprintfQuoteString 1.00µs ± 0% 0.99µs ± 0% -1.39% SprintfInt 117ns ± 0% 104ns ± 0% -11.11% SprintfIntInt 190ns ± 0% 175ns ± 0% -7.89% SprintfPrefixedInt 232ns ± 0% 212ns ± 0% -8.62% SprintfFloat 270ns ± 0% 255ns ± 0% -5.56% SprintfComplex 1.01µs ± 0% 0.99µs ± 0% -1.68% SprintfBoolean 127ns ± 0% 111ns ± 0% -12.60% SprintfHexString 220ns ± 0% 198ns ± 0% -10.00% SprintfHexBytes 261ns ± 0% 252ns ± 0% -3.45% SprintfBytes 600ns ± 0% 590ns ± 0% -1.67% SprintfStringer 684ns ± 0% 658ns ± 0% -3.80% SprintfStructure 2.57µs ± 0% 2.57µs ± 0% -0.12% ManyArgs 669ns ± 0% 646ns ± 0% -3.44% FprintInt 140ns ± 0% 136ns ± 0% -2.86% FprintfBytes 184ns ± 0% 181ns ± 0% -1.63% FprintIntNoAlloc 140ns ± 0% 136ns ± 0% -2.86% ScanInts 929µs ± 0% 921µs ± 0% -0.79% ScanRecursiveInt 122ms ± 0% 121ms ± 0% -0.11% ScanRecursiveIntReaderWrapper 122ms ± 0% 122ms ± 0% -0.18% Change-Id: I4d66780261b57b06ef600229e475462e7313f0d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/253748 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Keith Randall <khr@golang.org> Trust: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-09-09 17:24:23 -05:00
{name: "AtomicLoadAcq64", argLength: 2, typ: "(UInt64,Mem)"}, // Load from arg0. arg1=memory. Lock acquisition, returns loaded value and new memory.
cmd/compile: intrinsics for runtime/internal/atomic.Store8 For #10958, #24543, but makes sense on its own. Change-Id: I2a87dab66b82a1863e4b6512b1f8def51463ce2a Reviewed-on: https://go-review.googlesource.com/c/go/+/203284 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-23 10:20:49 -04:00
{name: "AtomicStore8", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
cmd/compile, runtime: add new lightweight atomics for ppc64x This change creates the infrastructure for new lightweight atomics primitives in runtime/internal/atomic: - LoadAcq, for load-acquire - StoreRel, for store-release - CasRel, for Compare-and-Swap-release and implements them for ppc64x. There is visible performance improvement in producer-consumer scenarios, like BenchmarkChanProdCons*: benchmark old ns/op new ns/op delta BenchmarkChanProdCons0-48 2034 2034 +0.00% BenchmarkChanProdCons10-48 1798 1608 -10.57% BenchmarkChanProdCons100-48 1596 1585 -0.69% BenchmarkChanProdConsWork0-48 2084 2046 -1.82% BenchmarkChanProdConsWork10-48 1829 1668 -8.80% BenchmarkChanProdConsWork100-48 1650 1650 +0.00% Fixes #21348 Change-Id: I1f6ce377e4a0fe4bd7f5f775e8036f50070ad8db Reviewed-on: https://go-review.googlesource.com/c/142277 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-08-06 15:36:16 -05:00
{name: "AtomicStore32", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
{name: "AtomicStore64", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
{name: "AtomicStorePtrNoWB", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
{name: "AtomicStoreRel32", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Lock release, returns memory.
cmd/compiler,cmd/go,sync: add internal {LoadAcq,StoreRel}64 on ppc64 Add an internal atomic intrinsic for load with acquire semantics (extending LoadAcq to 64b) and add LoadAcquintptr for internal use within the sync package. For other arches, this remaps to the appropriate atomic.Load{,64} intrinsic which should not alter code generation. Similarly, add StoreRel{uintptr,64} for consistency, and inline. Finally, add an exception to allow sync to directly use the runtime/internal/atomic package which avoids more convoluted workarounds (contributed by Lynn Boger). In an extreme example, sync.(*Pool).pin consumes 20% of wall time during fmt tests. This is reduced to 5% on ppc64le/power9. From the fmt benchmarks on ppc64le: name old time/op new time/op delta SprintfPadding 468ns ± 0% 451ns ± 0% -3.63% SprintfEmpty 73.3ns ± 0% 51.9ns ± 0% -29.20% SprintfString 135ns ± 0% 122ns ± 0% -9.63% SprintfTruncateString 232ns ± 0% 214ns ± 0% -7.76% SprintfTruncateBytes 216ns ± 0% 202ns ± 0% -6.48% SprintfSlowParsingPath 162ns ± 0% 142ns ± 0% -12.35% SprintfQuoteString 1.00µs ± 0% 0.99µs ± 0% -1.39% SprintfInt 117ns ± 0% 104ns ± 0% -11.11% SprintfIntInt 190ns ± 0% 175ns ± 0% -7.89% SprintfPrefixedInt 232ns ± 0% 212ns ± 0% -8.62% SprintfFloat 270ns ± 0% 255ns ± 0% -5.56% SprintfComplex 1.01µs ± 0% 0.99µs ± 0% -1.68% SprintfBoolean 127ns ± 0% 111ns ± 0% -12.60% SprintfHexString 220ns ± 0% 198ns ± 0% -10.00% SprintfHexBytes 261ns ± 0% 252ns ± 0% -3.45% SprintfBytes 600ns ± 0% 590ns ± 0% -1.67% SprintfStringer 684ns ± 0% 658ns ± 0% -3.80% SprintfStructure 2.57µs ± 0% 2.57µs ± 0% -0.12% ManyArgs 669ns ± 0% 646ns ± 0% -3.44% FprintInt 140ns ± 0% 136ns ± 0% -2.86% FprintfBytes 184ns ± 0% 181ns ± 0% -1.63% FprintIntNoAlloc 140ns ± 0% 136ns ± 0% -2.86% ScanInts 929µs ± 0% 921µs ± 0% -0.79% ScanRecursiveInt 122ms ± 0% 121ms ± 0% -0.11% ScanRecursiveIntReaderWrapper 122ms ± 0% 122ms ± 0% -0.18% Change-Id: I4d66780261b57b06ef600229e475462e7313f0d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/253748 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com> Reviewed-by: Keith Randall <khr@golang.org> Trust: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-09-09 17:24:23 -05:00
{name: "AtomicStoreRel64", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Lock release, returns memory.
cmd/compile/internal/ssa: intrinsify atomic.Xchg8 on amd64 For #68578 Change-Id: Ia9580579bfc4709945bfcf6ec3803d5d11812187 Reviewed-on: https://go-review.googlesource.com/c/go/+/606901 Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Martin Möhrmann <moehrmann@google.com> Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-08-19 13:58:42 -07:00
{name: "AtomicExchange8", argLength: 3, typ: "(UInt8,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
cmd/compile, runtime: add new lightweight atomics for ppc64x This change creates the infrastructure for new lightweight atomics primitives in runtime/internal/atomic: - LoadAcq, for load-acquire - StoreRel, for store-release - CasRel, for Compare-and-Swap-release and implements them for ppc64x. There is visible performance improvement in producer-consumer scenarios, like BenchmarkChanProdCons*: benchmark old ns/op new ns/op delta BenchmarkChanProdCons0-48 2034 2034 +0.00% BenchmarkChanProdCons10-48 1798 1608 -10.57% BenchmarkChanProdCons100-48 1596 1585 -0.69% BenchmarkChanProdConsWork0-48 2084 2046 -1.82% BenchmarkChanProdConsWork10-48 1829 1668 -8.80% BenchmarkChanProdConsWork100-48 1650 1650 +0.00% Fixes #21348 Change-Id: I1f6ce377e4a0fe4bd7f5f775e8036f50070ad8db Reviewed-on: https://go-review.googlesource.com/c/142277 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2018-08-06 15:36:16 -05:00
{name: "AtomicExchange32", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicExchange64", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicAdd32", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicAdd64", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicCompareAndSwap32", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Returns true if store happens and new memory.
{name: "AtomicCompareAndSwap64", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Returns true if store happens and new memory.
all: use "reports whether" consistently in the few places that didn't Go documentation style for boolean funcs is to say: // Foo reports whether ... func Foo() bool (rather than "returns true if") This CL also replaces 4 uses of "iff" with the same "reports whether" wording, which doesn't lose any meaning, and will prevent people from sending typo fixes when they don't realize it's "if and only if". In the past I think we've had the typo CLs updated to just say "reports whether". So do them all at once. (Inspired by the addition of another "returns true if" in CL 146938 in fd_plan9.go) Created with: $ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor) $ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor) Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f Reviewed-on: https://go-review.googlesource.com/c/147037 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-02 15:18:43 +00:00
{name: "AtomicCompareAndSwapRel32", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Lock release, reports whether store happens and new memory.
cmd/compile: fix typing of atomic logical operations For atomic AND and OR operations on memory, we currently have two views of the op. One just does the operation on the memory and returns just a memory. The other does the operation on the memory and returns the old value (before having the logical operation done to it) and memory. Update #61395 These two type differently, and there's currently some confusion in our rules about which is which. Use different names for the two different flavors so we don't get them confused. Change-Id: I07b4542db672b2cee98169ac42b67db73c482093 Reviewed-on: https://go-review.googlesource.com/c/go/+/594976 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Nicolas Hillegeer <aktau@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
2024-06-25 14:01:09 -07:00
// Older atomic logical operations which don't return the old value.
{name: "AtomicAnd8", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns memory.
{name: "AtomicOr8", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns memory.
{name: "AtomicAnd32", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns memory.
{name: "AtomicOr32", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns memory.
// Newer atomic logical operations which return the old value.
{name: "AtomicAnd64value", argLength: 3, typ: "(Uint64, Mem)", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicAnd32value", argLength: 3, typ: "(Uint32, Mem)", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicAnd8value", argLength: 3, typ: "(Uint8, Mem)", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicOr64value", argLength: 3, typ: "(Uint64, Mem)", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicOr32value", argLength: 3, typ: "(Uint32, Mem)", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicOr8value", argLength: 3, typ: "(Uint8, Mem)", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
cmd/compile: experiment which clobbers all dead pointer fields The experiment "clobberdead" clobbers all pointer fields that the compiler thinks are dead, just before and after every safepoint. Useful for debugging the generation of live pointer bitmaps. Helped find the following issues: Update #15936 Update #16026 Update #16095 Update #18860 Change-Id: Id1d12f86845e3d93bae903d968b1eac61fc461f9 Reviewed-on: https://go-review.googlesource.com/23924 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-06-08 22:02:08 -07:00
cmd/compile: improve atomic add intrinsics with ARMv8.1 new instruction ARMv8.1 has added new instruction (LDADDAL) for atomic memory operations. This CL improves existing atomic add intrinsics with the new instruction. Since the new instruction is only guaranteed to be present after ARMv8.1, we guard its usage with a conditional on CPU feature. Performance result on ARMv8.1 machine: name old time/op new time/op delta Xadd-224 1.05µs ± 6% 0.02µs ± 4% -98.06% (p=0.000 n=10+8) Xadd64-224 1.05µs ± 3% 0.02µs ±13% -98.10% (p=0.000 n=9+10) [Geo mean] 1.05µs 0.02µs -98.08% Performance result on ARMv8.0 machine: name old time/op new time/op delta Xadd-46 538ns ± 1% 541ns ± 1% +0.62% (p=0.000 n=9+9) Xadd64-46 505ns ± 1% 508ns ± 0% +0.48% (p=0.003 n=9+8) [Geo mean] 521ns 524ns +0.55% Change-Id: If4b5d8d0e2d6f84fe1492a4f5de0789910ad0ee9 Reviewed-on: https://go-review.googlesource.com/81877 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-11-03 02:05:28 +00:00
// Atomic operation variants
// These variants have the same semantics as above atomic operations.
// But they are used for generating more efficient code on certain modern machines, with run-time CPU feature detection.
cmd/compile: fix the typos in genericOps.go Change-Id: I7ff869e21e67cf6a193f7a92bf7b05f047ee005c GitHub-Last-Rev: bf01f582492c95ff169eab89b688bfb938695c50 GitHub-Pull-Request: golang/go#69957 Reviewed-on: https://go-review.googlesource.com/c/go/+/620778 Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-10-21 06:29:38 +00:00
// On ARM64, these are used when the LSE hardware feature is available (either known at compile time or detected at runtime). If LSE is not available,
all: fix some function names and typos in comment Change-Id: I07e7c8eaa5bd4bac0d576b2f2f4cd3f81b0b77a4 Reviewed-on: https://go-review.googlesource.com/c/go/+/630055 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Commit-Queue: Ian Lance Taylor <iant@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-11-20 21:56:27 +08:00
// then the basic atomic operations are used instead.
cmd/compiler,internal/runtime/atomic: optimize Store{64,32,8} on loong64 On Loong64, AMSWAPDB{W,V} instructions are supported by default, and AMSWAPDB{B,H} [1] is a new instruction added by LA664(Loongson 3A6000) and later microarchitectures. Therefore, AMSWAPDB{W,V} (full barrier) is used to implement AtomicStore{32,64}, and the traditional MOVB or the new AMSWAPDBB is used to implement AtomicStore8 according to the CPU feature. The StoreRelease barrier on Loong64 is "dbar 0x12", but it is still necessary to ensure consistency in the order of Store/Load [2]. LoweredAtomicStorezero{32,64} was removed because on loong64 the constant "0" uses the R0 register, and there is no performance difference between the implementations of LoweredAtomicStorezero{32,64} and LoweredAtomicStore{32,64}. goos: linux goarch: loong64 pkg: internal/runtime/atomic cpu: Loongson-3A5000-HV @ 2500.00MHz | bench.old | bench.new | | sec/op | sec/op vs base | AtomicStore64 19.61n ± 0% 13.61n ± 0% -30.60% (p=0.000 n=20) AtomicStore64-2 19.61n ± 0% 13.61n ± 0% -30.57% (p=0.000 n=20) AtomicStore64-4 19.62n ± 0% 13.61n ± 0% -30.63% (p=0.000 n=20) AtomicStore 19.61n ± 0% 13.61n ± 0% -30.60% (p=0.000 n=20) AtomicStore-2 19.62n ± 0% 13.61n ± 0% -30.63% (p=0.000 n=20) AtomicStore-4 19.62n ± 0% 13.62n ± 0% -30.58% (p=0.000 n=20) AtomicStore8 19.61n ± 0% 20.01n ± 0% +2.04% (p=0.000 n=20) AtomicStore8-2 19.62n ± 0% 20.02n ± 0% +2.01% (p=0.000 n=20) AtomicStore8-4 19.61n ± 0% 20.02n ± 0% +2.09% (p=0.000 n=20) geomean 19.61n 15.48n -21.08% goos: linux goarch: loong64 pkg: internal/runtime/atomic cpu: Loongson-3A6000 @ 2500.00MHz | bench.old | bench.new | | sec/op | sec/op vs base | AtomicStore64 18.03n ± 0% 12.81n ± 0% -28.93% (p=0.000 n=20) AtomicStore64-2 18.02n ± 0% 12.81n ± 0% -28.91% (p=0.000 n=20) AtomicStore64-4 18.01n ± 0% 12.81n ± 0% -28.87% (p=0.000 n=20) AtomicStore 18.02n ± 0% 12.81n ± 0% -28.91% (p=0.000 n=20) AtomicStore-2 18.01n ± 0% 12.81n ± 0% -28.87% (p=0.000 n=20) AtomicStore-4 18.01n ± 0% 12.81n ± 0% -28.87% (p=0.000 n=20) AtomicStore8 18.01n ± 0% 12.81n ± 0% -28.87% (p=0.000 n=20) AtomicStore8-2 18.01n ± 0% 12.81n ± 0% -28.87% (p=0.000 n=20) AtomicStore8-4 18.01n ± 0% 12.81n ± 0% -28.87% (p=0.000 n=20) geomean 18.01n 12.81n -28.89% [1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html [2]: https://gcc.gnu.org/git/?p=gcc.git;a=blob_plain;f=gcc/config/loongarch/sync.md Change-Id: I4ae5e8dd0e6f026129b6e503990a763ed40c6097 Reviewed-on: https://go-review.googlesource.com/c/go/+/581356 Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Qiqi Huang <huangqiqi@loongson.cn> Reviewed-by: Meidan Li <limeidan@loongson.cn> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
2024-09-13 18:47:56 +08:00
{name: "AtomicStore8Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
{name: "AtomicStore32Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
{name: "AtomicStore64Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns memory.
cmd/compile: improve atomic swap intrinsics on arm64 ARMv8.1 has added new instructions for atomic memory operations. This change builds on the previous change which added support for atomic add, 0a7ac93c27c9ade79fe0f66ae0bb81484c241ae5, to include similar support for atomic-compare-and-swap, atomic-swap, atomic-or, and atomic-and intrinsics. Since the new instructions are not guaranteed to be present, we guard their usages with a branch on a CPU feature. Peformance on an ARMv8.1 machine: name old time/op new time/op delta CompareAndSwap-16 37.9ns ±16% 24.1ns ± 4% -36.44% (p=0.000 n=10+9) CompareAndSwap64-16 38.6ns ±15% 24.1ns ± 3% -37.47% (p=0.000 n=10+10) name old time/op new time/op delta Swap-16 46.9ns ±32% 12.5ns ± 6% -73.40% (p=0.000 n=10+10) Swap64-16 53.4ns ± 1% 12.5ns ± 6% -76.56% (p=0.000 n=10+10) name old time/op new time/op delta Or8-16 8.81ns ± 0% 5.61ns ± 0% -36.32% (p=0.000 n=10+10) Or-16 7.21ns ± 0% 5.61ns ± 0% -22.19% (p=0.000 n=10+10) Or8Parallel-16 59.8ns ± 3% 12.5ns ± 2% -79.10% (p=0.000 n=10+10) OrParallel-16 51.7ns ± 3% 12.5ns ± 2% -75.84% (p=0.000 n=10+10) name old time/op new time/op delta And8-16 8.81ns ± 0% 5.61ns ± 0% -36.32% (p=0.000 n=10+10) And-16 7.21ns ± 0% 5.61ns ± 0% -22.19% (p=0.000 n=10+10) And8Parallel-16 59.1ns ± 6% 12.8ns ± 3% -78.33% (p=0.000 n=10+10) AndParallel-16 51.4ns ± 7% 12.8ns ± 3% -75.03% (p=0.000 n=10+10) Performance on an ARMv8.0 machine (no atomics instructions): name old time/op new time/op delta CompareAndSwap-16 61.3ns ± 0% 62.4ns ± 0% +1.70% (p=0.000 n=8+9) CompareAndSwap64-16 62.0ns ± 3% 61.3ns ± 2% ~ (p=0.093 n=10+10) name old time/op new time/op delta Swap-16 127ns ± 2% 131ns ± 2% +2.91% (p=0.001 n=10+10) Swap64-16 128ns ± 1% 131ns ± 2% +2.43% (p=0.001 n=10+10) name old time/op new time/op delta Or8-16 14.9ns ± 0% 15.3ns ± 0% +2.68% (p=0.000 n=10+10) Or-16 11.8ns ± 0% 12.3ns ± 0% +4.24% (p=0.000 n=10+10) Or8Parallel-16 137ns ± 1% 144ns ± 1% +4.97% (p=0.000 n=10+10) OrParallel-16 128ns ± 1% 136ns ± 1% +6.34% (p=0.000 n=10+10) name old time/op new time/op delta And8-16 14.9ns ± 0% 15.3ns ± 0% +2.68% (p=0.000 n=10+10) And-16 11.8ns ± 0% 12.3ns ± 0% +4.24% (p=0.000 n=10+10) And8Parallel-16 134ns ± 2% 141ns ± 1% +5.29% (p=0.000 n=10+10) AndParallel-16 125ns ± 2% 134ns ± 1% +7.10% (p=0.000 n=10+10) Fixes #39304 Change-Id: Idaca68701d4751650be6b4bedca3d57f51571712 Reviewed-on: https://go-review.googlesource.com/c/go/+/234217 Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Trust: fannie zhang <Fannie.Zhang@arm.com>
2020-11-04 16:18:23 +00:00
{name: "AtomicAdd32Variant", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicAdd64Variant", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
cmd/compile, internal/runtime/atomic: add Xchg8 for arm64 For #69735 Change-Id: I61a2e561684c538eea705e60c8ebda6be3ef31a7 GitHub-Last-Rev: 3c7f4ec845182d3ef1a007319d91027433163db3 GitHub-Pull-Request: golang/go#69751 Reviewed-on: https://go-review.googlesource.com/c/go/+/617595 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-10-07 20:53:01 +00:00
{name: "AtomicExchange8Variant", argLength: 3, typ: "(UInt8,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
cmd/compile: improve atomic swap intrinsics on arm64 ARMv8.1 has added new instructions for atomic memory operations. This change builds on the previous change which added support for atomic add, 0a7ac93c27c9ade79fe0f66ae0bb81484c241ae5, to include similar support for atomic-compare-and-swap, atomic-swap, atomic-or, and atomic-and intrinsics. Since the new instructions are not guaranteed to be present, we guard their usages with a branch on a CPU feature. Peformance on an ARMv8.1 machine: name old time/op new time/op delta CompareAndSwap-16 37.9ns ±16% 24.1ns ± 4% -36.44% (p=0.000 n=10+9) CompareAndSwap64-16 38.6ns ±15% 24.1ns ± 3% -37.47% (p=0.000 n=10+10) name old time/op new time/op delta Swap-16 46.9ns ±32% 12.5ns ± 6% -73.40% (p=0.000 n=10+10) Swap64-16 53.4ns ± 1% 12.5ns ± 6% -76.56% (p=0.000 n=10+10) name old time/op new time/op delta Or8-16 8.81ns ± 0% 5.61ns ± 0% -36.32% (p=0.000 n=10+10) Or-16 7.21ns ± 0% 5.61ns ± 0% -22.19% (p=0.000 n=10+10) Or8Parallel-16 59.8ns ± 3% 12.5ns ± 2% -79.10% (p=0.000 n=10+10) OrParallel-16 51.7ns ± 3% 12.5ns ± 2% -75.84% (p=0.000 n=10+10) name old time/op new time/op delta And8-16 8.81ns ± 0% 5.61ns ± 0% -36.32% (p=0.000 n=10+10) And-16 7.21ns ± 0% 5.61ns ± 0% -22.19% (p=0.000 n=10+10) And8Parallel-16 59.1ns ± 6% 12.8ns ± 3% -78.33% (p=0.000 n=10+10) AndParallel-16 51.4ns ± 7% 12.8ns ± 3% -75.03% (p=0.000 n=10+10) Performance on an ARMv8.0 machine (no atomics instructions): name old time/op new time/op delta CompareAndSwap-16 61.3ns ± 0% 62.4ns ± 0% +1.70% (p=0.000 n=8+9) CompareAndSwap64-16 62.0ns ± 3% 61.3ns ± 2% ~ (p=0.093 n=10+10) name old time/op new time/op delta Swap-16 127ns ± 2% 131ns ± 2% +2.91% (p=0.001 n=10+10) Swap64-16 128ns ± 1% 131ns ± 2% +2.43% (p=0.001 n=10+10) name old time/op new time/op delta Or8-16 14.9ns ± 0% 15.3ns ± 0% +2.68% (p=0.000 n=10+10) Or-16 11.8ns ± 0% 12.3ns ± 0% +4.24% (p=0.000 n=10+10) Or8Parallel-16 137ns ± 1% 144ns ± 1% +4.97% (p=0.000 n=10+10) OrParallel-16 128ns ± 1% 136ns ± 1% +6.34% (p=0.000 n=10+10) name old time/op new time/op delta And8-16 14.9ns ± 0% 15.3ns ± 0% +2.68% (p=0.000 n=10+10) And-16 11.8ns ± 0% 12.3ns ± 0% +4.24% (p=0.000 n=10+10) And8Parallel-16 134ns ± 2% 141ns ± 1% +5.29% (p=0.000 n=10+10) AndParallel-16 125ns ± 2% 134ns ± 1% +7.10% (p=0.000 n=10+10) Fixes #39304 Change-Id: Idaca68701d4751650be6b4bedca3d57f51571712 Reviewed-on: https://go-review.googlesource.com/c/go/+/234217 Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Trust: fannie zhang <Fannie.Zhang@arm.com>
2020-11-04 16:18:23 +00:00
{name: "AtomicExchange32Variant", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicExchange64Variant", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicCompareAndSwap32Variant", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Returns true if store happens and new memory.
{name: "AtomicCompareAndSwap64Variant", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Returns true if store happens and new memory.
cmd/compile: fix typing of atomic logical operations For atomic AND and OR operations on memory, we currently have two views of the op. One just does the operation on the memory and returns just a memory. The other does the operation on the memory and returns the old value (before having the logical operation done to it) and memory. Update #61395 These two type differently, and there's currently some confusion in our rules about which is which. Use different names for the two different flavors so we don't get them confused. Change-Id: I07b4542db672b2cee98169ac42b67db73c482093 Reviewed-on: https://go-review.googlesource.com/c/go/+/594976 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Nicolas Hillegeer <aktau@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Keith Randall <khr@google.com>
2024-06-25 14:01:09 -07:00
{name: "AtomicAnd64valueVariant", argLength: 3, typ: "(Uint64, Mem)", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicOr64valueVariant", argLength: 3, typ: "(Uint64, Mem)", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicAnd32valueVariant", argLength: 3, typ: "(Uint32, Mem)", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicOr32valueVariant", argLength: 3, typ: "(Uint32, Mem)", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicAnd8valueVariant", argLength: 3, typ: "(Uint8, Mem)", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicOr8valueVariant", argLength: 3, typ: "(Uint8, Mem)", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns old contents of *arg0 and new memory.
cmd/compile: improve atomic add intrinsics with ARMv8.1 new instruction ARMv8.1 has added new instruction (LDADDAL) for atomic memory operations. This CL improves existing atomic add intrinsics with the new instruction. Since the new instruction is only guaranteed to be present after ARMv8.1, we guard its usage with a conditional on CPU feature. Performance result on ARMv8.1 machine: name old time/op new time/op delta Xadd-224 1.05µs ± 6% 0.02µs ± 4% -98.06% (p=0.000 n=10+8) Xadd64-224 1.05µs ± 3% 0.02µs ±13% -98.10% (p=0.000 n=9+10) [Geo mean] 1.05µs 0.02µs -98.08% Performance result on ARMv8.0 machine: name old time/op new time/op delta Xadd-46 538ns ± 1% 541ns ± 1% +0.62% (p=0.000 n=9+9) Xadd64-46 505ns ± 1% 508ns ± 0% +0.48% (p=0.003 n=9+8) [Geo mean] 521ns 524ns +0.55% Change-Id: If4b5d8d0e2d6f84fe1492a4f5de0789910ad0ee9 Reviewed-on: https://go-review.googlesource.com/81877 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-11-03 02:05:28 +00:00
cmd/compile: intrinsify publicationBarrier This CL intrinsify asm call for publicationBarrier on ARM64. As for x86 we may completly removes any instructions due to strong memory oredering, but decided to leave it as is for compiler barrier. Benchmarks Go1 ARM64: name old time/op new time/op delta BinaryTree17-8 3.38s ± 1% 3.36s ± 1% ~ (p=0.095 n=5+5) Fannkuch11-8 2.93s ± 0% 2.84s ± 0% -3.26% (p=0.008 n=5+5) FmtFprintfEmpty-8 54.2ns ± 1% 54.0ns ± 1% ~ (p=0.690 n=5+5) FmtFprintfString-8 111ns ± 0% 109ns ± 0% -1.48% (p=0.008 n=5+5) FmtFprintfInt-8 140ns ± 0% 138ns ± 0% -1.10% (p=0.000 n=4+5) FmtFprintfIntInt-8 168ns ± 0% 169ns ± 0% +0.66% (p=0.008 n=5+5) FmtFprintfPrefixedInt-8 206ns ± 1% 195ns ± 0% -5.12% (p=0.008 n=5+5) FmtFprintfFloat-8 270ns ± 0% 269ns ± 0% -0.20% (p=0.024 n=5+5) FmtManyArgs-8 721ns ± 0% 733ns ± 0% +1.69% (p=0.008 n=5+5) GobDecode-8 9.75ms ± 1% 9.28ms ± 3% -4.88% (p=0.008 n=5+5) GobEncode-8 6.38ms ± 1% 6.34ms ± 1% ~ (p=0.095 n=5+5) Gzip-8 255ms ± 0% 254ms ± 0% -0.44% (p=0.008 n=5+5) Gunzip-8 41.8ms ± 1% 40.8ms ± 0% -2.40% (p=0.008 n=5+5) HTTPClientServer-8 65.1µs ± 1% 65.1µs ± 1% ~ (p=0.690 n=5+5) JSONEncode-8 11.7ms ± 0% 11.7ms ± 1% ~ (p=0.841 n=5+5) JSONDecode-8 60.2ms ± 1% 60.0ms ± 0% ~ (p=0.841 n=5+5) Mandelbrot200-8 5.85ms ± 0% 5.86ms ± 0% +0.22% (p=0.016 n=4+5) GoParse-8 4.38ms ± 0% 4.35ms ± 0% -0.60% (p=0.008 n=5+5) RegexpMatchEasy0_32-8 87.1ns ± 2% 88.3ns ± 1% ~ (p=0.151 n=5+5) RegexpMatchEasy0_1K-8 306ns ± 0% 306ns ± 1% ~ (p=0.143 n=5+5) RegexpMatchEasy1_32-8 86.3ns ± 0% 84.8ns ± 0% -1.81% (p=0.008 n=5+5) RegexpMatchEasy1_1K-8 491ns ± 2% 487ns ± 1% ~ (p=0.548 n=5+5) RegexpMatchMedium_32-8 7.50ns ± 0% 7.49ns ± 1% ~ (p=0.817 n=5+5) RegexpMatchMedium_1K-8 40.3µs ± 1% 39.9µs ± 0% -1.02% (p=0.008 n=5+5) RegexpMatchHard_32-8 2.10µs ± 1% 2.10µs ± 1% ~ (p=0.548 n=5+5) RegexpMatchHard_1K-8 62.4µs ± 1% 62.5µs ± 2% ~ (p=0.690 n=5+5) Revcomp-8 504ms ± 1% 502ms ± 1% ~ (p=0.095 n=5+5) Template-8 86.8ms ± 1% 86.5ms ± 1% ~ (p=0.222 n=5+5) TimeParse-8 330ns ± 0% 327ns ± 0% -0.84% (p=0.008 n=5+5) TimeFormat-8 383ns ± 1% 392ns ± 1% +2.42% (p=0.008 n=5+5) [Geo mean] 54.3µs 53.9µs -0.67% name old speed new speed delta GobDecode-8 78.7MB/s ± 1% 82.8MB/s ± 4% +5.16% (p=0.008 n=5+5) GobEncode-8 120MB/s ± 1% 121MB/s ± 1% ~ (p=0.095 n=5+5) Gzip-8 76.2MB/s ± 0% 76.5MB/s ± 0% +0.44% (p=0.008 n=5+5) Gunzip-8 464MB/s ± 1% 475MB/s ± 0% +2.45% (p=0.008 n=5+5) JSONEncode-8 166MB/s ± 0% 166MB/s ± 1% ~ (p=0.841 n=5+5) JSONDecode-8 32.2MB/s ± 1% 32.3MB/s ± 0% ~ (p=0.714 n=5+5) GoParse-8 13.2MB/s ± 0% 13.3MB/s ± 0% +0.59% (p=0.008 n=5+5) RegexpMatchEasy0_32-8 368MB/s ± 2% 362MB/s ± 1% ~ (p=0.151 n=5+5) RegexpMatchEasy0_1K-8 3.34GB/s ± 0% 3.34GB/s ± 1% ~ (p=0.127 n=5+5) RegexpMatchEasy1_32-8 371MB/s ± 0% 378MB/s ± 0% +1.84% (p=0.008 n=5+5) RegexpMatchEasy1_1K-8 2.09GB/s ± 2% 2.10GB/s ± 1% ~ (p=0.548 n=5+5) RegexpMatchMedium_32-8 133MB/s ± 0% 134MB/s ± 1% ~ (p=0.952 n=5+5) RegexpMatchMedium_1K-8 25.4MB/s ± 1% 25.6MB/s ± 0% +1.04% (p=0.008 n=5+5) RegexpMatchHard_32-8 15.3MB/s ± 1% 15.2MB/s ± 1% ~ (p=0.500 n=5+5) RegexpMatchHard_1K-8 16.4MB/s ± 1% 16.4MB/s ± 2% ~ (p=0.595 n=5+5) Revcomp-8 504MB/s ± 1% 506MB/s ± 1% ~ (p=0.095 n=5+5) Template-8 22.4MB/s ± 1% 22.4MB/s ± 1% ~ (p=0.206 n=5+5) [Geo mean] 120MB/s 121MB/s +0.71% Change-Id: I9cc10840b5c0d6bf759150f052c79f4c499e35e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/328290 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: David Chase <drchase@google.com>
2021-06-11 09:27:09 +00:00
// Publication barrier
{name: "PubBarrier", argLength: 1, hasSideEffects: true}, // Do data barrier. arg0=memory.
cmd/compile: experiment which clobbers all dead pointer fields The experiment "clobberdead" clobbers all pointer fields that the compiler thinks are dead, just before and after every safepoint. Useful for debugging the generation of live pointer bitmaps. Helped find the following issues: Update #15936 Update #16026 Update #16095 Update #18860 Change-Id: Id1d12f86845e3d93bae903d968b1eac61fc461f9 Reviewed-on: https://go-review.googlesource.com/23924 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-06-08 22:02:08 -07:00
// Clobber experiment op
{name: "Clobber", argLength: 0, typ: "Void", aux: "SymOff", symEffect: "None"}, // write an invalid pointer value to the given pointer slot of a stack variable
cmd/compile: add clobberdeadreg mode When -clobberdeadreg flag is set, the compiler inserts code that clobbers integer registers at call sites. This may be helpful for debugging register ABI. Only implemented on AMD64 for now. Change-Id: Ia203d3f891c30fd95d0103489056fe01d63a2899 Reviewed-on: https://go-review.googlesource.com/c/go/+/302809 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2021-03-17 19:15:38 -04:00
{name: "ClobberReg", argLength: 0, typ: "Void"}, // clobber a register
cmd/compile: add prefetch intrinsic support This CL provide new intrinsics to emit prefetch instructions for AMD64 and ARM64 platforms: Prefetch - prefetches data from memory address to cache; PrefetchStreamed - prefetches data from memory address, with a hint that this data is being streamed. This patch also provides prefetch calls pointed by RSC inside scanobject and greyobject of GC mark logic. Performance results provided by Michael: https://perf.golang.org/search?q=upload:20210901.9 Benchmark parameters: tree2 -heapsize=1000000000 -cpus=8 tree -n=18 parser peano Benchmarks AMD64 (Xeon - Cascade Lake): name old time/op new time/op delta Tree2-8 36.1ms ± 6% 33.4ms ± 5% -7.65% (p=0.000 n=9+9) Tree-8 326ms ± 1% 324ms ± 1% -0.44% (p=0.006 n=9+10) Parser-8 2.75s ± 1% 2.71s ± 1% -1.47% (p=0.008 n=5+5) Peano-8 63.1ms ± 1% 63.0ms ± 1% ~ (p=0.730 n=9+9) [Geo mean] 213ms 207ms -2.45% Benchmarks ARM64 (Kunpeng 920): name old time/op new time/op delta Tree2-8 50.3ms ± 8% 44.1ms ± 5% -12.24% (p=0.000 n=10+9) Tree-8 494ms ± 1% 493ms ± 1% ~ (p=0.684 n=10+10) Parser-8 3.99s ± 1% 3.93s ± 1% -1.37% (p=0.016 n=5+5) Peano-8 84.4ms ± 0% 84.1ms ± 1% ~ (p=0.068 n=8+10) [Geo mean] 302ms 291ms -3.67% Change-Id: I43e10bc2f9512dc49d7631dd8843a79036fa43d0 Reviewed-on: https://go-review.googlesource.com/c/go/+/328289 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-06-15 14:04:30 +00:00
// Prefetch instruction
{name: "PrefetchCache", argLength: 2, hasSideEffects: true}, // Do prefetch arg0 to cache. arg0=addr, arg1=memory.
{name: "PrefetchCacheStreamed", argLength: 2, hasSideEffects: true}, // Do non-temporal or streamed prefetch arg0 to cache. arg0=addr, arg1=memory.
[dev.simd] cmd/compile: adapters for simd This combines several CLs into a single patch of "glue" for the generated SIMD extensions. This glue includes GOEXPERIMENT checks that disable the creation of user-visible "simd" types and that disable the registration of "simd" intrinsics. The simd type checks were changed to work for either package "simd" or "internal/simd" so that moving that package won't be quite so fragile. cmd/compile, internal/simd: glue for adding SIMD extensions to Go cmd/compile: theft of Cherry's sample SIMD compilation Change-Id: Id44e2f4bafe74032c26de576a8691b6f7d977e01 Reviewed-on: https://go-review.googlesource.com/c/go/+/675598 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-03-31 10:45:23 +11:00
cmd/compile,internal/bytealg: add MemEq intrinsic for runtime.memequal Introduce a new MemEq SSA operation for runtime.memequal. The operation is initially implemented for arm64. The change adds opt rules (following existing rules for call to runtime.memequal), working with MemEq, and a later op version LoweredMemEq which may be lowered differently for more constant size cases in future (for other targets as well as for arm64). The new MemEq SSA operation does not have memory result, allowing cse of loads operations around it. Code size difference (for arm64 linux): Executable Old .text New .text Change ------------------------------------------------------- asm 1970420 1969668 -0.04% cgo 1741220 1740212 -0.06% compile 8956756 8959428 +0.03% cover 1879332 1878772 -0.03% link 2574116 2572660 -0.06% preprofile 867124 866820 -0.04% vet 2890404 2888596 -0.06% Change-Id: I6ab507929b861884d17d5818cfbd152cf7879751 Reviewed-on: https://go-review.googlesource.com/c/go/+/686655 Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Auto-Submit: Keith Randall <khr@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2025-07-05 23:16:36 +03:00
// Helper instruction which is semantically equivalent to calling runtime.memequal, but some targets may prefer to custom lower it later, e.g. for specific constant sizes.
{name: "MemEq", argLength: 4, commutative: true, typ: "Bool"}, // arg0=ptr0, arg1=ptr1, arg2=size, arg3=memory.
[dev.simd] cmd/compile: remove Add32x4 generic op This was for my early prototype, not used in real code. Change-Id: I154a027ae2335d12e44625c0e3ce42a7b7d84976 Reviewed-on: https://go-review.googlesource.com/c/go/+/705335 Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-09-18 16:08:06 -04:00
// SIMD
[dev.simd] cmd/compile, simd: remove mask load and stores We have convert mask to bits already, the API of mask load and stores are inconsistent with them, also mask load and stores could just be hidden behind peepholes. So this CL removes them, the next CL will add the peephole for them. Change-Id: Ifa7d23fb52bb0efd1785935ead4d703927f16d2b Reviewed-on: https://go-review.googlesource.com/c/go/+/710915 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-10-10 17:42:59 +00:00
{name: "ZeroSIMD", argLength: 0}, // zero value of a vector
[dev.simd] cmd/compile: add register-to-mask moves, other simd glue This includes code generated by simdgen CL 689955, here because of git-facilitated pilot error (the generated file should have been in the next CL but that is related to this one, so, oh well). Change-Id: Ibfea3f1cd93ca9cd12970edf15a013471677a6ba Reviewed-on: https://go-review.googlesource.com/c/go/+/689936 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-07-23 13:47:08 -04:00
// Convert integers to masks
{name: "Cvt16toMask8x16", argLength: 1}, // arg0 = integer mask value
{name: "Cvt32toMask8x32", argLength: 1}, // arg0 = integer mask value
{name: "Cvt64toMask8x64", argLength: 1}, // arg0 = integer mask value
{name: "Cvt8toMask16x8", argLength: 1}, // arg0 = integer mask value
{name: "Cvt16toMask16x16", argLength: 1}, // arg0 = integer mask value
{name: "Cvt32toMask16x32", argLength: 1}, // arg0 = integer mask value
{name: "Cvt8toMask32x4", argLength: 1}, // arg0 = integer mask value
{name: "Cvt8toMask32x8", argLength: 1}, // arg0 = integer mask value
{name: "Cvt16toMask32x16", argLength: 1}, // arg0 = integer mask value
{name: "Cvt8toMask64x2", argLength: 1}, // arg0 = integer mask value
{name: "Cvt8toMask64x4", argLength: 1}, // arg0 = integer mask value
{name: "Cvt8toMask64x8", argLength: 1}, // arg0 = integer mask value
[dev.simd] cmd/compile, simd: add value conversion ToBits for mask This CL is generated by CL 693598. Change-Id: I949d3b3b4e5670cb30f0fb9dc779f7359409b54c Reviewed-on: https://go-review.googlesource.com/c/go/+/693755 Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-08-06 19:03:52 +00:00
// Convert masks to integers
{name: "CvtMask8x16to16", argLength: 1}, // arg0 = mask
{name: "CvtMask8x32to32", argLength: 1}, // arg0 = mask
{name: "CvtMask8x64to64", argLength: 1}, // arg0 = mask
{name: "CvtMask16x8to8", argLength: 1}, // arg0 = mask
{name: "CvtMask16x16to16", argLength: 1}, // arg0 = mask
{name: "CvtMask16x32to32", argLength: 1}, // arg0 = mask
{name: "CvtMask32x4to8", argLength: 1}, // arg0 = mask
{name: "CvtMask32x8to8", argLength: 1}, // arg0 = mask
{name: "CvtMask32x16to16", argLength: 1}, // arg0 = mask
{name: "CvtMask64x2to8", argLength: 1}, // arg0 = mask
{name: "CvtMask64x4to8", argLength: 1}, // arg0 = mask
{name: "CvtMask64x8to8", argLength: 1}, // arg0 = mask
[dev.simd] cmd/compile, simd: add VPTEST Change-Id: Ia5103100eca2747fd10917ee2f32e3403e68e844 Reviewed-on: https://go-review.googlesource.com/c/go/+/702175 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Junyang Shao <shaojunyang@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Rob Lee <leerob7691@gmail.com>
2025-09-09 16:29:38 +00:00
// Returns true if arg0 is all zero.
{name: "IsZeroVec", argLength: 1},
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
}
cmd/compile, runtime: use PC of deferreturn for panic transfer this removes the old conditional-on-register-value handshake from the deferproc/deferprocstack logic. The "line" for the recovery-exit frame itself (not the defers that it runs) is the closing brace of the function. Reduces code size slightly (e.g. go command is 0.2% smaller) Sample output showing effect of this change, also what sort of code it requires to observe the effect: ``` package main import "os" func main() { g(len(os.Args) - 1) // stack[0] } var gi int var pi *int = &gi //go:noinline func g(i int) { switch i { case 0: defer func() { println("g0", i) q() // stack[2] if i == 0 }() for j := *pi; j < 1; j++ { defer func() { println("recover0", recover().(string)) }() } default: for j := *pi; j < 1; j++ { defer func() { println("g1", i) q() // stack[2] if i == 1 }() } defer func() { println("recover1", recover().(string)) }() } p() } // stack[1] (deferreturn) //go:noinline func p() { panic("p()") } //go:noinline func q() { panic("q()") // stack[3] } /* Sample output for "./foo foo": recover1 p() g1 1 panic: q() goroutine 1 [running]: main.q() .../main.go:46 +0x2c main.g.func3() .../main.go:29 +0x48 main.g(0x1?) .../main.go:37 +0x68 main.main() .../main.go:6 +0x28 */ ``` Change-Id: Ie39ea62ecc244213500380ea06d44024cadc2317 Reviewed-on: https://go-review.googlesource.com/c/go/+/650795 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-19 16:47:31 -05:00
// kind controls successors implicit exit
// ------------------------------------------------------------
// Exit [return mem] [] yes
// Ret [return mem] [] yes
// RetJmp [return mem] [] yes
// Plain [] [next]
// If [boolean Value] [then, else]
// First [] [always, never]
// Defer [mem] [nopanic, recovery] (control opcode should be OpStaticCall to runtime.defer*)
// JumpTable [integer Value] [succ1,succ2,..]
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
var genericBlocks = []blockData{
cmd/compile: implement jump tables Performance is kind of hard to exactly quantify. One big difference between jump tables and the old binary search scheme is that there's only 1 branch statement instead of O(n) of them. That can be both a blessing and a curse, and can make evaluating jump tables very hard to do. The single branch can become a choke point for the hardware branch predictor. A branch table jump must fit all of its state in a single branch predictor entry (technically, a branch target predictor entry). With binary search that predictor state can be spread among lots of entries. In cases where the case selection is repetitive and thus predictable, binary search can perform better. The big win for a jump table is that it doesn't consume so much of the branch predictor's resources. But that benefit is essentially never observed in microbenchmarks, because the branch predictor can easily keep state for all the binary search branches in a microbenchmark. So that benefit is really hard to measure. So predictable switch microbenchmarks are ~useless - they will almost always favor the binary search scheme. Fully unpredictable switch microbenchmarks are better, as they aren't lying to us quite so much. In a perfectly unpredictable situation, a jump table will expect to incur 1-1/N branch mispredicts, where a binary search would incur lg(N)/2 of them. That makes the crossover point at about N=4. But of course switches in real programs are seldom fully unpredictable, so we'll use a higher crossover point. Beyond the branch predictor, jump tables tend to execute more instructions per switch but have no additional instructions per case, which also argues for a larger crossover. As far as code size goes, with this CL cmd/go has a slightly smaller code segment and a slightly larger overall size (from the jump tables themselves which live in the data segment). This is a case where some FDO (feedback-directed optimization) would be really nice to have. #28262 Some large-program benchmarks might help make the case for this CL. Especially if we can turn on branch mispredict counters so we can see how much using jump tables can free up branch prediction resources that can be gainfully used elsewhere in the program. name old time/op new time/op delta Switch8Predictable 1.89ns ± 2% 1.27ns ± 3% -32.58% (p=0.000 n=9+10) Switch8Unpredictable 9.33ns ± 1% 7.50ns ± 1% -19.60% (p=0.000 n=10+9) Switch32Predictable 2.20ns ± 2% 1.64ns ± 1% -25.39% (p=0.000 n=10+9) Switch32Unpredictable 10.0ns ± 2% 7.6ns ± 2% -24.04% (p=0.000 n=10+10) Fixes #5496 Update #34381 Change-Id: I3ff56011d02be53f605ca5fd3fb96b905517c34f Reviewed-on: https://go-review.googlesource.com/c/go/+/357330 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@google.com>
2021-10-04 12:17:46 -07:00
{name: "Plain"}, // a single successor
{name: "If", controls: 1}, // if Controls[0] goto Succs[0] else goto Succs[1]
cmd/compile, runtime: use PC of deferreturn for panic transfer this removes the old conditional-on-register-value handshake from the deferproc/deferprocstack logic. The "line" for the recovery-exit frame itself (not the defers that it runs) is the closing brace of the function. Reduces code size slightly (e.g. go command is 0.2% smaller) Sample output showing effect of this change, also what sort of code it requires to observe the effect: ``` package main import "os" func main() { g(len(os.Args) - 1) // stack[0] } var gi int var pi *int = &gi //go:noinline func g(i int) { switch i { case 0: defer func() { println("g0", i) q() // stack[2] if i == 0 }() for j := *pi; j < 1; j++ { defer func() { println("recover0", recover().(string)) }() } default: for j := *pi; j < 1; j++ { defer func() { println("g1", i) q() // stack[2] if i == 1 }() } defer func() { println("recover1", recover().(string)) }() } p() } // stack[1] (deferreturn) //go:noinline func p() { panic("p()") } //go:noinline func q() { panic("q()") // stack[3] } /* Sample output for "./foo foo": recover1 p() g1 1 panic: q() goroutine 1 [running]: main.q() .../main.go:46 +0x2c main.g.func3() .../main.go:29 +0x48 main.g(0x1?) .../main.go:37 +0x68 main.main() .../main.go:6 +0x28 */ ``` Change-Id: Ie39ea62ecc244213500380ea06d44024cadc2317 Reviewed-on: https://go-review.googlesource.com/c/go/+/650795 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-02-19 16:47:31 -05:00
{name: "Defer", controls: 1}, // Succs[0]=defer queued, Succs[1]=defer recovery branch (jmp performed by runtime). Controls[0] is call op (of memory type).
cmd/compile: implement jump tables Performance is kind of hard to exactly quantify. One big difference between jump tables and the old binary search scheme is that there's only 1 branch statement instead of O(n) of them. That can be both a blessing and a curse, and can make evaluating jump tables very hard to do. The single branch can become a choke point for the hardware branch predictor. A branch table jump must fit all of its state in a single branch predictor entry (technically, a branch target predictor entry). With binary search that predictor state can be spread among lots of entries. In cases where the case selection is repetitive and thus predictable, binary search can perform better. The big win for a jump table is that it doesn't consume so much of the branch predictor's resources. But that benefit is essentially never observed in microbenchmarks, because the branch predictor can easily keep state for all the binary search branches in a microbenchmark. So that benefit is really hard to measure. So predictable switch microbenchmarks are ~useless - they will almost always favor the binary search scheme. Fully unpredictable switch microbenchmarks are better, as they aren't lying to us quite so much. In a perfectly unpredictable situation, a jump table will expect to incur 1-1/N branch mispredicts, where a binary search would incur lg(N)/2 of them. That makes the crossover point at about N=4. But of course switches in real programs are seldom fully unpredictable, so we'll use a higher crossover point. Beyond the branch predictor, jump tables tend to execute more instructions per switch but have no additional instructions per case, which also argues for a larger crossover. As far as code size goes, with this CL cmd/go has a slightly smaller code segment and a slightly larger overall size (from the jump tables themselves which live in the data segment). This is a case where some FDO (feedback-directed optimization) would be really nice to have. #28262 Some large-program benchmarks might help make the case for this CL. Especially if we can turn on branch mispredict counters so we can see how much using jump tables can free up branch prediction resources that can be gainfully used elsewhere in the program. name old time/op new time/op delta Switch8Predictable 1.89ns ± 2% 1.27ns ± 3% -32.58% (p=0.000 n=9+10) Switch8Unpredictable 9.33ns ± 1% 7.50ns ± 1% -19.60% (p=0.000 n=10+9) Switch32Predictable 2.20ns ± 2% 1.64ns ± 1% -25.39% (p=0.000 n=10+9) Switch32Unpredictable 10.0ns ± 2% 7.6ns ± 2% -24.04% (p=0.000 n=10+10) Fixes #5496 Update #34381 Change-Id: I3ff56011d02be53f605ca5fd3fb96b905517c34f Reviewed-on: https://go-review.googlesource.com/c/go/+/357330 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@google.com>
2021-10-04 12:17:46 -07:00
{name: "Ret", controls: 1}, // no successors, Controls[0] value is memory result
{name: "RetJmp", controls: 1}, // no successors, Controls[0] value is a tail call
{name: "Exit", controls: 1}, // no successors, Controls[0] value generates a panic
{name: "JumpTable", controls: 1}, // multiple successors, the integer Controls[0] selects which one
[dev.ssa] cmd/compile/internal/ssa: simplify how exit blocks are used Move to implicit (mostly) instead of explicit exit blocks. RET and RETJMP have no outgoing edges - they implicitly exit. CALL only has one outgoing edge, as its exception edge is implicit as well. Exit blocks are only used for unconditionally panicking code, like the failed branches of nil and bounds checks. There may now be more than one exit block. No merges happen at exit blocks. The only downside is it is harder to find all the places code can exit the method. See the reverse dominator code for an example. Change-Id: I42e2fd809a4bf81301ab993e29ad9f203ce48eb0 Reviewed-on: https://go-review.googlesource.com/14462 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-09-09 18:03:41 -07:00
cmd/compile: remove BlockDead state It is unused, remove the clutter. Change-Id: I51a44326b125ef79241459c463441f76a289cc08 Reviewed-on: https://go-review.googlesource.com/22586 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-04-28 15:04:10 -07:00
// transient block state used for dead code removal
[dev.ssa] cmd/compile/internal/ssa: simplify how exit blocks are used Move to implicit (mostly) instead of explicit exit blocks. RET and RETJMP have no outgoing edges - they implicitly exit. CALL only has one outgoing edge, as its exception edge is implicit as well. Exit blocks are only used for unconditionally panicking code, like the failed branches of nil and bounds checks. There may now be more than one exit block. No merges happen at exit blocks. The only downside is it is harder to find all the places code can exit the method. See the reverse dominator code for an example. Change-Id: I42e2fd809a4bf81301ab993e29ad9f203ce48eb0 Reviewed-on: https://go-review.googlesource.com/14462 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2015-09-09 18:03:41 -07:00
{name: "First"}, // 2 successors, always takes the first one (second is dead)
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
}
func init() {
[dev.simd] cmd/compile: adapters for simd This combines several CLs into a single patch of "glue" for the generated SIMD extensions. This glue includes GOEXPERIMENT checks that disable the creation of user-visible "simd" types and that disable the registration of "simd" intrinsics. The simd type checks were changed to work for either package "simd" or "internal/simd" so that moving that package won't be quite so fragile. cmd/compile, internal/simd: glue for adding SIMD extensions to Go cmd/compile: theft of Cherry's sample SIMD compilation Change-Id: Id44e2f4bafe74032c26de576a8691b6f7d977e01 Reviewed-on: https://go-review.googlesource.com/c/go/+/675598 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-03-31 10:45:23 +11:00
genericOps = append(genericOps, simdGenericOps()...)
cmd/compile: remove amd64 code from package gc and the core gen tool Parts of the SSA compiler in package gc contain amd64-specific code, most notably Prog generation. Move this code into package amd64, so that other architectures can be added more easily. In package gc, this change is just moving code. There are no functional changes or even any larger structural changes beyond changing function names (mostly for export). In the cmd/compile/internal/ssa/gen tool, more information is included in arch to remove the AMD64-specific behavior in the main portion of the tool. The generated opGen.go is identical. Change-Id: I8eb37c6e6df6de1b65fa7dab6f3bc32c29daf643 Reviewed-on: https://go-review.googlesource.com/20609 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-12 14:07:40 -08:00
archs = append(archs, arch{
cmd/compile: start on ARM port Start working on arm port. Gets close to correct code for fibonacci: func fib(n int) int { if n < 2 { return n } return fib(n-1) + fib(n-2) } Still a lot to do, but this is a good starting point. Cleaned up some arch-specific dependencies in regalloc. Change-Id: I4301c6c31a8402168e50dcfee8bcf7aee73ea9d5 Reviewed-on: https://go-review.googlesource.com/21000 Reviewed-by: David Chase <drchase@google.com>
2016-03-21 22:57:26 -07:00
name: "generic",
ops: genericOps,
blocks: genericBlocks,
generic: true,
cmd/compile: remove amd64 code from package gc and the core gen tool Parts of the SSA compiler in package gc contain amd64-specific code, most notably Prog generation. Move this code into package amd64, so that other architectures can be added more easily. In package gc, this change is just moving code. There are no functional changes or even any larger structural changes beyond changing function names (mostly for export). In the cmd/compile/internal/ssa/gen tool, more information is included in arch to remove the AMD64-specific behavior in the main portion of the tool. The generated opGen.go is identical. Change-Id: I8eb37c6e6df6de1b65fa7dab6f3bc32c29daf643 Reviewed-on: https://go-review.googlesource.com/20609 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-12 14:07:40 -08:00
})
[dev.ssa] cmd/compile/internal/ssa: autogenerate opcodes Revamp autogeneration. Get rid of gogenerate commands, they are more trouble than they are worth. (If the code won't compile, gogenerate doesn't work.) Generate opcode enums & tables. This means we only have to specify opcodes in one place instead of two. Add arch prefixes to opcodes so they will be globally unique. Change-Id: I175d0a89b701b2377bbe699f3756731b7c9f5a9f Reviewed-on: https://go-review.googlesource.com/10812 Reviewed-by: Alan Donovan <adonovan@google.com>
2015-06-06 16:03:33 -07:00
}
Reference in a new issue Copy permalink