The X3 (aka GP) register will potentially be loaded with the __global_pointer$ symbol
during program start up (usually by the dynamic linker). As such, non-Go code may depend
on the contents of GP and calculate offsets based on it, including code called via cgo
and signal handlers installed by non-Go code. As such, stop using the X3 register so
that there are fewer issues interacting between Go and non-Go code.
While here remove the X4 (TP) name from the assembler such that any references must
use the 'TP' name. This should reduce the likelihood of accidental use (like we do
for the 'g' register). The same applies for X3 (GP) when the -shared flag is given.
Updates #47100
Change-Id: I72e82b5ca3f80c46a781781345ca0432a4111b74
Reviewed-on: https://go-review.googlesource.com/c/go/+/351859
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
When doing a tail call the link register is live as the callee
will directly return to the caller (of the function that does the
tail call). Don't allocate or clobber the link register.
Fixes#49032.
Change-Id: I2d60f2354e5b6c14aa285c8983a9786687b90223
Reviewed-on: https://go-review.googlesource.com/c/go/+/358435
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
For certain type of method wrappers we used to generate a tail
call. That was disabled in CL 307234 when register ABI is used,
because with the current IR it was difficult to generate a tail
call with the arguments in the right places. The problem was that
the IR does not contain a CALL-like node with arguments; instead,
it contains an OAS node that adjusts the receiver, than an
OTAILCALL node that just contains the target, but no argument
(with the assumption that the OAS node will put the adjusted
receiver in the right place). With register ABI, putting
arguments in registers are done in SSA. The assignment (OAS)
doesn't put the receiver in register.
This CL changes the IR of a tail call to take an actual OCALL
node. Specifically, a tail call is represented as
OTAILCALL (OCALL target args...)
This way, the call target and args are connected through the OCALL
node. So the call can be analyzed in SSA and the args can be passed
in the right places.
(Alternatively, we could have OTAILCALL node directly take the
target and the args, without the OCALL node. Using an OCALL node is
convenient as there are existing code that processes OCALL nodes
which do not need to be changed. Also, a tail call is similar to
ORETURN (OCALL target args...), except it doesn't preserve the
frame. I did the former but I'm open to change.)
The SSA representation is similar. Previously, the IR lowers to
a Store the receiver then a BlockRetJmp which jumps to the target
(without putting the arg in register). Now we use a TailCall op,
which takes the target and the args. The call expansion pass and
the register allocator handles TailCall pretty much like a
StaticCall, and it will do the right ABI analysis and put the args
in the right places. (Args other than the receiver are already in
the right places. For register args it generates no code for them.
For stack args currently it generates a self copy. I'll work on
optimize that out.) BlockRetJmp is still used, signaling it is a
tail call. The actual call is made in the TailCall op so
BlockRetJmp generates no code (we could use BlockExit if we like).
This slightly reduces binary size:
old new
cmd/go 14003088 13953936
cmd/link 6275552 6271456
Change-Id: I2d16d8d419fe1f17554916d317427383e17e27f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/350145
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
Also, add the FABSS and FABSD pseudo instructions to the assembler.
The compiler could use FSGNJX[SD] directly but there doesn't seem
to be much advantage to doing so and the pseudo instructions are
easier to understand.
Change-Id: Ie8825b8aa8773c69cc4f07a32ef04abf4061d80d
Reviewed-on: https://go-review.googlesource.com/c/go/+/348989
Trust: Michael Munday <mike.munday@lowrisc.org>
Run-TryBot: Michael Munday <mike.munday@lowrisc.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
Add support to the assembler for F[N]M{ADD,SUB}[SD] instructions.
Argument order is:
OP RS1, RS2, RS3, RD
Also, add support for the FMA intrinsic to the compiler. Automatic
FMA matching is left to a future CL.
Change-Id: I47166c7393b2ab6bfc2e42aa8c1a8997c3a071b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/293030
Trust: Michael Munday <mike.munday@lowrisc.org>
Run-TryBot: Michael Munday <mike.munday@lowrisc.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
According to RISCV instruction set manual v2.2 Sec 6.1
MULHU followed by MUL will be fused into one multiply by microarchitecture
name old time/op new time/op delta
MulUintptr/small 11.2ns ±24% 9.2ns ± 0% -17.54% (p=0.000 n=10+9)
MulUintptr/large 15.9ns ± 0% 10.9ns ± 0% -31.55% (p=0.000 n=8+8)
Change-Id: I3d152218f83948cbc5c576bda29dc86e9b4206ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/338753
Trust: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
CL 294430 made packages in std and cmd modules use Go 1.17 gofmt format,
adding //go:build lines. This change applies the same formatting to some
more packages that 'go fmt' missed (e.g., syscall/js, runtime/msan), and
everything else that is easy and safe to modify in bulk.
Consider the top-level test directory, testdata, and vendor directories
out of scope, since there are many files that don't follow strict gofmt
formatting, often for intentional and legitimate reasons (testing gofmt
itself, invalid Go programs that shouldn't crash the compiler, etc.).
That makes it easy and safe to gofmt -w the .go files that are found
with gofmt -l with aforementioned directories filtered out:
$ gofmt -l . 2>/dev/null | \
grep -v '^test/' | \
grep -v '/testdata/' | \
grep -v '/vendor/' | wc -l
51
None of the 51 files are generated. After this change, the same command
prints 0.
For #41184.
Change-Id: Ia96ee2a0f998d6a167d4473bcad17ad09bc1d86e
Reviewed-on: https://go-review.googlesource.com/c/go/+/341009
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Most platforms only use a single MOV const operand - remove the MOV{B,H,W}const
operands from riscv64 and consistently use MOVDconst instead. The implementation
of all four is the same and there is no benefit gained from having multiple const
operands (in fact it requires a lot more rewrite rules).
Change-Id: I0ba7d7554e371a1de762ef5f3745e9c0c30d41ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/302610
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Munday <mike.munday@lowrisc.org>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
The 32 bit versions are easily implement with a single instruction, while the
8 bit versions require a bit more effort but use the same atomic instructions
via rewrite rules.
Change-Id: I42e8d457b239c8f75e39a8e282fc88c1bb292a99
Reviewed-on: https://go-review.googlesource.com/c/go/+/268098
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Implement runtime.duffzero and runtime.duffcopy for riscv64.
Use obj.ADUFFZERO/obj.ADUFFCOPY for medium size, word aligned
zeroing/moving.
Change-Id: I42ec622055630c94cb77e286d8d33dbe7c9f846c
Reviewed-on: https://go-review.googlesource.com/c/go/+/237797
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add additional rules to eliminate unnecessary sign/zero extension for riscv64.
Also where possible, replace an extension following a load with a different typed
load. This removes almost another 8,000 instructions from the go binary.
Of particular note, change Eq16/Eq8/Neq16/Neq8 to zero extend each value before
subtraction, rather than zero extending after subtraction. While this appears to
double the number of zero extensions, it often lets us completely eliminate them
as the load can already be performed in a properly typed manner.
As an example, prior to this change runtime.memequal16 was:
0000000000013028 <runtime.memequal16>:
13028: 00813183 ld gp,8(sp)
1302c: 00019183 lh gp,0(gp)
13030: 01013283 ld t0,16(sp)
13034: 00029283 lh t0,0(t0)
13038: 405181b3 sub gp,gp,t0
1303c: 03019193 slli gp,gp,0x30
13040: 0301d193 srli gp,gp,0x30
13044: 0011b193 seqz gp,gp
13048: 00310c23 sb gp,24(sp)
1304c: 00008067 ret
Whereas it now becomes:
0000000000012fa8 <runtime.memequal16>:
12fa8: 00813183 ld gp,8(sp)
12fac: 0001d183 lhu gp,0(gp)
12fb0: 01013283 ld t0,16(sp)
12fb4: 0002d283 lhu t0,0(t0)
12fb8: 405181b3 sub gp,gp,t0
12fbc: 0011b193 seqz gp,gp
12fc0: 00310c23 sb gp,24(sp)
12fc4: 00008067 ret
Change-Id: I16321feb18381241cab121c0097a126104c56c2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/264659
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Rather than handling sign and zero extension via rules, defer to the assembler
and use MOV pseudo-instructions. The instruction can also be omitted where the
type and size is already correct. This change results in more than 6,000
instructions being removed from the go binary (in part due to omitted
instructions, in part due to MOVBU having a more efficient implementation in
the assembler than what is used in the current ZeroExt8to{16,32,64} rules).
This will also allow for further rewriting to remove redundant sign/zero
extension.
Change-Id: I05e42fd9f09f40a69948be7de772cce8946c8744
Reviewed-on: https://go-review.googlesource.com/c/go/+/264658
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The original riscv64 port used the thread pointer (TP aka X4) register for
the g pointer, however this register is also used when TLS support is
required, resulting in a conflict (for example, when a signal is received
we have no way of readily knowing if X4 contains a pointer to the TCB or
a pointer to a g).
In order to support cgo, free up the X4 register by moving g to X27.
This unfortunately means that the X4 register is unused in non-cgo mode,
however the alternative is to not support cgo on this platform.
Update #36641
Change-Id: Idcaf3e8ccbe42972a1b8943aeefde7149d9c960a
Reviewed-on: https://go-review.googlesource.com/c/go/+/263477
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This is prerequisite to moving call expansion later into SSA,
and probably a good idea anyway. Passes tests.
This is the first minimal CL that does a 1-for-1 substitution
of *ssa.AuxCall for *obj.LSym. Next step (next CL) is to make
this change for all calls so that additional information can
be stored in AuxCall.
Change-Id: Ia3a7715648fd9fb1a176850767a726e6f5b959eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/237680
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This makes the intent clearer, allows for another ellipsis and will aid
in future rewriting. While here, document boolean loads to explain register
contents.
Change-Id: I933db2813826d88819366191fbbea8fcee5e4dda
Reviewed-on: https://go-review.googlesource.com/c/go/+/230120
Reviewed-by: Keith Randall <khr@golang.org>
PanicBounds and PanicExtend are lowered to runtime calls (with a
non-Go ABI), but are not currently marked as calls. Since liveness
analysis only emits stack maps at calls in the runtime, this means
these panic call sites in the runtime won't get a stack map. These
almost immediately turn into throws in the runtime, but there's still
a chance they'll try to grow the stack first, which would lead to a
different panic.
To fix this, mark these operations as calls.
Outside the runtime, we currently emit stack maps for everything that
isn't an unsafe-point, so these panic calls get stack maps by default.
However, we're about to move to emitting stack maps only at call
sites, at which point this will start to matter outside the runtime as
well.
I confirmed that this has no effect on anything but PCDATA/FUNCDATA in
runtime and net/http.
For #36365.
Change-Id: Ic5bb463fd152cc320c815dc04cf62005261ae169
Reviewed-on: https://go-review.googlesource.com/c/go/+/230539
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Implement multi-control branches for riscv64, switching to using the BNEZ
pseudo-instruction when rewriting conditionals. This will allow for further
branch optimisations to later be performed via rewrites.
Change-Id: I7f2c69f3c77494b403f26058c6bc8432d8070ad0
Reviewed-on: https://go-review.googlesource.com/c/go/+/226399
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
Where possible, fold constants into versions of instructions that take
an immediate. This avoids the need to allocate a register and load the
immediate into it.
Change-Id: If911ca41235e218490679aed2ce5f48bf807a2b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/222639
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Also rewrite subtraction of zero to NEG/NEGW.
Change-Id: I216e286d1860055f2a07fe2f772cd50f366ea097
Reviewed-on: https://go-review.googlesource.com/c/go/+/221691
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This allows for zero stores to be performed using the zero register, rather
than loading a separate register with zero.
Change-Id: Ic81d8dbcdacbb2ca2c3f77682ff5ad7cdc33d18d
Reviewed-on: https://go-review.googlesource.com/c/go/+/221684
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
SignExt32to64 can be implemented with a single ADDIW instruction, rather than
the two shifts that are in use currently.
Change-Id: Ie1bbaef4018f1ba5162773fc64fa5a887457cfc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/220922
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Use SUBW to perform a 32-bit subtraction, rather than zero extending from
32 to 64 bits. This reduces Eq32 and Neq32 to two instructions, rather than
the four instructions required previously.
Change-Id: Ib2798324881e9db842c864e91a0c1b1e48c4b67b
Reviewed-on: https://go-review.googlesource.com/c/go/+/220921
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Duplicate the register definitions and names to avoid importing the
cmd/internal/obj/riscv64 package. This makes it possible to build compiler rules
with a stable Go tool chain.
Fixes#36663
Change-Id: I09116a97bb037ca1bc00073306a82bb88862b1e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/216518
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Based on riscv-go port.
Updates #27532
Change-Id: Ia329daa243db63ff334053b8807ea96b97ce3acf
Reviewed-on: https://go-review.googlesource.com/c/go/+/204631
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
