This also required a "always use operation with least
OverrideBase" filter in choosing the machine instructions.
The order of generated HW operations is slightly
modified because the Float version of GetElem
appears earlier in the sorted operations list,
though it is not chosen to generate the HW Op.
Change-Id: I95fa67afca9c8b6f4f18941fdcaf69afdad8055b
Reviewed-on: https://go-review.googlesource.com/c/go/+/696375
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
this also removes AVX512 versions of the operations
that would use the same names, but not run on AVX2-only
includes files generated by simdgen CL 692355
Change-Id: Iff29042245b7688133fed49a03e681e85235b8a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/692335
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Generated by simdgen CL 693599
This turned out to require some additional work in
other places, including filling in missing
methods (use OverwriteBase to get FP versions).
Also includes a test.
Change-Id: I2efe8967837834745f9cae661d4d4dcbb5390b6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/693758
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
This CL fixes some errors in prog generation for imm operations, please
see the changes in ssa.go for details.
This CL also implements the jump table for non-const immediate arg. The
current implementation exhaust 0-255, the bound-checked version will be
in the next CL.
This CL is partially generated by CL 694375.
Change-Id: I75fe9900430b4fca5b39b0c0958a13b20b1104b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/694395
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
- Absolute -> Abs
- ApproximateReciprocal -> Reciprocal
- Other derived apis also changed.
- Round -> RoundToEven
- Other derived apis also changed.
- Drop DotProdBroadcast
- Fused(Mul|Add)(Mul|Add)? -> remove the "Fused"
- MulEvenWiden -> remove 64bit
- MulLow -> Mul, add unit
- PairDotProd -> DotProdPairs
- make AddDotProdPairs machine ops only - peepholes will be in another
CL at dev.simd.
- PopCount -> OnesCount
- Saturated* -> *Saturated
- Fix (Add|Sub)Saturated uint mappings.
- UnsignedSignedQuadDotProdAccumulate -> AddDotProdQuadruple
- The "DotProdQuadruple" instruction does not exist, so no peepholes for
this.
This CL is generated by CL 694095.
Change-Id: If4110cc04ab96240cf56f2348d35ed2a719687de
Reviewed-on: https://go-review.googlesource.com/c/go/+/694115
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 693336.
Change-Id: Ic1712d49fcad0544fa3c19b0249d8bc65b347104
Reviewed-on: https://go-review.googlesource.com/c/go/+/693375
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 693335.
Change-Id: Ie9adda526573f979ec7e4f535033ba29236cc5cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/693355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Generated by arch/internal/simdgen CL 693175
These methods are not public because of simdgen-induced name/signature
issues, and because their addition was motivated by the need for
emulation tools.
The specific name signature problems are:
1) one set of instructions has the "Masked" suffix (because of how
that is incorporated into names) and the other set does not (though I
suppose the operation could be renamed).
2) because the AVX2 instruction is bytes-only, to get the signature
right, requires "OverwriteBase" but OverwriteBase also requires
OverwriteClass and "simdgen does not support [OverwriteClass] in
inputs".
3) the default operation order is false, true, but we want this in a
"x.Merged(y, mask)" that pairs with "x.Masked(mask)" where the true
case is x and the false case is y/zero, but the default ordering for
VPBLENDVB and VPBLENDMB is false->x and true->y.
4) VPBLENDVB only comes in byte width, which causes problems
for floats.
All this may get fixed in the future, for now it is just an
implementation detail.
Change-Id: I61b655c7011e2c33f8644f704f886133c89d2f15
Reviewed-on: https://go-review.googlesource.com/c/go/+/693155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
This CL is generated by CL 693136.
Change-Id: Ifd2278d3f927efa008a14cc5e592e7c14b7120ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/693157
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Generated by arch/internal/simdgen CL 689735
A small number of conversions for testing purposes
Change-Id: I4d52c643d08c02794c3fea9778bb1ecbb5507de4
Reviewed-on: https://go-review.googlesource.com/c/go/+/689716
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
generated by simdgen CL 689655
(which names a different CL, because it was submitted before realizing
that git had lost a from a stack, somehow)
Change-Id: Iab2868e848c221de98995ba0c632f97e2ee97670
Reviewed-on: https://go-review.googlesource.com/c/go/+/692336
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is partially generated by CL 689775.
Change-Id: I0c36fd2a44706c88db1a1d5ea4a6d0b9f891d85f
Reviewed-on: https://go-review.googlesource.com/c/go/+/689795
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 687975.
Change-Id: I21707d108773cc6d8e6f07aaed60e756faa1e6cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/687995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This CL also added some tests for them.
This CL is generated by CL 687919.
Change-Id: I9ddd2cd23bb98ecca91bfbeaffd62faa4bd85e0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/687939
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This CL is generated by CL 687915.
Change-Id: I1a2fb031c086b2b23fd135c48f8494ba5122493a
Reviewed-on: https://go-review.googlesource.com/c/go/+/687916
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This CL does:
1. Removes ShiftRightSignExtended, default signed vectors to shift
arithmetic, and unsigned to shift logical.
2. Add the missing Shifts which were left out by YAML error in the
generator.
This CL is generated by CL 687595.
Change-Id: I663115498adb91c82e89a8476e6748794e997cfa
Reviewed-on: https://go-review.googlesource.com/c/go/+/687596
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This CL is generated by CL 686817.
Change-Id: I19b8e468594514b2b1c99f8ad766f78b5e194c80
Reviewed-on: https://go-review.googlesource.com/c/go/+/686876
TryBot-Bypass: David Chase <drchase@google.com>
Reviewed-by: David Chase <drchase@google.com>
This CL is generated by CL 686775.
Change-Id: I10606cfdd4be015c8d251ba4275e1191d5bf0944
Reviewed-on: https://go-review.googlesource.com/c/go/+/686695
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL contains codes generated by CL 686556.
Change-Id: I4d7287476b478efdc186a64c12de33528c7fb0af
Reviewed-on: https://go-review.googlesource.com/c/go/+/686476
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 686575.
Change-Id: I1483189a1ae9bed51446fd69daab3f7b128549ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/686516
Reviewed-by: David Chase <drchase@google.com>
TryBot-Bypass: David Chase <drchase@google.com>
This CL is generated by CL 686555.
Change-Id: I0efb86a919692cd97c1c5b6365d77361a30bf7cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/686496
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This is generated by simdgen CL 684080
and should be submitted after it.
Also includes tests.
Change-Id: I1d680911134d8fb92f4deccae4ec373f3ed9f752
Reviewed-on: https://go-review.googlesource.com/c/go/+/684115
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is created by simdgen CL 684055 and
should be submitted after it.
Also includes a test.
Change-Id: I2ad7ae51d11cfc19745e866150e2eaf010d4ea49
Reviewed-on: https://go-review.googlesource.com/c/go/+/684077
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 684275.
Change-Id: Ie1efd0979af0ef0a56781bf9013071bf4d2c52c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/684175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This CL is generated by CL 683475.
Change-Id: I9e3ac6aff6f711cb26ff85e4c8729d9e2cc38e7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/683715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This CL is generated by simdgen CL 683836
and this CL should be submitted after its
generator.
Change-Id: I1aa893b185826ad1f9fb60b85c75eda31f70623b
Reviewed-on: https://go-review.googlesource.com/c/go/+/683797
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
These changes are required to make gp-returning simd
ops work. amd64/ssa.go includes a new code generator
helper, gc/main.go initializes intrinsics AFTER
the types, ssa/_gen/*AMD64.go add another register
shape to the simd ops function.
This CL should be submitted after simdgen CL 683858
which generated some of the changes.
Change-Id: I0af752ba8882fa131b875ff9c741ef70afbc60d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/683816
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
This is paired with simdgen CL 683055
Change-Id: I91d2c08a97ddd7cf06dd24478d552b962846131c
Reviewed-on: https://go-review.googlesource.com/c/go/+/683035
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This is paired with a matching simdgen CL 682679
Change-Id: Id494d40b5e64b723a47c1682b71e523a77b0eb87
Reviewed-on: https://go-review.googlesource.com/c/go/+/682656
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This CL only keeps one shape of those fused mul/add/sub operations. The
rest of the instructions will be generated during lowering as an
optimization.
This CL is generated by CL 682436.
Change-Id: Iadee1786185289838e04f3aa8f333844cfacc02e
Reviewed-on: https://go-review.googlesource.com/c/go/+/682435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This CL is generated by CL 682135.
Change-Id: I6f004b2eca6323f1ff22555c85db993386f24c6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/682155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
I re-ran the generator in arch/internal/simd to
verify a clean move of the intrinsics helpers, and
these changes (which look correct) appeared.
Change-Id: I28a0e8bd144d47aec216f557f238362f238d0428
Reviewed-on: https://go-review.googlesource.com/c/go/+/681499
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
This CL is generated by CL 680595.
Change-Id: I5e06ea9bc6a62593fc3b00fd44c119a5ed0d9e90
Reviewed-on: https://go-review.googlesource.com/c/go/+/681299
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 680215.
Change-Id: Ie085e65e0473a8e96170702d7265d379ec8812ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/681298
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 678515.
Change-Id: Iac7c424bbbffc2514dff3495d6c408fa9c998c2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/681296
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL is generated by CL 678195.
Change-Id: Ica600229a4e9623fa45f3b5aa370cdd6d9c31686
Reviewed-on: https://go-review.googlesource.com/c/go/+/681295
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This is to accomodate dot product instructions.
Change-Id: I88b21f848d7a51ad036bb3555c30f12b72571b2b
Reviewed-on: https://go-review.googlesource.com/c/go/+/680235
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Junyang Shao <shaojunyang@google.com>
This CL is generated by tool in CL 667155.
Change-Id: I3829d0d2c96fe7000e2dd025a3006f96957d777a
Reviewed-on: https://go-review.googlesource.com/c/go/+/675618
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Junyang Shao <shaojunyang@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This combines several CLs into a single patch of "glue"
for the generated SIMD extensions.
This glue includes GOEXPERIMENT checks that disable
the creation of user-visible "simd" types and
that disable the registration of "simd" intrinsics.
The simd type checks were changed to work for either
package "simd" or "internal/simd" so that moving that
package won't be quite so fragile.
cmd/compile, internal/simd: glue for adding SIMD extensions to Go
cmd/compile: theft of Cherry's sample SIMD compilation
Change-Id: Id44e2f4bafe74032c26de576a8691b6f7d977e01
Reviewed-on: https://go-review.googlesource.com/c/go/+/675598
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
