cmd/compile: schedule increments after flags

ScoreInductionInc was introduced in 19f05770b0.
The goal was to keep the i++ in-place in a register.

Placing ScoreInductionInc later than ScoreFlags further improves
the generated code, mainly for code involving carry chains.

For example, the math/big.addVW_ref inner loop was:

    LEAQ    1(CX), R8
    ADDQ    DX, R9
    MOVQ    R9, (AX)(CX*8)
    SBBQ    R9, R9
    NEGQ    R9
    MOVQ    R8, CX

After this commit:

    ADDQ    DX, R9
    MOVQ    R9, (AX)(CX*8)
    SBBQ    R9, R9
    NEGQ    R9
    INCQ    CX

This is almost uniformly an improvement, across GOARCHes.
There are a few functions where this perturbs regalloc and causes
a little bit of movement, but they are rare and appear to be the
usual uninteresting regalloc change noise.

Change-Id: I883a92e4511136f478cf49471ba8b628434393dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/773660
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: golang-scoped@luci-project-accounts.iam.gserviceaccount.com <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>

src/cmd/compile/internal/ssa/schedule.go

@@ -22,8 +22,8 @@ const (
 	ScoreMemory
 	ScoreReadFlags
 	ScoreDefault
-	ScoreInductionInc // an increment of an induction variable
 	ScoreFlags
+	ScoreInductionInc // an increment of an induction variable
 	ScoreControl // towards bottom of block
 )
 

test/codegen/schedule.go

@@ -6,12 +6,28 @@
 
 package codegen
 
+import "math/bits"
+
 func f(n int) int {
 	r := 0
-	// arm64:-"MOVD  R"
+	// arm64:-"MOVD R[0-9]+, R[0-9]+"
 	// amd64:-"LEAQ" "INCQ"
 	for i := range n {
 		r += i
 	}
 	return r
 }
+
+// addVW has loop body with both a flag-producing op and an induction-variable increment.
+// It ensures that the induction increment doesn't get pointlessly spilled.
+func addVWLike(z, x []uint, y uint) uint {
+	c := y
+	// arm64:-"MOVD R[0-9]+, R[0-9]+"
+	// amd64:-"LEAQ" "INCQ"
+	for i := range z {
+		zi, cc := bits.Add(x[i], c, 0)
+		z[i] = zi
+		c = cc
+	}
+	return c
+}