cmd/compile: experimental loop iterator capture semantics change

Adds:
GOEXPERIMENT=loopvar (expected way of invoking)
-d=loopvar={-1,0,1,2,11,12} (for per-package control and/or logging)
-d=loopvarhash=... (for hash debugging)

loopvar=11,12 are for testing, benchmarking, and debugging.

If enabled,for loops of the form `for x,y := range thing`, if x and/or
y are addressed or captured by a closure, are transformed by renaming
x/y to a temporary and prepending an assignment to the body of the
loop x := tmp_x.  This changes the loop semantics by making each
iteration's instance of x be distinct from the others (currently they
are all aliased, and when this matters, it is almost always a bug).

3-range with captured iteration variables are also transformed,
though it is a more complex transformation.

"Optimized" to do a simpler transformation for
3-clause for where the increment is empty.

(Prior optimization of address-taking under Return disabled, because
it was incorrect; returns can have loops for children.  Restored in
a later CL.)

Includes support for -d=loopvarhash=<binary string> intended for use
with hash search and GOCOMPILEDEBUG=loopvarhash=<binary string>
(use `gossahash -e loopvarhash command-that-fails`).

Minor feature upgrades to hash-triggered features; clients can specify
that file-position hashes use only the most-inline position, and/or that
they use only the basenames of source files (not the full directory path).
Most-inlined is the right choice for debugging loop-iteration change
once the semantics are linked to the package across inlining; basename-only
makes it tractable to write tests (which, otherwise, depend on the full
pathname of the source file and thus vary).

Updates #57969.

Change-Id: I180a51a3f8d4173f6210c861f10de23de8a1b1db
Reviewed-on: https://go-review.googlesource.com/c/go/+/411904
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

src/cmd/compile/internal/base/flag.go

@@ -186,7 +186,61 @@ func ParseFlags() {
 	}
 
 	if Debug.Gossahash != "" {
-		hashDebug = NewHashDebug("gosshash", Debug.Gossahash, nil)
+		hashDebug = NewHashDebug("gossahash", Debug.Gossahash, nil)
+	}
+
+	// Three inputs govern loop iteration variable rewriting, hash, experiment, flag.
+	// The loop variable rewriting is:
+	// IF non-empty hash, then hash determines behavior (function+line match) (*)
+	// ELSE IF experiment and flag==0, then experiment (set flag=1)
+	// ELSE flag (note that build sets flag per-package), with behaviors:
+	//  -1 => no change to behavior.
+	//   0 => no change to behavior (unless non-empty hash, see above)
+	//   1 => apply change to likely-iteration-variable-escaping loops
+	//   2 => apply change, log results
+	//   11 => apply change EVERYWHERE, do not log results (for debugging/benchmarking)
+	//   12 => apply change EVERYWHERE, log results (for debugging/benchmarking)
+	//
+	// The expected uses of the these inputs are, in believed most-likely to least likely:
+	//  GOEXPERIMENT=loopvar -- apply change to entire application
+	//  -gcflags=some_package=-d=loopvar=1 -- apply change to some_package (**)
+	//  -gcflags=some_package=-d=loopvar=2 -- apply change to some_package, log it
+	//  GOEXPERIMENT=loopvar -gcflags=some_package=-d=loopvar=-1 -- apply change to all but one package
+	//  GOCOMPILEDEBUG=loopvarhash=... -- search for failure cause
+	//
+	//  (*) For debugging purposes, providing loopvar flag >= 11 will expand the hash-eligible set of loops to all.
+	// (**) Currently this applies to all code in the compilation of some_package, including
+	//     inlines from other packages that may have been compiled w/o the change.
+
+	if Debug.LoopVarHash != "" {
+		// This first little bit controls the inputs for debug-hash-matching.
+		basenameOnly := false
+		mostInlineOnly := true
+		if strings.HasPrefix(Debug.LoopVarHash, "FS") {
+			// Magic handshake for testing, use file suffixes only when hashing on a position.
+			// i.e., rather than /tmp/asdfasdfasdf/go-test-whatever/foo_test.go,
+			// hash only on "foo_test.go", so that it will be the same hash across all runs.
+			Debug.LoopVarHash = Debug.LoopVarHash[2:]
+			basenameOnly = true
+		}
+		if strings.HasPrefix(Debug.LoopVarHash, "IL") {
+			// When hash-searching on a position that is an inline site, default is to use the
+			// most-inlined position only.  This makes the hash faster, plus there's no point
+			// reporting a problem with all the inlining; there's only one copy of the source.
+			// However, if for some reason you wanted it per-site, you can get this.  (The default
+			// hash-search behavior for compiler debugging is at an inline site.)
+			Debug.LoopVarHash = Debug.LoopVarHash[2:]
+			mostInlineOnly = false
+		}
+		// end of testing trickiness
+		LoopVarHash = NewHashDebug("loopvarhash", Debug.LoopVarHash, nil)
+		if Debug.LoopVar < 11 { // >= 11 means all loops are rewrite-eligible
+			Debug.LoopVar = 1 // 1 means those loops that syntactically escape their dcl vars are eligible.
+		}
+		LoopVarHash.SetInlineSuffixOnly(mostInlineOnly)
+		LoopVarHash.SetFileSuffixOnly(basenameOnly)
+	} else if buildcfg.Experiment.LoopVar && Debug.LoopVar == 0 {
+		Debug.LoopVar = 1
 	}
 
 	if Debug.Fmahash != "" {