go/src/cmd/compile/internal/test/reproduciblebuilds_test.go

113 lines
2.9 KiB
Go
Raw Normal View History

cmd/compile: make builds reproducible in presence of **byte and **int8 CL 39915 introduced sorting of signats by ShortString for reproducible builds. But ShortString treats types byte and uint8 identically; same for rune and uint32. CL 39915 attempted to compensate for this by only adding the underlying type (uint8) to signats in addsignat. This only works for byte and uint8. For e.g. *byte and *uint, both get added, and their sort order is random, leading to non-reproducible builds. One fix would be to add yet another type printing mode that doesn't eliminate byte and rune, and use it for sorting signats. But the formatting routines are complicated enough as it is. Instead, just sort first by ShortString and then by String. We can't just use String, because ShortString makes distinctions that String doesn't. ShortString is really preferred here; String is serving only as a backstop for handling of bytes and runes. The long series of types in the test helps increase the odds of failure, allowing a smaller number of iterations in the test. On my machine, a full test takes 700ms. Passes toolstash-check. Updates #19961 Fixes #20272 name old alloc/op new alloc/op delta Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5) Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5) GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5) Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5) SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5) Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5) GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5) Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5) Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5) XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5) [Geo mean] 72.0MB 72.0MB +0.07% name old allocs/op new allocs/op delta Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5) Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5) GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5) Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5) SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5) Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5) GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5) Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5) Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5) XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5) [Geo mean] 739k 741k +0.26% Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7 Reviewed-on: https://go-review.googlesource.com/42955 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-05-06 23:19:41 -07:00
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package test
cmd/compile: make builds reproducible in presence of **byte and **int8 CL 39915 introduced sorting of signats by ShortString for reproducible builds. But ShortString treats types byte and uint8 identically; same for rune and uint32. CL 39915 attempted to compensate for this by only adding the underlying type (uint8) to signats in addsignat. This only works for byte and uint8. For e.g. *byte and *uint, both get added, and their sort order is random, leading to non-reproducible builds. One fix would be to add yet another type printing mode that doesn't eliminate byte and rune, and use it for sorting signats. But the formatting routines are complicated enough as it is. Instead, just sort first by ShortString and then by String. We can't just use String, because ShortString makes distinctions that String doesn't. ShortString is really preferred here; String is serving only as a backstop for handling of bytes and runes. The long series of types in the test helps increase the odds of failure, allowing a smaller number of iterations in the test. On my machine, a full test takes 700ms. Passes toolstash-check. Updates #19961 Fixes #20272 name old alloc/op new alloc/op delta Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5) Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5) GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5) Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5) SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5) Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5) GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5) Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5) Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5) XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5) [Geo mean] 72.0MB 72.0MB +0.07% name old allocs/op new allocs/op delta Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5) Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5) GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5) Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5) SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5) Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5) GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5) Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5) Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5) XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5) [Geo mean] 739k 741k +0.26% Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7 Reviewed-on: https://go-review.googlesource.com/42955 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-05-06 23:19:41 -07:00
import (
"bytes"
"internal/testenv"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"testing"
)
func TestReproducibleBuilds(t *testing.T) {
tests := []string{
"issue20272.go",
"issue27013.go",
"issue30202.go",
}
cmd/compile: make builds reproducible in presence of **byte and **int8 CL 39915 introduced sorting of signats by ShortString for reproducible builds. But ShortString treats types byte and uint8 identically; same for rune and uint32. CL 39915 attempted to compensate for this by only adding the underlying type (uint8) to signats in addsignat. This only works for byte and uint8. For e.g. *byte and *uint, both get added, and their sort order is random, leading to non-reproducible builds. One fix would be to add yet another type printing mode that doesn't eliminate byte and rune, and use it for sorting signats. But the formatting routines are complicated enough as it is. Instead, just sort first by ShortString and then by String. We can't just use String, because ShortString makes distinctions that String doesn't. ShortString is really preferred here; String is serving only as a backstop for handling of bytes and runes. The long series of types in the test helps increase the odds of failure, allowing a smaller number of iterations in the test. On my machine, a full test takes 700ms. Passes toolstash-check. Updates #19961 Fixes #20272 name old alloc/op new alloc/op delta Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5) Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5) GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5) Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5) SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5) Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5) GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5) Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5) Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5) XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5) [Geo mean] 72.0MB 72.0MB +0.07% name old allocs/op new allocs/op delta Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5) Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5) GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5) Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5) SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5) Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5) GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5) Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5) Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5) XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5) [Geo mean] 739k 741k +0.26% Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7 Reviewed-on: https://go-review.googlesource.com/42955 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-05-06 23:19:41 -07:00
testenv.MustHaveGoBuild(t)
iters := 10
if testing.Short() {
iters = 4
}
t.Parallel()
for _, test := range tests {
test := test
t.Run(test, func(t *testing.T) {
t.Parallel()
var want []byte
tmp, err := ioutil.TempFile("", "")
if err != nil {
t.Fatalf("temp file creation failed: %v", err)
}
defer os.Remove(tmp.Name())
defer tmp.Close()
for i := 0; i < iters; i++ {
// Note: use -c 2 to expose any nondeterminism which is the result
// of the runtime scheduler.
out, err := exec.Command(testenv.GoToolPath(t), "tool", "compile", "-c", "2", "-o", tmp.Name(), filepath.Join("testdata", "reproducible", test)).CombinedOutput()
if err != nil {
t.Fatalf("failed to compile: %v\n%s", err, out)
}
obj, err := ioutil.ReadFile(tmp.Name())
if err != nil {
t.Fatalf("failed to read object file: %v", err)
}
if i == 0 {
want = obj
} else {
if !bytes.Equal(want, obj) {
t.Fatalf("builds produced different output after %d iters (%d bytes vs %d bytes)", i, len(want), len(obj))
}
}
cmd/compile: make builds reproducible in presence of **byte and **int8 CL 39915 introduced sorting of signats by ShortString for reproducible builds. But ShortString treats types byte and uint8 identically; same for rune and uint32. CL 39915 attempted to compensate for this by only adding the underlying type (uint8) to signats in addsignat. This only works for byte and uint8. For e.g. *byte and *uint, both get added, and their sort order is random, leading to non-reproducible builds. One fix would be to add yet another type printing mode that doesn't eliminate byte and rune, and use it for sorting signats. But the formatting routines are complicated enough as it is. Instead, just sort first by ShortString and then by String. We can't just use String, because ShortString makes distinctions that String doesn't. ShortString is really preferred here; String is serving only as a backstop for handling of bytes and runes. The long series of types in the test helps increase the odds of failure, allowing a smaller number of iterations in the test. On my machine, a full test takes 700ms. Passes toolstash-check. Updates #19961 Fixes #20272 name old alloc/op new alloc/op delta Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5) Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5) GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5) Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5) SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5) Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5) GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5) Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5) Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5) XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5) [Geo mean] 72.0MB 72.0MB +0.07% name old allocs/op new allocs/op delta Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5) Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5) GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5) Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5) SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5) Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5) GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5) Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5) Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5) XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5) [Geo mean] 739k 741k +0.26% Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7 Reviewed-on: https://go-review.googlesource.com/42955 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-05-06 23:19:41 -07:00
}
})
cmd/compile: make builds reproducible in presence of **byte and **int8 CL 39915 introduced sorting of signats by ShortString for reproducible builds. But ShortString treats types byte and uint8 identically; same for rune and uint32. CL 39915 attempted to compensate for this by only adding the underlying type (uint8) to signats in addsignat. This only works for byte and uint8. For e.g. *byte and *uint, both get added, and their sort order is random, leading to non-reproducible builds. One fix would be to add yet another type printing mode that doesn't eliminate byte and rune, and use it for sorting signats. But the formatting routines are complicated enough as it is. Instead, just sort first by ShortString and then by String. We can't just use String, because ShortString makes distinctions that String doesn't. ShortString is really preferred here; String is serving only as a backstop for handling of bytes and runes. The long series of types in the test helps increase the odds of failure, allowing a smaller number of iterations in the test. On my machine, a full test takes 700ms. Passes toolstash-check. Updates #19961 Fixes #20272 name old alloc/op new alloc/op delta Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5) Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5) GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5) Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5) SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5) Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5) GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5) Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5) Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5) XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5) [Geo mean] 72.0MB 72.0MB +0.07% name old allocs/op new allocs/op delta Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5) Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5) GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5) Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5) SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5) Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5) GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5) Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5) Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5) XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5) [Geo mean] 739k 741k +0.26% Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7 Reviewed-on: https://go-review.googlesource.com/42955 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2017-05-06 23:19:41 -07:00
}
}
func TestIssue38068(t *testing.T) {
testenv.MustHaveGoBuild(t)
t.Parallel()
// Compile a small package with and without the concurrent
// backend, then check to make sure that the resulting archives
// are identical. Note: this uses "go tool compile" instead of
// "go build" since the latter will generate differnent build IDs
// if it sees different command line flags.
scenarios := []struct {
tag string
args string
libpath string
}{
{tag: "serial", args: "-c=1"},
{tag: "concurrent", args: "-c=2"}}
tmpdir, err := ioutil.TempDir("", "TestIssue38068")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpdir)
src := filepath.Join("testdata", "reproducible", "issue38068.go")
for i := range scenarios {
s := &scenarios[i]
s.libpath = filepath.Join(tmpdir, s.tag+".a")
// Note: use of "-p" required in order for DWARF to be generated.
cmd := exec.Command(testenv.GoToolPath(t), "tool", "compile", "-trimpath", "-p=issue38068", "-buildid=", s.args, "-o", s.libpath, src)
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%v: %v:\n%s", cmd.Args, err, out)
}
}
readBytes := func(fn string) []byte {
payload, err := ioutil.ReadFile(fn)
if err != nil {
t.Fatalf("failed to read executable '%s': %v", fn, err)
}
return payload
}
b1 := readBytes(scenarios[0].libpath)
b2 := readBytes(scenarios[1].libpath)
if !bytes.Equal(b1, b2) {
t.Fatalf("concurrent and serial builds produced different output")
}
}