testing: make failure in benchmark cause non-zero exit status

Moves the implementation of RunBenchmarks to a non-exported function
that returns whether the execution was OK, and uses that to identify
failure in benchmarks.The exported function is kept for compatibility.

Like before, benchmarks will only be executed if tests and examples
pass. The PASS message will not be printed if there was a failure in
a benchmark.

Example output

	BenchmarkThatCallsFatal-8	--- FAIL: BenchmarkThatCallsFatal-8
		x_test.go:6: called by benchmark
	FAIL
	exit status 1
	FAIL	_/.../src/cmd/go/testdata/src/benchfatal	0.009s

Fixes #14307.

Change-Id: I6f3ddadc7da8a250763168cc099ae8b325a79602
Reviewed-on: https://go-review.googlesource.com/19889
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit is contained in:
Caio Marcelo de Oliveira Filho 2016-02-26 16:36:50 -03:00 committed by Ian Lance Taylor
parent c8ef0df06c
commit 3cb870d47b
4 changed files with 24 additions and 3 deletions

View file

@ -515,13 +515,12 @@ func (m *M) Run() int {
testOk := RunTests(m.matchString, m.tests)
exampleOk := RunExamples(m.matchString, m.examples)
stopAlarm()
if !testOk || !exampleOk {
if !testOk || !exampleOk || !runBenchmarksInternal(m.matchString, m.benchmarks) {
fmt.Println("FAIL")
after()
return 1
}
fmt.Println("PASS")
RunBenchmarks(m.matchString, m.benchmarks)
after()
return 0
}