Stowage/go
2
0
Fork 0
mirror of https://github.com/golang/go.git synced 2025-12-08 06:10:04 +00:00
Code Issues Projects Releases Packages Wiki Activity

[dev.boringcrypto] all: merge master into dev.boringcrypto

Browse source 
Change-Id: I64346fbdbee03e28297ec202d5c8292d7fd60c2c
This commit is contained in:
Filippo Valsorda 2018-11-14 13:48:01 -05:00
commit de153ac2a1
153 changed files with 4674 additions and 1665 deletions
Download patch file Download diff file Expand all files Collapse all files

16
.github/ISSUE_TEMPLATE vendored
View file

@ -1,24 +1,36 @@
Please answer these questions before submitting your issue. Thanks!
<!-- Please answer these questions before submitting your issue. Thanks! -->
### What version of Go are you using (`go version`)?
<pre>
$ go version
</pre>
### Does this issue reproduce with the latest release?
### What operating system and processor architecture are you using (`go env`)?
<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
</pre></details>
### What did you do?
<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
-->
### What did you expect to see?
### What did you see instead?

4
doc/contribute.html
View file

@ -393,8 +393,8 @@ Remember you can always visit Gerrit to see the fine-grained review.
<p>
It is not possible to fully sync Gerrit and GitHub, at least at the moment,
so we recommend learning Gerrit.
It's different but powerful and familiarity
with help you understand the flow.
It's different but powerful and familiarity with it will help you understand
the flow.
</p>
<h3>Overview</h3>

15
doc/devel/release.html
View file

@ -41,6 +41,14 @@ See the <a href="https://github.com/golang/go/issues?q=milestone%3AGo1.11.1">Go
1.11.1 milestone</a> on our issue tracker for details.
</p>
<p>
go1.11.2 (released 2018/11/02) includes fixes to the compiler, linker,
documentation, go command, and the <code>database/sql</code> and
<code>go/types</code> packages.
See the <a href="https://github.com/golang/go/issues?q=milestone%3AGo1.11.2">Go
1.11.2 milestone</a> on our issue tracker for details.
</p>
<h2 id="go1.10">go1.10 (released 2018/02/16)</h2>
<p>
@ -83,6 +91,13 @@ See the <a href="https://github.com/golang/go/issues?q=milestone%3AGo1.10.4">Go
1.10.4 milestone</a> on our issue tracker for details.
</p>
<p>
go1.10.5 (released 2018/11/02) includes fixes to the go command, linker, runtime
and the <code>database/sql</code> package.
See the <a href="https://github.com/golang/go/issues?q=milestone%3AGo1.10.5">Go
1.10.5 milestone</a> on our issue tracker for details.
</p>
<h2 id="go1.9">go1.9 (released 2017/08/24)</h2>
<p>

10
doc/effective_go.html
View file

@ -246,14 +246,16 @@ func Compile(str string) (*Regexp, error) {
<p>
If every doc comment begins with the name of the item it describes,
the output of <code>godoc</code> can usefully be run through <code>grep</code>.
you can use the <a href="/cmd/go/#hdr-Show_documentation_for_package_or_symbol">doc</a>
subcommand of the <a href="/cmd/go/">go</a> tool
and run the output through <code>grep</code>.
Imagine you couldn't remember the name "Compile" but were looking for
the parsing function for regular expressions, so you ran
the command,
</p>
<pre>
$ godoc regexp | grep -i parse
$ go doc -all regexp | grep -i parse
</pre>
<p>
@ -264,10 +266,10 @@ which recalls the word you're looking for.
</p>
<pre>
$ godoc regexp | grep parse
$ go doc -all regexp | grep -i parse
Compile parses a regular expression and returns, if successful, a Regexp
MustCompile is like Compile but panics if the expression cannot be parsed.
parsed. It simplifies safe initialization of global variables holding
cannot be parsed. It simplifies safe initialization of global variables
$
</pre>

282
doc/go1.12.html Normal file
View file

@ -0,0 +1,282 @@
<!--{
"Title": "Go 1.12 Release Notes",
"Path": "/doc/go1.12",
"Template": true
}-->
<!--
NOTE: In this document and others in this directory, the convention is to
set fixed-width phrases with non-fixed-width spaces, as in
<code>hello</code> <code>world</code>.
Do not send CLs removing the interior tags from such phrases.
-->
<style>
ul li { margin: 0.5em 0; }
</style>
<h2 id="introduction">DRAFT RELEASE NOTES - Introduction to Go 1.12</h2>
<p>
<strong>
Go 1.12 is not yet released. These are work-in-progress
release notes. Go 1.12 is expected to be released in February 2019.
</strong>
</p>
<p>
The latest Go release, version 1.12, arrives six months after <a href="go1.11">Go 1.11</a>.
Most of its changes are in TODO.
As always, the release maintains the Go 1 <a href="/doc/go1compat.html">promise of compatibility</a>.
We expect almost all Go programs to continue to compile and run as before.
</p>
<h2 id="language">Changes to the language</h2>
<p>
There are no changes to the language specification.
</p>
<h2 id="tools">Tools</h2>
<h3 id="gocache">Build cache requirement</h3>
<p>
The build cache is now required as a step toward eliminating
<code>$GOPATH/pkg</code>. Setting the environment variable
<code>GOCACHE=off</code> to disable the
<a href="/cmd/go/#hdr-Build_and_test_caching">build cache</a>
has no effect in Go 1.12.
</p>
<h3 id="godoc">Godoc</h3>
<p>
In Go 1.12, <code>godoc</code> no longer has a command-line interface and
is only a web server. Users should use <code>go</code> <code>doc</code>
for command-line help output instead.
</p>
<h2 id="library">Core library</h2>
<p>
All of the changes to the standard library are minor.
</p>
<h3 id="minor_library_changes">Minor changes to the library</h3>
<p>
As always, there are various minor changes and updates to the library,
made with the Go 1 <a href="/doc/go1compat">promise of compatibility</a>
in mind.
</p>
<!-- CL 142717: https://golang.org/cl/142717: cmd/compile: remove obsolete "safe" mode -->
<!-- CL 144340: https://golang.org/cl/144340: cmd/compile: add -lang flag to specify language version -->
<!-- CL 141977: https://golang.org/cl/141977: cmd/doc: add -all flag to print all documentation for package -->
<!-- CL 126656: https://golang.org/cl/126656: cmd/go: add $GOFLAGS environment variable -->
<!-- CL 115677: https://golang.org/cl/115677: cmd/vet: check embedded field tags too -->
<dl id="build"><dt><a href="/pkg/build/">build</a></dt>
<dd>
<p><!-- CL 61511 -->
TODO: <a href="https://golang.org/cl/61511">https://golang.org/cl/61511</a>: support frame-pointer for arm64
</p>
</dl><!-- build -->
<dl id="bytes, strings"><dt><a href="/pkg/bytes, strings/">bytes, strings</a></dt>
<dd>
<p><!-- CL 137855 -->
TODO: <a href="https://golang.org/cl/137855">https://golang.org/cl/137855</a>: add ReplaceAll
</p>
<p><!-- CL 145098 -->
TODO: <a href="https://golang.org/cl/145098">https://golang.org/cl/145098</a>: fix Reader.UnreadRune returning without error on a zero Reader
</p>
</dl><!-- bytes, strings -->
<dl id="crypto/tls, net/http"><dt><a href="/pkg/crypto/tls, net/http/">crypto/tls, net/http</a></dt>
<dd>
<p><!-- CL 143177 -->
TODO: <a href="https://golang.org/cl/143177">https://golang.org/cl/143177</a>: reject HTTP requests to HTTPS server
</p>
</dl><!-- crypto/tls, net/http -->
<dl id="expvar"><dt><a href="/pkg/expvar/">expvar</a></dt>
<dd>
<p><!-- CL 139537 -->
TODO: <a href="https://golang.org/cl/139537">https://golang.org/cl/139537</a>: add Map.Delete
</p>
</dl><!-- expvar -->
<dl id="fmt"><dt><a href="/pkg/fmt/">fmt</a></dt>
<dd>
<p><!-- CL 142737 -->
TODO: <a href="https://golang.org/cl/142737">https://golang.org/cl/142737</a>: print maps in key-sorted order
</p>
</dl><!-- fmt -->
<dl id="go/build, cmd/go"><dt><a href="/pkg/go/build, cmd/go/">go/build, cmd/go</a></dt>
<dd>
<p><!-- CL 146023 -->
TODO: <a href="https://golang.org/cl/146023">https://golang.org/cl/146023</a>: add &#34;hurd&#34; as a GOOS value
</p>
</dl><!-- go/build, cmd/go -->
<dl id="go/doc"><dt><a href="/pkg/go/doc/">go/doc</a></dt>
<dd>
<p><!-- CL 140958 -->
TODO: <a href="https://golang.org/cl/140958">https://golang.org/cl/140958</a>: add new mode bit PreserveAST to control clearing of data in AST
</p>
</dl><!-- go/doc -->
<dl id="godoc, cmd/godoc"><dt><a href="/pkg/godoc, cmd/godoc/">godoc, cmd/godoc</a></dt>
<dd>
<p><!-- CL 141397 -->
TODO: <a href="https://golang.org/cl/141397">https://golang.org/cl/141397</a>: remove CLI support
</p>
</dl><!-- godoc, cmd/godoc -->
<dl id="image"><dt><a href="/pkg/image/">image</a></dt>
<dd>
<p><!-- CL 118755 -->
TODO: <a href="https://golang.org/cl/118755">https://golang.org/cl/118755</a>: make RegisterFormat safe for concurrent use
</p>
</dl><!-- image -->
<dl id="image/png"><dt><a href="/pkg/image/png/">image/png</a></dt>
<dd>
<p><!-- CL 134235 -->
TODO: <a href="https://golang.org/cl/134235">https://golang.org/cl/134235</a>: pack image data for small bitdepth paletted images
</p>
</dl><!-- image/png -->
<dl id="internal/poll"><dt><a href="/pkg/internal/poll/">internal/poll</a></dt>
<dd>
<p><!-- CL 130676 -->
TODO: <a href="https://golang.org/cl/130676">https://golang.org/cl/130676</a>: use F_FULLFSYNC fcntl for FD.Fsync on OS X
</p>
</dl><!-- internal/poll -->
<dl id="io"><dt><a href="/pkg/io/">io</a></dt>
<dd>
<p><!-- CL 139457 -->
TODO: <a href="https://golang.org/cl/139457">https://golang.org/cl/139457</a>: export StringWriter
</p>
</dl><!-- io -->
<dl id="math/bits"><dt><a href="/pkg/math/bits/">math/bits</a></dt>
<dd>
<p><!-- CL 123157 -->
TODO: <a href="https://golang.org/cl/123157">https://golang.org/cl/123157</a>: add extended precision Add, Sub, Mul, Div
</p>
</dl><!-- math/bits -->
<dl id="net"><dt><a href="/pkg/net/">net</a></dt>
<dd>
<p><!-- CL 113997 -->
TODO: <a href="https://golang.org/cl/113997">https://golang.org/cl/113997</a>: use splice(2) on Linux when reading from UnixConn, rework splice tests
</p>
</dl><!-- net -->
<dl id="net/http"><dt><a href="/pkg/net/http/">net/http</a></dt>
<dd>
<p><!-- CL 130115 -->
TODO: <a href="https://golang.org/cl/130115">https://golang.org/cl/130115</a>: add Client.CloseIdleConnections
</p>
<p><!-- CL 145398 -->
TODO: <a href="https://golang.org/cl/145398">https://golang.org/cl/145398</a>: in Transport, don&#39;t error on non-chunked response with Trailer header
</p>
</dl><!-- net/http -->
<dl id="os"><dt><a href="/pkg/os/">os</a></dt>
<dd>
<p><!-- CL 125443 -->
TODO: <a href="https://golang.org/cl/125443">https://golang.org/cl/125443</a>: add ExitCode method to ProcessState
</p>
<p><!-- CL 135075 -->
TODO: <a href="https://golang.org/cl/135075">https://golang.org/cl/135075</a>: add ModeCharDevice to ModeType
</p>
<p><!-- CL 139418 -->
TODO: <a href="https://golang.org/cl/139418">https://golang.org/cl/139418</a>: add UserHomeDir
</p>
</dl><!-- os -->
<dl id="reflect"><dt><a href="/pkg/reflect/">reflect</a></dt>
<dd>
<p><!-- CL 33572 -->
TODO: <a href="https://golang.org/cl/33572">https://golang.org/cl/33572</a>: add Value.MapRange method and MapIter type
</p>
</dl><!-- reflect -->
<dl id="runtime"><dt><a href="/pkg/runtime/">runtime</a></dt>
<dd>
<p><!-- CL 135395 -->
TODO: <a href="https://golang.org/cl/135395">https://golang.org/cl/135395</a>: use MADV_FREE on Linux if available
</p>
</dl><!-- runtime -->
<dl id="strings"><dt><a href="/pkg/strings/">strings</a></dt>
<dd>
<p><!-- CL 122835 -->
TODO: <a href="https://golang.org/cl/122835">https://golang.org/cl/122835</a>: add Builder.Cap
</p>
</dl><!-- strings -->
<dl id="syscall"><dt><a href="/pkg/syscall/">syscall</a></dt>
<dd>
<p><!-- CL 125456 -->
TODO: <a href="https://golang.org/cl/125456">https://golang.org/cl/125456</a>: implement Unix Socket for Windows
</p>
<p><!-- CL 138595 -->
TODO: <a href="https://golang.org/cl/138595">https://golang.org/cl/138595</a>: FreeBSD 12 ino64 support
</p>
</dl><!-- syscall -->
<dl id="syscall/js"><dt><a href="/pkg/syscall/js/">syscall/js</a></dt>
<dd>
<p><!-- CL 141644 -->
TODO: <a href="https://golang.org/cl/141644">https://golang.org/cl/141644</a>: add Wrapper interface to support external Value wrapper types
</p>
<p><!-- CL 143137 -->
TODO: <a href="https://golang.org/cl/143137">https://golang.org/cl/143137</a>: make zero js.Value represent &#34;undefined&#34;
</p>
<p><!-- CL 144384 -->
TODO: <a href="https://golang.org/cl/144384">https://golang.org/cl/144384</a>: add the Value.Truthy method
</p>
</dl><!-- syscall/js -->
<dl id="testing"><dt><a href="/pkg/testing/">testing</a></dt>
<dd>
<p><!-- CL 139258 -->
TODO: <a href="https://golang.org/cl/139258">https://golang.org/cl/139258</a>: implement -benchtime=100x
</p>
</dl><!-- testing -->

2
doc/go_faq.html
View file

@ -804,7 +804,7 @@ type Fooer interface {
<p>
A type must then implement the <code>ImplementsFooer</code> method to be a
<code>Fooer</code>, clearly documenting the fact and announcing it in
<a href="/cmd/godoc/">godoc</a>'s output.
<a href="/cmd/go/#hdr-Show_documentation_for_package_or_symbol">go doc</a>'s output.
</p>
<pre>

49
misc/cgo/errors/ptr_test.go
View file

@ -357,6 +357,55 @@ var ptrTests = []ptrTest{
body: `r, _, _ := os.Pipe(); r.SetDeadline(time.Now().Add(C.US * time.Microsecond))`,
fail: false,
},
{
// Test for double evaluation of channel receive.
name: "chan-recv",
c: `void f(char** p) {}`,
imports: []string{"time"},
body: `c := make(chan []*C.char, 2); c <- make([]*C.char, 1); go func() { time.Sleep(10 * time.Second); panic("received twice from chan") }(); C.f(&(<-c)[0]);`,
fail: false,
},
{
// Test that converting the address of a struct field
// to unsafe.Pointer still just checks that field.
// Issue #25941.
name: "struct-field",
c: `void f(void* p) {}`,
imports: []string{"unsafe"},
support: `type S struct { p *int; a [8]byte; u uintptr }`,
body: `s := &S{p: new(int)}; C.f(unsafe.Pointer(&s.a))`,
fail: false,
},
{
// Test that converting multiple struct field
// addresses to unsafe.Pointer still just checks those
// fields. Issue #25941.
name: "struct-field-2",
c: `void f(void* p, int r, void* s) {}`,
imports: []string{"unsafe"},
support: `type S struct { a [8]byte; p *int; b int64; }`,
body: `s := &S{p: new(int)}; C.f(unsafe.Pointer(&s.a), 32, unsafe.Pointer(&s.b))`,
fail: false,
},
{
// Test that second argument to cgoCheckPointer is
// evaluated when a deferred function is deferred, not
// when it is run.
name: "defer2",
c: `void f(char **pc) {}`,
support: `type S1 struct { s []*C.char }; type S2 struct { ps *S1 }`,
body: `p := &S2{&S1{[]*C.char{nil}}}; defer C.f(&p.ps.s[0]); p.ps = nil`,
fail: false,
},
{
// Test that indexing into a function call still
// examines only the slice being indexed.
name: "buffer",
c: `void f(void *p) {}`,
imports: []string{"bytes", "unsafe"},
body: `var b bytes.Buffer; b.WriteString("a"); C.f(unsafe.Pointer(&b.Bytes()[0]))`,
fail: false,
},
}
func TestPointerChecks(t *testing.T) {

20
misc/cgo/test/issue28545.go Normal file
View file

@ -0,0 +1,20 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Failed to add type conversion for negative constant.
// No runtime test; just make sure it compiles.
package cgotest
/*
#include <complex.h>
static void issue28545F(char **p, int n, complex double a) {}
*/
import "C"
func issue28545G(p **C.char) {
C.issue28545F(p, -1, (0))
C.issue28545F(p, 2+3, complex(1, 1))
}

22
misc/cgo/test/twoargs.go Normal file
View file

@ -0,0 +1,22 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Crash from call with two arguments that need pointer checking.
// No runtime test; just make sure it compiles.
package cgotest
/*
static void twoargs1(void *p, int n) {}
static void *twoargs2() { return 0; }
static int twoargs3(void * p) { return 0; }
*/
import "C"
import "unsafe"
func twoargsF() {
v := []string{}
C.twoargs1(C.twoargs2(), C.twoargs3(unsafe.Pointer(&v)))
}

2
src/bufio/bufio.go
View file

@ -187,6 +187,8 @@ func (b *Reader) Discard(n int) (discarded int, err error) {
// The bytes are taken from at most one Read on the underlying Reader,
// hence n may be less than len(p).
// At EOF, the count will be zero and err will be io.EOF.
//
// To read exactly len(p) bytes, use io.ReadFull(b, p).
func (b *Reader) Read(p []byte) (n int, err error) {
n = len(p)
if n == 0 {

393
src/cmd/cgo/gcc.go
View file

@ -721,19 +721,19 @@ func (p *Package) mangleName(n *Name) {
// This returns whether the package needs to import unsafe as _cgo_unsafe.
func (p *Package) rewriteCalls(f *File) bool {
needsUnsafe := false
// Walk backward so that in C.f1(C.f2()) we rewrite C.f2 first.
for _, call := range f.Calls {
// This is a call to C.xxx; set goname to "xxx".
goname := call.Call.Fun.(*ast.SelectorExpr).Sel.Name
if goname == "malloc" {
if call.Done {
continue
}
name := f.Name[goname]
if name.Kind != "func" {
// Probably a type conversion.
continue
}
if p.rewriteCall(f, call, name) {
needsUnsafe = true
start := f.offset(call.Call.Pos())
end := f.offset(call.Call.End())
str, nu := p.rewriteCall(f, call)
if str != "" {
f.Edit.Replace(start, end, str)
if nu {
needsUnsafe = true
}
}
}
return needsUnsafe
@ -743,8 +743,29 @@ func (p *Package) rewriteCalls(f *File) bool {
// If any pointer checks are required, we rewrite the call into a
// function literal that calls _cgoCheckPointer for each pointer
// argument and then calls the original function.
// This returns whether the package needs to import unsafe as _cgo_unsafe.
func (p *Package) rewriteCall(f *File, call *Call, name *Name) bool {
// This returns the rewritten call and whether the package needs to
// import unsafe as _cgo_unsafe.
// If it returns the empty string, the call did not need to be rewritten.
func (p *Package) rewriteCall(f *File, call *Call) (string, bool) {
// This is a call to C.xxx; set goname to "xxx".
// It may have already been mangled by rewriteName.
var goname string
switch fun := call.Call.Fun.(type) {
case *ast.SelectorExpr:
goname = fun.Sel.Name
case *ast.Ident:
goname = strings.TrimPrefix(fun.Name, "_C2func_")
goname = strings.TrimPrefix(goname, "_Cfunc_")
}
if goname == "" || goname == "malloc" {
return "", false
}
name := f.Name[goname]
if name == nil || name.Kind != "func" {
// Probably a type conversion.
return "", false
}
params := name.FuncType.Params
args := call.Call.Args
@ -752,7 +773,7 @@ func (p *Package) rewriteCall(f *File, call *Call, name *Name) bool {
// less than the number of parameters.
// This will be caught when the generated file is compiled.
if len(args) < len(params) {
return false
return "", false
}
any := false
@ -763,111 +784,143 @@ func (p *Package) rewriteCall(f *File, call *Call, name *Name) bool {
}
}
if !any {
return false
return "", false
}
// We need to rewrite this call.
//
// We are going to rewrite C.f(p) to
// func (_cgo0 ptype) {
// Rewrite C.f(p) to
// func() {
// _cgo0 := p
// _cgoCheckPointer(_cgo0)
// C.f(_cgo0)
// }(p)
// Using a function literal like this lets us do correct
// argument type checking, and works correctly if the call is
// deferred.
// }()
// Using a function literal like this lets us evaluate the
// function arguments only once while doing pointer checks.
// This is particularly useful when passing additional arguments
// to _cgoCheckPointer, as done in checkIndex and checkAddr.
//
// When the function argument is a conversion to unsafe.Pointer,
// we unwrap the conversion before checking the pointer,
// and then wrap again when calling C.f. This lets us check
// the real type of the pointer in some cases. See issue #25941.
//
// When the call to C.f is deferred, we use an additional function
// literal to evaluate the arguments at the right time.
// defer func() func() {
// _cgo0 := p
// return func() {
// _cgoCheckPointer(_cgo0)
// C.f(_cgo0)
// }
// }()()
// This works because the defer statement evaluates the first
// function literal in order to get the function to call.
var sb bytes.Buffer
sb.WriteString("func(")
sb.WriteString("func() ")
if call.Deferred {
sb.WriteString("func() ")
}
needsUnsafe := false
for i, param := range params {
if i > 0 {
sb.WriteString(", ")
}
fmt.Fprintf(&sb, "_cgo%d ", i)
ptype := p.rewriteUnsafe(param.Go)
if ptype != param.Go {
needsUnsafe = true
}
sb.WriteString(gofmtLine(ptype))
}
sb.WriteString(")")
result := false
twoResults := false
if !call.Deferred {
// Check whether this call expects two results.
for _, ref := range f.Ref {
if ref.Expr != &call.Call.Fun {
continue
}
if ref.Context == ctxCall2 {
sb.WriteString("(")
result = true
twoResults = true
}
break
}
// Check whether this call expects two results.
for _, ref := range f.Ref {
if ref.Expr != &call.Call.Fun {
continue
}
if ref.Context == ctxCall2 {
sb.WriteString(" (")
// Add the result type, if any.
if name.FuncType.Result != nil {
rtype := p.rewriteUnsafe(name.FuncType.Result.Go)
if rtype != name.FuncType.Result.Go {
needsUnsafe = true
}
sb.WriteString(gofmtLine(rtype))
result = true
twoResults = true
}
break
// Add the second result type, if any.
if twoResults {
if name.FuncType.Result == nil {
// An explicit void result looks odd but it
// seems to be how cgo has worked historically.
sb.WriteString("_Ctype_void")
}
sb.WriteString(", error)")
}
}
// Add the result type, if any.
if name.FuncType.Result != nil {
rtype := p.rewriteUnsafe(name.FuncType.Result.Go)
if rtype != name.FuncType.Result.Go {
sb.WriteString("{ ")
// Define _cgoN for each argument value.
// Write _cgoCheckPointer calls to sbCheck.
var sbCheck bytes.Buffer
for i, param := range params {
arg, nu := p.mangle(f, &args[i])
if nu {
needsUnsafe = true
}
if !twoResults {
sb.WriteString(" ")
// Explicitly convert untyped constants to the
// parameter type, to avoid a type mismatch.
if p.isConst(f, arg) {
ptype := p.rewriteUnsafe(param.Go)
if ptype != param.Go {
needsUnsafe = true
}
arg = &ast.CallExpr{
Fun: ptype,
Args: []ast.Expr{arg},
}
}
sb.WriteString(gofmtLine(rtype))
result = true
}
// Add the second result type, if any.
if twoResults {
if name.FuncType.Result == nil {
// An explicit void result looks odd but it
// seems to be how cgo has worked historically.
sb.WriteString("_Ctype_void")
}
sb.WriteString(", error)")
}
sb.WriteString(" { ")
for i, param := range params {
arg := args[i]
if !p.needsPointerCheck(f, param.Go, arg) {
if !p.needsPointerCheck(f, param.Go, args[i]) {
fmt.Fprintf(&sb, "_cgo%d := %s; ", i, gofmtLine(arg))
continue
}
// Check for &a[i].
if p.checkIndex(&sb, f, arg, i) {
if p.checkIndex(&sb, &sbCheck, arg, i) {
continue
}
// Check for &x.
if p.checkAddr(&sb, arg, i) {
if p.checkAddr(&sb, &sbCheck, arg, i) {
continue
}
fmt.Fprintf(&sb, "_cgoCheckPointer(_cgo%d); ", i)
fmt.Fprintf(&sb, "_cgo%d := %s; ", i, gofmtLine(arg))
fmt.Fprintf(&sbCheck, "_cgoCheckPointer(_cgo%d); ", i)
}
if call.Deferred {
sb.WriteString("return func() { ")
}
// Write out the calls to _cgoCheckPointer.
sb.WriteString(sbCheck.String())
if result {
sb.WriteString("return ")
}
// Now we are ready to call the C function.
// To work smoothly with rewriteRef we leave the call in place
// and just insert our new arguments between the function
// and the old arguments.
f.Edit.Insert(f.offset(call.Call.Fun.Pos()), sb.String())
m, nu := p.mangle(f, &call.Call.Fun)
if nu {
needsUnsafe = true
}
sb.WriteString(gofmtLine(m))
sb.Reset()
sb.WriteString("(")
for i := range params {
if i > 0 {
@ -875,11 +928,17 @@ func (p *Package) rewriteCall(f *File, call *Call, name *Name) bool {
}
fmt.Fprintf(&sb, "_cgo%d", i)
}
sb.WriteString("); }")
sb.WriteString("); ")
if call.Deferred {
sb.WriteString("}")
}
sb.WriteString("}")
if call.Deferred {
sb.WriteString("()")
}
sb.WriteString("()")
f.Edit.Insert(f.offset(call.Call.Lparen), sb.String())
return needsUnsafe
return sb.String(), needsUnsafe
}
// needsPointerCheck returns whether the type t needs a pointer check.
@ -986,11 +1045,66 @@ func (p *Package) hasPointer(f *File, t ast.Expr, top bool) bool {
}
}
// mangle replaces references to C names in arg with the mangled names,
// rewriting calls when it finds them.
// It removes the corresponding references in f.Ref and f.Calls, so that we
// don't try to do the replacement again in rewriteRef or rewriteCall.
func (p *Package) mangle(f *File, arg *ast.Expr) (ast.Expr, bool) {
needsUnsafe := false
f.walk(arg, ctxExpr, func(f *File, arg interface{}, context astContext) {
px, ok := arg.(*ast.Expr)
if !ok {
return
}
sel, ok := (*px).(*ast.SelectorExpr)
if ok {
if l, ok := sel.X.(*ast.Ident); !ok || l.Name != "C" {
return
}
for _, r := range f.Ref {
if r.Expr == px {
*px = p.rewriteName(f, r)
r.Done = true
break
}
}
return
}
call, ok := (*px).(*ast.CallExpr)
if !ok {
return
}
for _, c := range f.Calls {
if !c.Done && c.Call.Lparen == call.Lparen {
cstr, nu := p.rewriteCall(f, c)
if cstr != "" {
// Smuggle the rewritten call through an ident.
*px = ast.NewIdent(cstr)
if nu {
needsUnsafe = true
}
c.Done = true
}
}
}
})
return *arg, needsUnsafe
}
// checkIndex checks whether arg the form &a[i], possibly inside type
// conversions. If so, and if a has no side effects, it writes
// _cgoCheckPointer(_cgoNN, a) to sb and returns true. This tells
// _cgoCheckPointer to check the complete contents of the slice.
func (p *Package) checkIndex(sb *bytes.Buffer, f *File, arg ast.Expr, i int) bool {
// conversions. If so, it writes
// _cgoIndexNN := a
// _cgoNN := &cgoIndexNN[i] // with type conversions, if any
// to sb, and writes
// _cgoCheckPointer(_cgoNN, _cgoIndexNN)
// to sbCheck, and returns true. This tells _cgoCheckPointer to check
// the complete contents of the slice or array being indexed, but no
// other part of the memory allocation.
func (p *Package) checkIndex(sb, sbCheck *bytes.Buffer, arg ast.Expr, i int) bool {
// Strip type conversions.
x := arg
for {
@ -1008,22 +1122,29 @@ func (p *Package) checkIndex(sb *bytes.Buffer, f *File, arg ast.Expr, i int) boo
if !ok {
return false
}
if p.hasSideEffects(f, index.X) {
return false
}
fmt.Fprintf(sb, "_cgoCheckPointer(_cgo%d, %s); ", i, gofmtLine(index.X))
fmt.Fprintf(sb, "_cgoIndex%d := %s; ", i, gofmtLine(index.X))
origX := index.X
index.X = ast.NewIdent(fmt.Sprintf("_cgoIndex%d", i))
fmt.Fprintf(sb, "_cgo%d := %s; ", i, gofmtLine(arg))
index.X = origX
fmt.Fprintf(sbCheck, "_cgoCheckPointer(_cgo%d, _cgoIndex%d); ", i, i)
return true
}
// checkAddr checks whether arg has the form &x, possibly inside type
// conversions. If so it writes _cgoCheckPointer(_cgoNN, true) to sb
// and returns true. This tells _cgoCheckPointer to check just the
// contents of the pointer being passed, not any other part of the
// memory allocation. This is run after checkIndex, which looks for
// the special case of &a[i], which requires different checks.
func (p *Package) checkAddr(sb *bytes.Buffer, arg ast.Expr, i int) bool {
// conversions. If so it writes
// _cgoBaseNN := &x
// _cgoNN := _cgoBaseNN // with type conversions, if any
// to sb, and writes
// _cgoCheckPointer(_cgoBaseNN, true)
// to sbCheck, and returns true. This tells _cgoCheckPointer to check
// just the contents of the pointer being passed, not any other part
// of the memory allocation. This is run after checkIndex, which looks
// for the special case of &a[i], which requires different checks.
func (p *Package) checkAddr(sb, sbCheck *bytes.Buffer, arg ast.Expr, i int) bool {
// Strip type conversions.
px := &arg
for {
@ -1037,27 +1158,20 @@ func (p *Package) checkAddr(sb *bytes.Buffer, arg ast.Expr, i int) bool {
return false
}
fmt.Fprintf(sb, "_cgoBase%d := %s; ", i, gofmtLine(*px))
origX := *px
*px = ast.NewIdent(fmt.Sprintf("_cgoBase%d", i))
fmt.Fprintf(sb, "_cgo%d := %s; ", i, gofmtLine(arg))
*px = origX
// Use "0 == 0" to do the right thing in the unlikely event
// that "true" is shadowed.
fmt.Fprintf(sb, "_cgoCheckPointer(_cgo%d, 0 == 0); ", i)
fmt.Fprintf(sbCheck, "_cgoCheckPointer(_cgoBase%d, 0 == 0); ", i)
return true
}
// hasSideEffects returns whether the expression x has any side
// effects. x is an expression, not a statement, so the only side
// effect is a function call.
func (p *Package) hasSideEffects(f *File, x ast.Expr) bool {
found := false
f.walk(x, ctxExpr,
func(f *File, x interface{}, context astContext) {
if _, ok := x.(*ast.CallExpr); ok {
found = true
}
})
return found
}
// isType returns whether the expression is definitely a type.
// This is conservative--it returns false for an unknown identifier.
func (p *Package) isType(t ast.Expr) bool {
@ -1087,6 +1201,9 @@ func (p *Package) isType(t ast.Expr) bool {
return true
}
if strings.HasPrefix(t.Name, "_Ctype_") {
return true
}
case *ast.StarExpr:
return p.isType(t.X)
case *ast.ArrayType, *ast.StructType, *ast.FuncType, *ast.InterfaceType,
@ -1097,6 +1214,46 @@ func (p *Package) isType(t ast.Expr) bool {
return false
}
// isConst returns whether x is an untyped constant expression.
func (p *Package) isConst(f *File, x ast.Expr) bool {
switch x := x.(type) {
case *ast.BasicLit:
return true
case *ast.SelectorExpr:
id, ok := x.X.(*ast.Ident)
if !ok || id.Name != "C" {
return false
}
name := f.Name[x.Sel.Name]
if name != nil {
return name.IsConst()
}
case *ast.Ident:
return x.Name == "nil" ||
strings.HasPrefix(x.Name, "_Ciconst_") ||
strings.HasPrefix(x.Name, "_Cfconst_") ||
strings.HasPrefix(x.Name, "_Csconst_")
case *ast.UnaryExpr:
return p.isConst(f, x.X)
case *ast.BinaryExpr:
return p.isConst(f, x.X) && p.isConst(f, x.Y)
case *ast.ParenExpr:
return p.isConst(f, x.X)
case *ast.CallExpr:
// Calling the builtin function complex on two untyped
// constants returns an untyped constant.
// TODO: It's possible to construct a case that will
// erroneously succeed if there is a local function
// named "complex", shadowing the builtin, that returns
// a numeric type. I can't think of any cases that will
// erroneously fail.
if id, ok := x.Fun.(*ast.Ident); ok && id.Name == "complex" && len(x.Args) == 2 {
return p.isConst(f, x.Args[0]) && p.isConst(f, x.Args[1])
}
}
return false
}
// rewriteUnsafe returns a version of t with references to unsafe.Pointer
// rewritten to use _cgo_unsafe.Pointer instead.
func (p *Package) rewriteUnsafe(t ast.Expr) ast.Expr {
@ -1205,11 +1362,13 @@ func (p *Package) rewriteRef(f *File) {
*r.Expr = expr
// Record source-level edit for cgo output.
repl := gofmt(expr)
if r.Name.Kind != "type" {
repl = "(" + repl + ")"
if !r.Done {
repl := gofmt(expr)
if r.Name.Kind != "type" {
repl = "(" + repl + ")"
}
f.Edit.Replace(f.offset(old.Pos()), f.offset(old.End()), repl)
}
f.Edit.Replace(f.offset(old.Pos()), f.offset(old.End()), repl)
}
// Remove functions only used as expressions, so their respective

2
src/cmd/cgo/main.go
View file

@ -81,6 +81,7 @@ func nameKeys(m map[string]*Name) []string {
type Call struct {
Call *ast.CallExpr
Deferred bool
Done bool
}
// A Ref refers to an expression of the form C.xxx in the AST.
@ -88,6 +89,7 @@ type Ref struct {
Name *Name
Expr *ast.Expr
Context astContext
Done bool
}
func (r *Ref) Pos() token.Pos {

16
src/cmd/compile/internal/gc/align.go
View file

@ -208,7 +208,7 @@ func dowidth(t *types.Type) {
}
t.Width = -2
t.Align = 0
t.Align = 0 // 0 means use t.Width, below
et := t.Etype
switch et {
@ -222,7 +222,7 @@ func dowidth(t *types.Type) {
}
}
w := int64(0)
var w int64
switch et {
default:
Fatalf("dowidth: unknown type: %v", t)
@ -366,7 +366,7 @@ func dowidth(t *types.Type) {
t.Width = w
if t.Align == 0 {
if w > 8 || w&(w-1) != 0 || w == 0 {
if w == 0 || w > 8 || w&(w-1) != 0 {
Fatalf("invalid alignment for %v", t)
}
t.Align = uint8(w)
@ -423,12 +423,11 @@ func checkwidth(t *types.Type) {
return
}
if t.Deferwidth() {
return
// if type has not yet been pushed on deferredTypeStack yet, do it now
if !t.Deferwidth() {
t.SetDeferwidth(true)
deferredTypeStack = append(deferredTypeStack, t)
}
t.SetDeferwidth(true)
deferredTypeStack = append(deferredTypeStack, t)
}
func defercheckwidth() {
@ -443,6 +442,7 @@ func resumecheckwidth() {
if defercalc == 0 {
Fatalf("resumecheckwidth")
}
for len(deferredTypeStack) > 0 {
t := deferredTypeStack[len(deferredTypeStack)-1]
deferredTypeStack = deferredTypeStack[:len(deferredTypeStack)-1]

2
src/cmd/compile/internal/gc/closure.go
View file

@ -16,7 +16,7 @@ func (p *noder) funcLit(expr *syntax.FuncLit) *Node {
xfunc := p.nod(expr, ODCLFUNC, nil, nil)
xfunc.Func.SetIsHiddenClosure(Curfn != nil)
xfunc.Func.Nname = p.setlineno(expr, newfuncname(nblank.Sym)) // filled in by typecheckclosure
xfunc.Func.Nname = newfuncnamel(p.pos(expr), nblank.Sym) // filled in by typecheckclosure
xfunc.Func.Nname.Name.Param.Ntype = xtype
xfunc.Func.Nname.Name.Defn = xfunc

8
src/cmd/compile/internal/gc/dcl.go
View file

@ -208,12 +208,6 @@ func newnoname(s *types.Sym) *Node {
return n
}
// newfuncname generates a new name node for a function or method.
// TODO(rsc): Use an ODCLFUNC node instead. See comment in CL 7360.
func newfuncname(s *types.Sym) *Node {
return newfuncnamel(lineno, s)
}
// newfuncnamel generates a new name node for a function or method.
// TODO(rsc): Use an ODCLFUNC node instead. See comment in CL 7360.
func newfuncnamel(pos src.XPos, s *types.Sym) *Node {
@ -1013,7 +1007,7 @@ func dclfunc(sym *types.Sym, tfn *Node) *Node {
}
fn := nod(ODCLFUNC, nil, nil)
fn.Func.Nname = newfuncname(sym)
fn.Func.Nname = newfuncnamel(lineno, sym)
fn.Func.Nname.Name.Defn = fn
fn.Func.Nname.Name.Param.Ntype = tfn
declare(fn.Func.Nname, PFUNC)

68
src/cmd/compile/internal/gc/noder.go
View file

@ -237,7 +237,7 @@ func (p *noder) node() {
types.Block = 1
imported_unsafe = false
p.lineno(p.file.PkgName)
p.setlineno(p.file.PkgName)
mkpackage(p.file.PkgName.Value)
xtop = append(xtop, p.decls(p.file.DeclList)...)
@ -259,7 +259,7 @@ func (p *noder) decls(decls []syntax.Decl) (l []*Node) {
var cs constState
for _, decl := range decls {
p.lineno(decl)
p.setlineno(decl)
switch decl := decl.(type) {
case *syntax.ImportDecl:
p.importDecl(decl)
@ -335,7 +335,7 @@ func (p *noder) varDecl(decl *syntax.VarDecl) []*Node {
exprs = p.exprList(decl.Values)
}
p.lineno(decl)
p.setlineno(decl)
return variter(names, typ, exprs)
}
@ -433,7 +433,9 @@ func (p *noder) declNames(names []*syntax.Name) []*Node {
}
func (p *noder) declName(name *syntax.Name) *Node {
return p.setlineno(name, dclname(p.name(name)))
n := dclname(p.name(name))
n.Pos = p.pos(name)
return n
}
func (p *noder) funcDecl(fun *syntax.FuncDecl) *Node {
@ -459,7 +461,7 @@ func (p *noder) funcDecl(fun *syntax.FuncDecl) *Node {
name = nblank.Sym // filled in by typecheckfunc
}
f.Func.Nname = p.setlineno(fun.Name, newfuncname(name))
f.Func.Nname = newfuncnamel(p.pos(fun.Name), name)
f.Func.Nname.Name.Defn = f
f.Func.Nname.Name.Param.Ntype = t
@ -502,7 +504,7 @@ func (p *noder) signature(recv *syntax.Field, typ *syntax.FuncType) *Node {
func (p *noder) params(params []*syntax.Field, dddOk bool) []*Node {
var nodes []*Node
for i, param := range params {
p.lineno(param)
p.setlineno(param)
nodes = append(nodes, p.param(param, dddOk, i+1 == len(params)))
}
return nodes
@ -552,15 +554,14 @@ func (p *noder) exprs(exprs []syntax.Expr) []*Node {
}
func (p *noder) expr(expr syntax.Expr) *Node {
p.lineno(expr)
p.setlineno(expr)
switch expr := expr.(type) {
case nil, *syntax.BadExpr:
return nil
case *syntax.Name:
return p.mkname(expr)
case *syntax.BasicLit:
return p.setlineno(expr, nodlit(p.basicLit(expr)))
return nodlit(p.basicLit(expr))
case *syntax.CompositeLit:
n := p.nod(expr, OCOMPLIT, nil, nil)
if expr.Type != nil {
@ -587,7 +588,9 @@ func (p *noder) expr(expr syntax.Expr) *Node {
obj.Name.SetUsed(true)
return oldname(restrictlookup(expr.Sel.Value, obj.Name.Pkg))
}
return p.setlineno(expr, nodSym(OXDOT, obj, p.name(expr.Sel)))
n := nodSym(OXDOT, obj, p.name(expr.Sel))
n.Pos = p.pos(expr) // lineno may have been changed by p.expr(expr.X)
return n
case *syntax.IndexExpr:
return p.nod(expr, OINDEX, p.expr(expr.X), p.expr(expr.Index))
case *syntax.SliceExpr:
@ -771,7 +774,7 @@ func (p *noder) chanDir(dir syntax.ChanDir) types.ChanDir {
func (p *noder) structType(expr *syntax.StructType) *Node {
var l []*Node
for i, field := range expr.FieldList {
p.lineno(field)
p.setlineno(field)
var n *Node
if field.Name == nil {
n = p.embedded(field.Type)
@ -784,7 +787,7 @@ func (p *noder) structType(expr *syntax.StructType) *Node {
l = append(l, n)
}
p.lineno(expr)
p.setlineno(expr)
n := p.nod(expr, OTSTRUCT, nil, nil)
n.List.Set(l)
return n
@ -793,7 +796,7 @@ func (p *noder) structType(expr *syntax.StructType) *Node {
func (p *noder) interfaceType(expr *syntax.InterfaceType) *Node {
var l []*Node
for _, method := range expr.MethodList {
p.lineno(method)
p.setlineno(method)
var n *Node
if method.Name == nil {
n = p.nodSym(method, ODCLFIELD, oldname(p.packname(method.Type)), nil)
@ -882,7 +885,7 @@ func (p *noder) stmt(stmt syntax.Stmt) *Node {
}
func (p *noder) stmtFall(stmt syntax.Stmt, fallOK bool) *Node {
p.lineno(stmt)
p.setlineno(stmt)
switch stmt := stmt.(type) {
case *syntax.EmptyStmt:
return nil
@ -1010,7 +1013,7 @@ func (p *noder) assignList(expr syntax.Expr, defn *Node, colas bool) []*Node {
newOrErr := false
for i, expr := range exprs {
p.lineno(expr)
p.setlineno(expr)
res[i] = nblank
name, ok := expr.(*syntax.Name)
@ -1132,7 +1135,7 @@ func (p *noder) switchStmt(stmt *syntax.SwitchStmt) *Node {
func (p *noder) caseClauses(clauses []*syntax.CaseClause, tswitch *Node, rbrace syntax.Pos) []*Node {
var nodes []*Node
for i, clause := range clauses {
p.lineno(clause)
p.setlineno(clause)
if i > 0 {
p.closeScope(clause.Pos())
}
@ -1188,7 +1191,7 @@ func (p *noder) selectStmt(stmt *syntax.SelectStmt) *Node {
func (p *noder) commClauses(clauses []*syntax.CommClause, rbrace syntax.Pos) []*Node {
var nodes []*Node
for i, clause := range clauses {
p.lineno(clause)
p.setlineno(clause)
if i > 0 {
p.closeScope(clause.Pos())
}
@ -1361,33 +1364,28 @@ func (p *noder) wrapname(n syntax.Node, x *Node) *Node {
}
func (p *noder) nod(orig syntax.Node, op Op, left, right *Node) *Node {
return p.setlineno(orig, nod(op, left, right))
return nodl(p.pos(orig), op, left, right)
}
func (p *noder) nodSym(orig syntax.Node, op Op, left *Node, sym *types.Sym) *Node {
return p.setlineno(orig, nodSym(op, left, sym))
n := nodSym(op, left, sym)
n.Pos = p.pos(orig)
return n
}
func (p *noder) setlineno(src_ syntax.Node, dst *Node) *Node {
pos := src_.Pos()
if !pos.IsKnown() {
// TODO(mdempsky): Shouldn't happen. Fix package syntax.
return dst
func (p *noder) pos(n syntax.Node) src.XPos {
// TODO(gri): orig.Pos() should always be known - fix package syntax
xpos := lineno
if pos := n.Pos(); pos.IsKnown() {
xpos = p.makeXPos(pos)
}
dst.Pos = p.makeXPos(pos)
return dst
return xpos
}
func (p *noder) lineno(n syntax.Node) {
if n == nil {
return
func (p *noder) setlineno(n syntax.Node) {
if n != nil {
lineno = p.pos(n)
}
pos := n.Pos()
if !pos.IsKnown() {
// TODO(mdempsky): Shouldn't happen. Fix package syntax.
return
}
lineno = p.makeXPos(pos)
}
// error is called concurrently if files are parsed concurrently.

2
src/cmd/compile/internal/gc/obj.go
View file

@ -81,7 +81,7 @@ func printObjHeader(bout *bio.Writer) {
if localpkg.Name == "main" {
fmt.Fprintf(bout, "main\n")
}
fmt.Fprintf(bout, "\n") // header ends with blank line
fmt.Fprintf(bout, "\n") // header ends with blank line
}
func startArchiveEntry(bout *bio.Writer) int64 {

12
src/cmd/compile/internal/gc/subr.go
View file

@ -696,14 +696,22 @@ func convertop(src *types.Type, dst *types.Type, why *string) Op {
}
// Conversions from regular to go:notinheap are not allowed
// (unless it's unsafe.Pointer). This is a runtime-specific
// rule.
// (unless it's unsafe.Pointer). These are runtime-specific
// rules.
// (a) Disallow (*T) to (*U) where T is go:notinheap but U isn't.
if src.IsPtr() && dst.IsPtr() && dst.Elem().NotInHeap() && !src.Elem().NotInHeap() {
if why != nil {
*why = fmt.Sprintf(":\n\t%v is go:notinheap, but %v is not", dst.Elem(), src.Elem())
}
return 0
}
// (b) Disallow string to []T where T is go:notinheap.
if src.IsString() && dst.IsSlice() && dst.Elem().NotInHeap() && (dst.Elem().Etype == types.Bytetype.Etype || dst.Elem().Etype == types.Runetype.Etype) {
if why != nil {
*why = fmt.Sprintf(":\n\t%v is go:notinheap", dst.Elem())
}
return 0
}
// 1. src can be assigned to dst.
op := assignop(src, dst, why)

8
src/cmd/compile/internal/gc/walk.go
View file

@ -3297,6 +3297,14 @@ func walkcompare(n *Node, init *Nodes) *Node {
}
if expr == nil {
expr = nodbool(n.Op == OEQ)
// We still need to use cmpl and cmpr, in case they contain
// an expression which might panic. See issue 23837.
t := temp(cmpl.Type)
a1 := nod(OAS, t, cmpl)
a1 = typecheck(a1, Etop)
a2 := nod(OAS, t, cmpr)
a2 = typecheck(a2, Etop)
init.Append(a1, a2)
}
n = finishcompare(n, expr, init)
return n

11
src/cmd/compile/internal/ssa/gen/AMD64.rules
View file

@ -286,7 +286,16 @@
(Move [7] dst src mem) ->
(MOVLstore [3] dst (MOVLload [3] src mem)
(MOVLstore dst (MOVLload src mem) mem))
(Move [s] dst src mem) && s > 8 && s < 16 ->
(Move [9] dst src mem) ->
(MOVBstore [8] dst (MOVBload [8] src mem)
(MOVQstore dst (MOVQload src mem) mem))
(Move [10] dst src mem) ->
(MOVWstore [8] dst (MOVWload [8] src mem)
(MOVQstore dst (MOVQload src mem) mem))
(Move [12] dst src mem) ->
(MOVLstore [8] dst (MOVLload [8] src mem)
(MOVQstore dst (MOVQload src mem) mem))
(Move [s] dst src mem) && s == 11 || s >= 13 && s <= 15 ->
(MOVQstore [s-8] dst (MOVQload [s-8] src mem)
(MOVQstore dst (MOVQload src mem) mem))

102
src/cmd/compile/internal/ssa/rewriteAMD64.go
View file

@ -932,7 +932,7 @@ func rewriteValueAMD64(v *Value) bool {
case OpMod8u:
return rewriteValueAMD64_OpMod8u_0(v)
case OpMove:
return rewriteValueAMD64_OpMove_0(v) || rewriteValueAMD64_OpMove_10(v)
return rewriteValueAMD64_OpMove_0(v) || rewriteValueAMD64_OpMove_10(v) || rewriteValueAMD64_OpMove_20(v)
case OpMul16:
return rewriteValueAMD64_OpMul16_0(v)
case OpMul32:
@ -62736,8 +62736,95 @@ func rewriteValueAMD64_OpMove_10(v *Value) bool {
v.AddArg(v1)
return true
}
// match: (Move [9] dst src mem)
// cond:
// result: (MOVBstore [8] dst (MOVBload [8] src mem) (MOVQstore dst (MOVQload src mem) mem))
for {
if v.AuxInt != 9 {
break
}
_ = v.Args[2]
dst := v.Args[0]
src := v.Args[1]
mem := v.Args[2]
v.reset(OpAMD64MOVBstore)
v.AuxInt = 8
v.AddArg(dst)
v0 := b.NewValue0(v.Pos, OpAMD64MOVBload, typ.UInt8)
v0.AuxInt = 8
v0.AddArg(src)
v0.AddArg(mem)
v.AddArg(v0)
v1 := b.NewValue0(v.Pos, OpAMD64MOVQstore, types.TypeMem)
v1.AddArg(dst)
v2 := b.NewValue0(v.Pos, OpAMD64MOVQload, typ.UInt64)
v2.AddArg(src)
v2.AddArg(mem)
v1.AddArg(v2)
v1.AddArg(mem)
v.AddArg(v1)
return true
}
// match: (Move [10] dst src mem)
// cond:
// result: (MOVWstore [8] dst (MOVWload [8] src mem) (MOVQstore dst (MOVQload src mem) mem))
for {
if v.AuxInt != 10 {
break
}
_ = v.Args[2]
dst := v.Args[0]
src := v.Args[1]
mem := v.Args[2]
v.reset(OpAMD64MOVWstore)
v.AuxInt = 8
v.AddArg(dst)
v0 := b.NewValue0(v.Pos, OpAMD64MOVWload, typ.UInt16)
v0.AuxInt = 8
v0.AddArg(src)
v0.AddArg(mem)
v.AddArg(v0)
v1 := b.NewValue0(v.Pos, OpAMD64MOVQstore, types.TypeMem)
v1.AddArg(dst)
v2 := b.NewValue0(v.Pos, OpAMD64MOVQload, typ.UInt64)
v2.AddArg(src)
v2.AddArg(mem)
v1.AddArg(v2)
v1.AddArg(mem)
v.AddArg(v1)
return true
}
// match: (Move [12] dst src mem)
// cond:
// result: (MOVLstore [8] dst (MOVLload [8] src mem) (MOVQstore dst (MOVQload src mem) mem))
for {
if v.AuxInt != 12 {
break
}
_ = v.Args[2]
dst := v.Args[0]
src := v.Args[1]
mem := v.Args[2]
v.reset(OpAMD64MOVLstore)
v.AuxInt = 8
v.AddArg(dst)
v0 := b.NewValue0(v.Pos, OpAMD64MOVLload, typ.UInt32)
v0.AuxInt = 8
v0.AddArg(src)
v0.AddArg(mem)
v.AddArg(v0)
v1 := b.NewValue0(v.Pos, OpAMD64MOVQstore, types.TypeMem)
v1.AddArg(dst)
v2 := b.NewValue0(v.Pos, OpAMD64MOVQload, typ.UInt64)
v2.AddArg(src)
v2.AddArg(mem)
v1.AddArg(v2)
v1.AddArg(mem)
v.AddArg(v1)
return true
}
// match: (Move [s] dst src mem)
// cond: s > 8 && s < 16
// cond: s == 11 || s >= 13 && s <= 15
// result: (MOVQstore [s-8] dst (MOVQload [s-8] src mem) (MOVQstore dst (MOVQload src mem) mem))
for {
s := v.AuxInt
@ -62745,7 +62832,7 @@ func rewriteValueAMD64_OpMove_10(v *Value) bool {
dst := v.Args[0]
src := v.Args[1]
mem := v.Args[2]
if !(s > 8 && s < 16) {
if !(s == 11 || s >= 13 && s <= 15) {
break
}
v.reset(OpAMD64MOVQstore)
@ -62830,6 +62917,15 @@ func rewriteValueAMD64_OpMove_10(v *Value) bool {
v.AddArg(v2)
return true
}
return false
}
func rewriteValueAMD64_OpMove_20(v *Value) bool {
b := v.Block
_ = b
config := b.Func.Config
_ = config
typ := &b.Func.Config.Types
_ = typ
// match: (Move [s] dst src mem)
// cond: s > 16 && s%16 != 0 && s%16 > 8 && !config.useSSE
// result: (Move [s-s%16] (OffPtr <dst.Type> dst [s%16]) (OffPtr <src.Type> src [s%16]) (MOVQstore [8] dst (MOVQload [8] src mem) (MOVQstore dst (MOVQload src mem) mem)))

6
src/cmd/compile/internal/types/sym.go
View file

@ -39,9 +39,9 @@ type Sym struct {
const (
symOnExportList = 1 << iota // added to exportlist (no need to add again)
symUniq
symSiggen
symAsm
symAlgGen
symSiggen // type symbol has been generated
symAsm // on asmlist, for writing to -asmhdr
symAlgGen // algorithm table has been generated
)
func (sym *Sym) OnExportList() bool { return sym.flags&symOnExportList != 0 }

12
src/cmd/compile/internal/types/type.go
View file

@ -141,7 +141,7 @@ type Type struct {
Extra interface{}
// Width is the width of this Type in bytes.
Width int64
Width int64 // valid if Align > 0
methods Fields
allMethods Fields
@ -156,16 +156,16 @@ type Type struct {
Vargen int32 // unique name for OTYPE/ONAME
Etype EType // kind of type
Align uint8 // the required alignment of this type, in bytes
Align uint8 // the required alignment of this type, in bytes (0 means Width and Align have not yet been computed)
flags bitset8
}
const (
typeNotInHeap = 1 << iota // type cannot be heap allocated
typeBroke // broken type definition
typeNoalg // suppress hash and eq algorithm generation
typeDeferwidth
typeNotInHeap = 1 << iota // type cannot be heap allocated
typeBroke // broken type definition
typeNoalg // suppress hash and eq algorithm generation
typeDeferwidth // width computation has been deferred and type is on deferredTypeStack
typeRecur
)

20
src/cmd/doc/doc_test.go
View file

@ -481,6 +481,26 @@ var tests = []test{
`unexportedTypedConstant`, // No unexported constant.
},
},
// Type -all.
{
"type",
[]string{"-all", p, `ExportedType`},
[]string{
`type ExportedType struct {`, // Type definition as source.
`Comment about exported type`, // Include comment afterwards.
`const ConstGroup4 ExportedType = ExportedType\{\}`, // Related constants.
`ExportedTypedConstant ExportedType = iota`,
`Constants tied to ExportedType`,
`func ExportedTypeConstructor\(\) \*ExportedType`,
`Comment about constructor for exported type.`,
`func ReturnExported\(\) ExportedType`,
`func \(ExportedType\) ExportedMethod\(a int\) bool`,
`Comment about exported method.`,
},
[]string{
`unexportedType`,
},
},
// Type T1 dump (alias).
{
"type T1",

5
src/cmd/doc/main.go
View file

@ -133,10 +133,7 @@ func do(writer io.Writer, flagSet *flag.FlagSet, args []string) (err error) {
}
// We have a package.
if showAll {
if symbol != "" {
return fmt.Errorf("-all valid only for package, not symbol: %s", symbol)
}
if showAll && symbol == "" {
pkg.allDoc()
return
}

30
src/cmd/go/go_test.go
View file

@ -1462,8 +1462,38 @@ func TestInstallIntoGOPATH(t *testing.T) {
func TestBuildOutputToDevNull(t *testing.T) {
tg := testgo(t)
defer tg.cleanup()
fi1, err1 := os.Lstat(os.DevNull)
tg.setenv("GOPATH", filepath.Join(tg.pwd(), "testdata"))
tg.run("build", "-o", os.DevNull, "go-cmd-test")
fi2, err2 := os.Lstat(os.DevNull)
if err1 == nil {
if err2 != nil {
t.Errorf("second stat of /dev/null failed: %v", err2)
} else if !os.SameFile(fi1, fi2) {
t.Errorf("/dev/null changed: now %v was %v", fi1, fi2)
}
}
}
// Issue 28549.
func TestTestOutputToDevNull(t *testing.T) {
tg := testgo(t)
defer tg.cleanup()
fi1, err1 := os.Lstat(os.DevNull)
tg.makeTempdir()
tg.setenv("GOPATH", tg.path("."))
tg.tempFile("src/p/p.go", "package p\n")
tg.tempFile("src/p/p_test.go", "package p\nimport \"testing\"\nfunc TestX(t *testing.T) {}\n")
tg.run("test", "-o", os.DevNull, "-c", "p")
tg.mustNotExist("p.test")
fi2, err2 := os.Lstat(os.DevNull)
if err1 == nil {
if err2 != nil {
t.Errorf("second stat of /dev/null failed: %v", err2)
} else if !os.SameFile(fi1, fi2) {
t.Errorf("/dev/null changed: now %v was %v", fi1, fi2)
}
}
}
func TestPackageMainTestImportsArchiveNotBinary(t *testing.T) {

2
src/cmd/go/internal/imports/build.go
View file

@ -207,5 +207,5 @@ func init() {
}
}
const goosList = "aix android darwin dragonfly freebsd js linux nacl netbsd openbsd plan9 solaris windows zos "
const goosList = "aix android darwin dragonfly freebsd hurd js linux nacl netbsd openbsd plan9 solaris windows zos "
const goarchList = "386 amd64 amd64p32 arm armbe arm64 arm64be ppc64 ppc64le mips mipsle mips64 mips64le mips64p32 mips64p32le ppc riscv riscv64 s390 s390x sparc sparc64 wasm "

20
src/cmd/go/internal/test/test.go
View file

@ -887,15 +887,19 @@ func builderTest(b *work.Builder, p *load.Package) (buildAction, runAction, prin
target = filepath.Join(base.Cwd, target)
}
}
pmain.Target = target
installAction = &work.Action{
Mode: "test build",
Func: work.BuildInstallFunc,
Deps: []*work.Action{buildAction},
Package: pmain,
Target: target,
if target == os.DevNull {
runAction = buildAction
} else {
pmain.Target = target
installAction = &work.Action{
Mode: "test build",
Func: work.BuildInstallFunc,
Deps: []*work.Action{buildAction},
Package: pmain,
Target: target,
}
runAction = installAction // make sure runAction != nil even if not running test
}
runAction = installAction // make sure runAction != nil even if not running test
}
var vetRunAction *work.Action
if testC {

13
src/cmd/go/internal/work/buildid.go
View file

@ -322,13 +322,16 @@ func assemblerIsGas() bool {
}
}
// gccgoBuildIDELFFile creates an assembler file that records the
// action's build ID in an SHF_EXCLUDE section.
func (b *Builder) gccgoBuildIDELFFile(a *Action) (string, error) {
// gccgoBuildIDFile creates an assembler file that records the
// action's build ID in an SHF_EXCLUDE section for ELF files or
// in a CSECT in XCOFF files.
func (b *Builder) gccgoBuildIDFile(a *Action) (string, error) {
sfile := a.Objdir + "_buildid.s"
var buf bytes.Buffer
if cfg.Goos != "solaris" || assemblerIsGas() {
if cfg.Goos == "aix" {
fmt.Fprintf(&buf, "\t.csect .go.buildid[XO]\n")
} else if cfg.Goos != "solaris" || assemblerIsGas() {
fmt.Fprintf(&buf, "\t"+`.section .go.buildid,"e"`+"\n")
} else if cfg.Goarch == "sparc" || cfg.Goarch == "sparc64" {
fmt.Fprintf(&buf, "\t"+`.section ".go.buildid",#exclude`+"\n")
@ -347,7 +350,7 @@ func (b *Builder) gccgoBuildIDELFFile(a *Action) (string, error) {
fmt.Fprintf(&buf, "%#02x", a.buildID[i])
}
fmt.Fprintf(&buf, "\n")
if cfg.Goos != "solaris" {
if cfg.Goos != "solaris" && cfg.Goos != "aix" {
secType := "@progbits"
if cfg.Goarch == "arm" {
secType = "%progbits"

8
src/cmd/go/internal/work/exec.go
View file

@ -699,8 +699,8 @@ func (b *Builder) build(a *Action) (err error) {
// This is read by readGccgoArchive in cmd/internal/buildid/buildid.go.
if a.buildID != "" && cfg.BuildToolchainName == "gccgo" {
switch cfg.Goos {
case "android", "dragonfly", "freebsd", "linux", "netbsd", "openbsd", "solaris":
asmfile, err := b.gccgoBuildIDELFFile(a)
case "aix", "android", "dragonfly", "freebsd", "linux", "netbsd", "openbsd", "solaris":
asmfile, err := b.gccgoBuildIDFile(a)
if err != nil {
return err
}
@ -2297,6 +2297,10 @@ func (b *Builder) gccArchArgs() []string {
return []string{"-mabi=64"}
case "mips", "mipsle":
return []string{"-mabi=32", "-march=mips32"}
case "ppc64":
if cfg.Goos == "aix" {
return []string{"-maix64"}
}
}
return nil
}

38
src/cmd/go/internal/work/gccgo.go
View file

@ -186,7 +186,15 @@ func (gccgoToolchain) pack(b *Builder, a *Action, afile string, ofiles []string)
for _, f := range ofiles {
absOfiles = append(absOfiles, mkAbs(objdir, f))
}
return b.run(a, p.Dir, p.ImportPath, nil, "ar", "rc", mkAbs(objdir, afile), absOfiles)
var arArgs []string
if cfg.Goos == "aix" && cfg.Goarch == "ppc64" {
// AIX puts both 32-bit and 64-bit objects in the same archive.
// Tell the AIX "ar" command to only care about 64-bit objects.
// AIX "ar" command does not know D option.
arArgs = []string{"-X64"}
}
return b.run(a, p.Dir, p.ImportPath, nil, "ar", arArgs, "rc", mkAbs(objdir, afile), absOfiles)
}
func (tools gccgoToolchain) link(b *Builder, root *Action, out, importcfg string, allactions []*Action, buildmode, desc string) error {
@ -342,17 +350,24 @@ func (tools gccgoToolchain) link(b *Builder, root *Action, out, importcfg string
}
}
ldflags = append(ldflags, "-Wl,--whole-archive")
wholeArchive := []string{"-Wl,--whole-archive"}
noWholeArchive := []string{"-Wl,--no-whole-archive"}
if cfg.Goos == "aix" {
wholeArchive = nil
noWholeArchive = nil
}
ldflags = append(ldflags, wholeArchive...)
ldflags = append(ldflags, afiles...)
ldflags = append(ldflags, "-Wl,--no-whole-archive")
ldflags = append(ldflags, noWholeArchive...)
ldflags = append(ldflags, cgoldflags...)
ldflags = append(ldflags, envList("CGO_LDFLAGS", "")...)
if root.Package != nil {
ldflags = append(ldflags, root.Package.CgoLDFLAGS...)
}
ldflags = str.StringList("-Wl,-(", ldflags, "-Wl,-)")
if cfg.Goos != "aix" {
ldflags = str.StringList("-Wl,-(", ldflags, "-Wl,-)")
}
if root.buildID != "" {
// On systems that normally use gold or the GNU linker,
@ -363,11 +378,17 @@ func (tools gccgoToolchain) link(b *Builder, root *Action, out, importcfg string
}
}
var rLibPath string
if cfg.Goos == "aix" {
rLibPath = "-Wl,-blibpath="
} else {
rLibPath = "-Wl,-rpath="
}
for _, shlib := range shlibs {
ldflags = append(
ldflags,
"-L"+filepath.Dir(shlib),
"-Wl,-rpath="+filepath.Dir(shlib),
rLibPath+filepath.Dir(shlib),
"-l"+strings.TrimSuffix(
strings.TrimPrefix(filepath.Base(shlib), "lib"),
".so"))
@ -412,7 +433,10 @@ func (tools gccgoToolchain) link(b *Builder, root *Action, out, importcfg string
case "c-shared":
ldflags = append(ldflags, "-shared", "-nostdlib", "-Wl,--whole-archive", "-lgolibbegin", "-Wl,--no-whole-archive", "-lgo", "-lgcc_s", "-lgcc", "-lc", "-lgcc")
case "shared":
ldflags = append(ldflags, "-zdefs", "-shared", "-nostdlib", "-lgo", "-lgcc_s", "-lgcc", "-lc")
if cfg.Goos != "aix" {
ldflags = append(ldflags, "-zdefs")
}
ldflags = append(ldflags, "-shared", "-nostdlib", "-lgo", "-lgcc_s", "-lgcc", "-lc")
default:
base.Fatalf("-buildmode=%s not supported for gccgo", buildmode)

13
src/cmd/go/testdata/script/test_devnull.txt vendored Normal file
View file

@ -0,0 +1,13 @@
# go test -c -o NUL
# should work (see golang.org/issue/28035).
cd x
go test -o=$devnull -c
! exists x.test$exe
-- x/x_test.go --
package x_test
import (
"testing"
)
func TestNUL(t *testing.T) {
}

83
src/cmd/internal/buildid/buildid.go
View file

@ -6,6 +6,7 @@ package buildid
import (
"bytes"
"cmd/internal/xcoff"
"debug/elf"
"fmt"
"io"
@ -40,6 +41,9 @@ func ReadFile(name string) (id string, err error) {
return "", err
}
if string(buf) != "!<arch>\n" {
if string(buf) == "<bigaf>\n" {
return readGccgoBigArchive(name, f)
}
return readBinary(name, f)
}
@ -157,6 +161,85 @@ func readGccgoArchive(name string, f *os.File) (string, error) {
}
}
// readGccgoBigArchive tries to parse the archive as an AIX big
// archive file, and fetch the build ID from the _buildid.o entry.
// The _buildid.o entry is written by (*Builder).gccgoBuildIDXCOFFFile
// in cmd/go/internal/work/exec.go.
func readGccgoBigArchive(name string, f *os.File) (string, error) {
bad := func() (string, error) {
return "", &os.PathError{Op: "parse", Path: name, Err: errBuildIDMalformed}
}
// Read fixed-length header.
if _, err := f.Seek(0, io.SeekStart); err != nil {
return "", err
}
var flhdr [128]byte
if _, err := io.ReadFull(f, flhdr[:]); err != nil {
return "", err
}
// Read first member offset.
offStr := strings.TrimSpace(string(flhdr[68:88]))
off, err := strconv.ParseInt(offStr, 10, 64)
if err != nil {
return bad()
}
for {
if off == 0 {
// No more entries, no build ID.
return "", nil
}
if _, err := f.Seek(off, io.SeekStart); err != nil {
return "", err
}
// Read member header.
var hdr [112]byte
if _, err := io.ReadFull(f, hdr[:]); err != nil {
return "", err
}
// Read member name length.
namLenStr := strings.TrimSpace(string(hdr[108:112]))
namLen, err := strconv.ParseInt(namLenStr, 10, 32)
if err != nil {
return bad()
}
if namLen == 10 {
var nam [10]byte
if _, err := io.ReadFull(f, nam[:]); err != nil {
return "", err
}
if string(nam[:]) == "_buildid.o" {
sizeStr := strings.TrimSpace(string(hdr[0:20]))
size, err := strconv.ParseInt(sizeStr, 10, 64)
if err != nil {
return bad()
}
off += int64(len(hdr)) + namLen + 2
if off&1 != 0 {
off++
}
sr := io.NewSectionReader(f, off, size)
x, err := xcoff.NewFile(sr)
if err != nil {
return bad()
}
data := x.CSect(".go.buildid")
if data == nil {
return bad()
}
return string(data), nil
}
}
// Read next member offset.
offStr = strings.TrimSpace(string(hdr[20:40]))
off, err = strconv.ParseInt(offStr, 10, 64)
if err != nil {
return bad()
}
}
}
var (
goBuildPrefix = []byte("\xff Go build ID: \"")
goBuildEnd = []byte("\"\n \xff")

10
src/cmd/internal/obj/x86/issue19518_test.go
View file

@ -11,7 +11,6 @@ import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
@ -68,13 +67,8 @@ func objdumpOutput(t *testing.T) []byte {
testenv.GoToolPath(t), "build", "-o",
filepath.Join(tmpdir, "output"))
var env []string
for _, v := range os.Environ() {
if !strings.HasPrefix(v, "GOARCH=") {
env = append(env, v)
}
}
cmd.Env = append(env, "GOARCH=amd64")
cmd.Env = append(os.Environ(), "GOARCH=amd64", "GOOS=linux")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("error %s output %s", err, out)

8
src/cmd/internal/obj/x86/obj6_test.go
View file

@ -99,13 +99,7 @@ func asmOutput(t *testing.T, s string) []byte {
testenv.GoToolPath(t), "tool", "asm", "-S", "-dynlink",
"-o", filepath.Join(tmpdir, "output.6"), tmpfile.Name())
var env []string
for _, v := range os.Environ() {
if !strings.HasPrefix(v, "GOARCH=") {
env = append(env, v)
}
}
cmd.Env = append(env, "GOARCH=amd64")
cmd.Env = append(os.Environ(), "GOARCH=amd64", "GOOS=linux")
asmout, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("error %s output %s", err, asmout)

1
src/cmd/internal/objfile/objfile.go
View file

@ -61,6 +61,7 @@ var openers = []func(io.ReaderAt) (rawFile, error){
openMacho,
openPE,
openPlan9,
openXcoff,
}
// Open opens the named file.

133
src/cmd/internal/objfile/xcoff.go Normal file
View file

@ -0,0 +1,133 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Parsing of XCOFF executable (AIX)
package objfile
import (
"cmd/internal/xcoff"
"debug/dwarf"
"fmt"
"io"
"unicode"
)
type xcoffFile struct {
xcoff *xcoff.File
}
func openXcoff(r io.ReaderAt) (rawFile, error) {
f, err := xcoff.NewFile(r)
if err != nil {
return nil, err
}
return &xcoffFile{f}, nil
}
func (f *xcoffFile) symbols() ([]Sym, error) {
var syms []Sym
for _, s := range f.xcoff.Symbols {
const (
N_UNDEF = 0 // An undefined (extern) symbol
N_ABS = -1 // An absolute symbol (e_value is a constant, not an address)
N_DEBUG = -2 // A debugging symbol
)
sym := Sym{Name: s.Name, Addr: s.Value, Code: '?'}
switch s.SectionNumber {
case N_UNDEF:
sym.Code = 'U'
case N_ABS:
sym.Code = 'C'
case N_DEBUG:
sym.Code = '?'
default:
if s.SectionNumber < 0 || len(f.xcoff.Sections) < int(s.SectionNumber) {
return nil, fmt.Errorf("invalid section number in symbol table")
}
sect := f.xcoff.Sections[s.SectionNumber-1]
// debug/xcoff returns an offset in the section not the actual address
sym.Addr += sect.VirtualAddress
if s.AuxCSect.SymbolType&0x3 == xcoff.XTY_LD {
// The size of a function is contained in the
// AUX_FCN entry
sym.Size = s.AuxFcn.Size
} else {
sym.Size = s.AuxCSect.Length
}
sym.Size = s.AuxCSect.Length
switch sect.Type {
case xcoff.STYP_TEXT:
if s.AuxCSect.StorageMappingClass == xcoff.XMC_RO {
sym.Code = 'R'
} else {
sym.Code = 'T'
}
case xcoff.STYP_DATA:
sym.Code = 'D'
case xcoff.STYP_BSS:
sym.Code = 'B'
}
if s.StorageClass == xcoff.C_HIDEXT {
// Local symbol
sym.Code = unicode.ToLower(sym.Code)
}
}
syms = append(syms, sym)
}
return syms, nil
}
func (f *xcoffFile) pcln() (textStart uint64, symtab, pclntab []byte, err error) {
if sect := f.xcoff.Section(".text"); sect != nil {
textStart = sect.VirtualAddress
}
if sect := f.xcoff.Section(".gosymtab"); sect != nil {
if symtab, err = sect.Data(); err != nil {
return 0, nil, nil, err
}
}
if sect := f.xcoff.Section(".gopclntab"); sect != nil {
if pclntab, err = sect.Data(); err != nil {
return 0, nil, nil, err
}
}
return textStart, symtab, pclntab, nil
}
func (f *xcoffFile) text() (textStart uint64, text []byte, err error) {
sect := f.xcoff.Section(".text")
if sect == nil {
return 0, nil, fmt.Errorf("text section not found")
}
textStart = sect.VirtualAddress
text, err = sect.Data()
return
}
func (f *xcoffFile) goarch() string {
switch f.xcoff.TargetMachine {
case xcoff.U802TOCMAGIC:
return "ppc"
case xcoff.U64_TOCMAGIC:
return "ppc64"
}
return ""
}
func (f *xcoffFile) loadAddress() (uint64, error) {
return 0, fmt.Errorf("unknown load address")
}
func (f *xcoffFile) dwarf() (*dwarf.Data, error) {
return f.xcoff.DWARF()
}

7
src/cmd/link/internal/ld/lib.go
View file

@ -856,6 +856,13 @@ func loadobjfile(ctxt *Link, lib *sym.Library) {
continue
}
// Skip other special (non-object-file) sections that
// build tools may have added. Such sections must have
// short names so that the suffix is not truncated.
if len(arhdr.name) < 16 && !strings.HasSuffix(arhdr.name, ".o") {
continue
}
pname := fmt.Sprintf("%s(%s)", lib.File, arhdr.name)
l = atolwhex(arhdr.size)
ldobj(ctxt, f, lib, l, pname, lib.File)

46
src/cmd/link/link_test.go
View file

@ -6,6 +6,7 @@ import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
@ -70,3 +71,48 @@ func main() {}
t.Fatalf("failed to link main.o: %v, output: %s\n", err, out)
}
}
// TestIssue28429 ensures that the linker does not attempt to link
// sections not named *.o. Such sections may be used by a build system
// to, for example, save facts produced by a modular static analysis
// such as golang.org/x/tools/go/analysis.
func TestIssue28429(t *testing.T) {
testenv.MustHaveGoBuild(t)
tmpdir, err := ioutil.TempDir("", "issue28429-")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpdir)
write := func(name, content string) {
err := ioutil.WriteFile(filepath.Join(tmpdir, name), []byte(content), 0666)
if err != nil {
t.Fatal(err)
}
}
runGo := func(args ...string) {
cmd := exec.Command(testenv.GoToolPath(t), args...)
cmd.Dir = tmpdir
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("'go %s' failed: %v, output: %s",
strings.Join(args, " "), err, out)
}
}
// Compile a main package.
write("main.go", "package main; func main() {}")
runGo("tool", "compile", "-p", "main", "main.go")
runGo("tool", "pack", "c", "main.a", "main.o")
// Add an extra section with a short, non-.o name.
// This simulates an alternative build system.
write(".facts", "this is not an object file")
runGo("tool", "pack", "r", "main.a", ".facts")
// Verify that the linker does not attempt
// to compile the extra section.
runGo("tool", "link", "main.a")
}

37
src/cmd/nm/nm_test.go
View file

@ -56,17 +56,18 @@ func testMain(m *testing.M) int {
func TestNonGoExecs(t *testing.T) {
testfiles := []string{
"elf/testdata/gcc-386-freebsd-exec",
"elf/testdata/gcc-amd64-linux-exec",
"macho/testdata/gcc-386-darwin-exec",
"macho/testdata/gcc-amd64-darwin-exec",
// "pe/testdata/gcc-amd64-mingw-exec", // no symbols!
"pe/testdata/gcc-386-mingw-exec",
"plan9obj/testdata/amd64-plan9-exec",
"plan9obj/testdata/386-plan9-exec",
"debug/elf/testdata/gcc-386-freebsd-exec",
"debug/elf/testdata/gcc-amd64-linux-exec",
"debug/macho/testdata/gcc-386-darwin-exec",
"debug/macho/testdata/gcc-amd64-darwin-exec",
// "debug/pe/testdata/gcc-amd64-mingw-exec", // no symbols!
"debug/pe/testdata/gcc-386-mingw-exec",
"debug/plan9obj/testdata/amd64-plan9-exec",
"debug/plan9obj/testdata/386-plan9-exec",
"cmd/internal/xcoff/testdata/gcc-ppc64-aix-dwarf2-exec",
}
for _, f := range testfiles {
exepath := filepath.Join(runtime.GOROOT(), "src", "debug", f)
exepath := filepath.Join(runtime.GOROOT(), "src", f)
cmd := exec.Command(testnmpath, exepath)
out, err := cmd.CombinedOutput()
if err != nil {
@ -139,6 +140,20 @@ func testGoExec(t *testing.T, iscgo, isexternallinker bool) {
if err != nil {
t.Fatalf("go tool nm: %v\n%s", err, string(out))
}
relocated := func(code string) bool {
if runtime.GOOS == "aix" {
// On AIX, .data and .bss addresses are changed by the loader.
// Therefore, the values returned by the exec aren't the same
// than the ones inside the symbol table.
switch code {
case "D", "d", "B", "b":
return true
}
}
return false
}
scanner := bufio.NewScanner(bytes.NewBuffer(out))
dups := make(map[string]bool)
for scanner.Scan() {
@ -149,7 +164,9 @@ func testGoExec(t *testing.T, iscgo, isexternallinker bool) {
name := f[2]
if addr, found := names[name]; found {
if want, have := addr, "0x"+f[0]; have != want {
t.Errorf("want %s address for %s symbol, but have %s", want, name, have)
if !relocated(f[1]) {
t.Errorf("want %s address for %s symbol, but have %s", want, name, have)
}
}
delete(names, name)
}

7
src/cmd/trace/annotations.go
View file

@ -8,7 +8,6 @@ import (
"bytes"
"fmt"
"html/template"
"internal/trace"
"log"
"math"
"net/http"
@ -17,6 +16,8 @@ import (
"strconv"
"strings"
"time"
trace "internal/traceparser"
)
func init() {
@ -308,7 +309,7 @@ func analyzeAnnotations() (annotationAnalysisResult, error) {
}
}
// combine region info.
analyzeGoroutines(events)
analyzeGoroutines(res)
for goid, stats := range gs {
// gs is a global var defined in goroutines.go as a result
// of analyzeGoroutines. TODO(hyangah): fix this not to depend
@ -321,7 +322,7 @@ func analyzeAnnotations() (annotationAnalysisResult, error) {
}
var frame trace.Frame
if s.Start != nil {
frame = *s.Start.Stk[0]
frame = *res.Stacks[s.Start.StkID][0]
}
id := regionTypeID{Frame: frame, Type: s.Name}
regions[id] = append(regions[id], regionDesc{UserRegionDesc: s, G: goid})

14
src/cmd/trace/annotations_test.go
View file

@ -11,7 +11,7 @@ import (
"context"
"flag"
"fmt"
traceparser "internal/trace"
"internal/traceparser"
"io/ioutil"
"reflect"
"runtime/debug"
@ -338,10 +338,8 @@ func traceProgram(t *testing.T, f func(), name string) error {
trace.Stop()
saveTrace(buf, name)
res, err := traceparser.Parse(buf, name+".faketrace")
if err == traceparser.ErrTimeOrder {
t.Skipf("skipping due to golang.org/issue/16755: %v", err)
} else if err != nil {
res, err := traceparser.ParseBuffer(buf)
if err != nil {
return err
}
@ -370,15 +368,15 @@ func childrenNames(task *taskDesc) (ret []string) {
return ret
}
func swapLoaderData(res traceparser.ParseResult, err error) {
func swapLoaderData(res *traceparser.Parsed, err error) {
// swap loader's data.
parseTrace() // fool loader.once.
loader.res = res
loader.err = err
analyzeGoroutines(nil) // fool gsInit once.
gs = traceparser.GoroutineStats(res.Events)
analyzeGoroutines(res) // fool gsInit once.
gs = res.GoroutineStats()
}

11
src/cmd/trace/goroutines.go
View file

@ -9,7 +9,6 @@ package main
import (
"fmt"
"html/template"
"internal/trace"
"log"
"net/http"
"reflect"
@ -17,6 +16,8 @@ import (
"strconv"
"sync"
"time"
trace "internal/traceparser"
)
func init() {
@ -38,15 +39,15 @@ var (
)
// analyzeGoroutines generates statistics about execution of all goroutines and stores them in gs.
func analyzeGoroutines(events []*trace.Event) {
func analyzeGoroutines(res *trace.Parsed) {
gsInit.Do(func() {
gs = trace.GoroutineStats(events)
gs = res.GoroutineStats()
})
}
// httpGoroutines serves list of goroutine groups.
func httpGoroutines(w http.ResponseWriter, r *http.Request) {
events, err := parseEvents()
events, err := parseTrace()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
@ -89,7 +90,7 @@ Goroutines: <br>
func httpGoroutine(w http.ResponseWriter, r *http.Request) {
// TODO(hyangah): support format=csv (raw data)
events, err := parseEvents()
events, err := parseTrace()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return

49
src/cmd/trace/main.go
View file

@ -5,12 +5,12 @@
package main
import (
"bufio"
"bytes"
"cmd/internal/browser"
"flag"
"fmt"
"html/template"
"internal/trace"
trace "internal/traceparser"
"io"
"log"
"net"
@ -115,8 +115,22 @@ func main() {
dief("%v\n", err)
}
if *debugFlag {
trace.Print(res.Events)
if *debugFlag { // match go tool trace -d (except for Offset and Seq)
f := func(ev *trace.Event) {
desc := trace.EventDescriptions[ev.Type]
w := new(bytes.Buffer)
fmt.Fprintf(w, "%v %v p=%v g=%v", ev.Ts, desc.Name, ev.P, ev.G)
for i, a := range desc.Args {
fmt.Fprintf(w, " %v=%v", a, ev.Args[i])
}
for i, a := range desc.SArgs {
fmt.Fprintf(w, " %v=%v", a, ev.SArgs[i])
}
fmt.Println(w.String())
}
for i := 0; i < len(res.Events); i++ {
f(res.Events[i])
}
os.Exit(0)
}
reportMemoryUsage("after parsing trace")
@ -141,36 +155,23 @@ var ranges []Range
var loader struct {
once sync.Once
res trace.ParseResult
res *trace.Parsed
err error
}
// parseEvents is a compatibility wrapper that returns only
// the Events part of trace.ParseResult returned by parseTrace.
func parseEvents() ([]*trace.Event, error) {
res, err := parseTrace()
if err != nil {
return nil, err
}
return res.Events, err
}
func parseTrace() (trace.ParseResult, error) {
func parseTrace() (*trace.Parsed, error) {
loader.once.Do(func() {
tracef, err := os.Open(traceFile)
x, err := trace.New(traceFile)
if err != nil {
loader.err = fmt.Errorf("failed to open trace file: %v", err)
loader.err = err
return
}
defer tracef.Close()
// Parse and symbolize.
res, err := trace.Parse(bufio.NewReader(tracef), programBinary)
err = x.Parse(0, x.MaxTs, nil)
if err != nil {
loader.err = fmt.Errorf("failed to parse trace: %v", err)
loader.err = err
return
}
loader.res = res
loader.res = x
})
return loader.res, loader.err
}

64
src/cmd/trace/pprof.go
View file

@ -9,7 +9,6 @@ package main
import (
"bufio"
"fmt"
"internal/trace"
"io"
"io/ioutil"
"net/http"
@ -21,6 +20,8 @@ import (
"strconv"
"time"
trace "internal/traceparser"
"github.com/google/pprof/profile"
)
@ -60,22 +61,22 @@ type interval struct {
begin, end int64 // nanoseconds.
}
func pprofByGoroutine(compute func(io.Writer, map[uint64][]interval, []*trace.Event) error) func(w io.Writer, r *http.Request) error {
func pprofByGoroutine(compute func(io.Writer, map[uint64][]interval, *trace.Parsed) error) func(w io.Writer, r *http.Request) error {
return func(w io.Writer, r *http.Request) error {
id := r.FormValue("id")
events, err := parseEvents()
res, err := parseTrace()
if err != nil {
return err
}
gToIntervals, err := pprofMatchingGoroutines(id, events)
gToIntervals, err := pprofMatchingGoroutines(id, res)
if err != nil {
return err
}
return compute(w, gToIntervals, events)
return compute(w, gToIntervals, res)
}
}
func pprofByRegion(compute func(io.Writer, map[uint64][]interval, []*trace.Event) error) func(w io.Writer, r *http.Request) error {
func pprofByRegion(compute func(io.Writer, map[uint64][]interval, *trace.Parsed) error) func(w io.Writer, r *http.Request) error {
return func(w io.Writer, r *http.Request) error {
filter, err := newRegionFilter(r)
if err != nil {
@ -85,7 +86,7 @@ func pprofByRegion(compute func(io.Writer, map[uint64][]interval, []*trace.Event
if err != nil {
return err
}
events, _ := parseEvents()
events, _ := parseTrace()
return compute(w, gToIntervals, events)
}
@ -94,7 +95,7 @@ func pprofByRegion(compute func(io.Writer, map[uint64][]interval, []*trace.Event
// pprofMatchingGoroutines parses the goroutine type id string (i.e. pc)
// and returns the ids of goroutines of the matching type and its interval.
// If the id string is empty, returns nil without an error.
func pprofMatchingGoroutines(id string, events []*trace.Event) (map[uint64][]interval, error) {
func pprofMatchingGoroutines(id string, p *trace.Parsed) (map[uint64][]interval, error) {
if id == "" {
return nil, nil
}
@ -102,7 +103,7 @@ func pprofMatchingGoroutines(id string, events []*trace.Event) (map[uint64][]int
if err != nil {
return nil, fmt.Errorf("invalid goroutine type: %v", id)
}
analyzeGoroutines(events)
analyzeGoroutines(p)
var res map[uint64][]interval
for _, g := range gs {
if g.PC != pc {
@ -171,17 +172,25 @@ func pprofMatchingRegions(filter *regionFilter) (map[uint64][]interval, error) {
return gToIntervals, nil
}
func stklen(p *trace.Parsed, ev *trace.Event) int {
if ev.StkID == 0 {
return 0
}
return len(p.Stacks[ev.StkID])
}
// computePprofIO generates IO pprof-like profile (time spent in IO wait, currently only network blocking event).
func computePprofIO(w io.Writer, gToIntervals map[uint64][]interval, events []*trace.Event) error {
prof := make(map[uint64]Record)
func computePprofIO(w io.Writer, gToIntervals map[uint64][]interval, res *trace.Parsed) error {
events := res.Events
prof := make(map[uint32]Record)
for _, ev := range events {
if ev.Type != trace.EvGoBlockNet || ev.Link == nil || ev.StkID == 0 || len(ev.Stk) == 0 {
if ev.Type != trace.EvGoBlockNet || ev.Link == nil || ev.StkID == 0 || stklen(res, ev) == 0 {
continue
}
overlapping := pprofOverlappingDuration(gToIntervals, ev)
if overlapping > 0 {
rec := prof[ev.StkID]
rec.stk = ev.Stk
rec.stk = res.Stacks[ev.StkID]
rec.n++
rec.time += overlapping.Nanoseconds()
prof[ev.StkID] = rec
@ -191,8 +200,9 @@ func computePprofIO(w io.Writer, gToIntervals map[uint64][]interval, events []*t
}
// computePprofBlock generates blocking pprof-like profile (time spent blocked on synchronization primitives).
func computePprofBlock(w io.Writer, gToIntervals map[uint64][]interval, events []*trace.Event) error {
prof := make(map[uint64]Record)
func computePprofBlock(w io.Writer, gToIntervals map[uint64][]interval, res *trace.Parsed) error {
events := res.Events
prof := make(map[uint32]Record)
for _, ev := range events {
switch ev.Type {
case trace.EvGoBlockSend, trace.EvGoBlockRecv, trace.EvGoBlockSelect,
@ -203,13 +213,13 @@ func computePprofBlock(w io.Writer, gToIntervals map[uint64][]interval, events [
default:
continue
}
if ev.Link == nil || ev.StkID == 0 || len(ev.Stk) == 0 {
if ev.Link == nil || ev.StkID == 0 || stklen(res, ev) == 0 {
continue
}
overlapping := pprofOverlappingDuration(gToIntervals, ev)
if overlapping > 0 {
rec := prof[ev.StkID]
rec.stk = ev.Stk
rec.stk = res.Stacks[ev.StkID]
rec.n++
rec.time += overlapping.Nanoseconds()
prof[ev.StkID] = rec
@ -219,16 +229,17 @@ func computePprofBlock(w io.Writer, gToIntervals map[uint64][]interval, events [
}
// computePprofSyscall generates syscall pprof-like profile (time spent blocked in syscalls).
func computePprofSyscall(w io.Writer, gToIntervals map[uint64][]interval, events []*trace.Event) error {
prof := make(map[uint64]Record)
func computePprofSyscall(w io.Writer, gToIntervals map[uint64][]interval, res *trace.Parsed) error {
events := res.Events
prof := make(map[uint32]Record)
for _, ev := range events {
if ev.Type != trace.EvGoSysCall || ev.Link == nil || ev.StkID == 0 || len(ev.Stk) == 0 {
if ev.Type != trace.EvGoSysCall || ev.Link == nil || ev.StkID == 0 || stklen(res, ev) == 0 {
continue
}
overlapping := pprofOverlappingDuration(gToIntervals, ev)
if overlapping > 0 {
rec := prof[ev.StkID]
rec.stk = ev.Stk
rec.stk = res.Stacks[ev.StkID]
rec.n++
rec.time += overlapping.Nanoseconds()
prof[ev.StkID] = rec
@ -239,17 +250,18 @@ func computePprofSyscall(w io.Writer, gToIntervals map[uint64][]interval, events
// computePprofSched generates scheduler latency pprof-like profile
// (time between a goroutine become runnable and actually scheduled for execution).
func computePprofSched(w io.Writer, gToIntervals map[uint64][]interval, events []*trace.Event) error {
prof := make(map[uint64]Record)
func computePprofSched(w io.Writer, gToIntervals map[uint64][]interval, res *trace.Parsed) error {
events := res.Events
prof := make(map[uint32]Record)
for _, ev := range events {
if (ev.Type != trace.EvGoUnblock && ev.Type != trace.EvGoCreate) ||
ev.Link == nil || ev.StkID == 0 || len(ev.Stk) == 0 {
ev.Link == nil || ev.StkID == 0 || stklen(res, ev) == 0 {
continue
}
overlapping := pprofOverlappingDuration(gToIntervals, ev)
if overlapping > 0 {
rec := prof[ev.StkID]
rec.stk = ev.Stk
rec.stk = res.Stacks[ev.StkID]
rec.n++
rec.time += overlapping.Nanoseconds()
prof[ev.StkID] = rec
@ -327,7 +339,7 @@ func serveSVGProfile(prof func(w io.Writer, r *http.Request) error) http.Handler
}
}
func buildProfile(prof map[uint64]Record) *profile.Profile {
func buildProfile(prof map[uint32]Record) *profile.Profile {
p := &profile.Profile{
PeriodType: &profile.ValueType{Type: "trace", Unit: "count"},
Period: 1,

168
src/cmd/trace/trace.go
View file

@ -7,7 +7,7 @@ package main
import (
"encoding/json"
"fmt"
"internal/trace"
trace "internal/traceparser"
"io"
"log"
"math"
@ -23,7 +23,7 @@ import (
func init() {
http.HandleFunc("/trace", httpTrace)
http.HandleFunc("/jsontrace", httpJsonTrace)
http.HandleFunc("/jsontrace", httpJSONTrace)
http.HandleFunc("/trace_viewer_html", httpTraceViewerHTML)
}
@ -38,7 +38,7 @@ func httpTrace(w http.ResponseWriter, r *http.Request) {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
html := strings.ReplaceAll(templTrace, "{{PARAMS}}", r.Form.Encode())
html := strings.Replace(templTrace, "{{PARAMS}}", r.Form.Encode(), -1)
w.Write([]byte(html))
}
@ -165,8 +165,8 @@ func httpTraceViewerHTML(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, filepath.Join(runtime.GOROOT(), "misc", "trace", "trace_viewer_full.html"))
}
// httpJsonTrace serves json trace, requested from within templTrace HTML.
func httpJsonTrace(w http.ResponseWriter, r *http.Request) {
// httpJSONTrace serves json trace, requested from within templTrace HTML.
func httpJSONTrace(w http.ResponseWriter, r *http.Request) {
defer debug.FreeOSMemory()
defer reportMemoryUsage("after httpJsonTrace")
// This is an AJAX handler, so instead of http.Error we use log.Printf to log errors.
@ -188,7 +188,7 @@ func httpJsonTrace(w http.ResponseWriter, r *http.Request) {
log.Printf("failed to parse goid parameter %q: %v", goids, err)
return
}
analyzeGoroutines(res.Events)
analyzeGoroutines(res)
g, ok := gs[goid]
if !ok {
log.Printf("failed to find goroutine %d", goid)
@ -202,7 +202,7 @@ func httpJsonTrace(w http.ResponseWriter, r *http.Request) {
params.endTime = lastTimestamp()
}
params.maing = goid
params.gs = trace.RelatedGoroutines(res.Events, goid)
params.gs = res.RelatedGoroutines(goid)
} else if taskids := r.FormValue("taskid"); taskids != "" {
taskid, err := strconv.ParseUint(taskids, 10, 64)
if err != nil {
@ -264,12 +264,13 @@ func httpJsonTrace(w http.ResponseWriter, r *http.Request) {
}
c := viewerDataTraceConsumer(w, start, end)
if err := generateTrace(params, c); err != nil {
if err := generateTrace(res, params, c); err != nil {
log.Printf("failed to generate trace: %v", err)
return
}
}
// Range is a named range
type Range struct {
Name string
Start int
@ -279,13 +280,13 @@ type Range struct {
// splitTrace splits the trace into a number of ranges,
// each resulting in approx 100MB of json output
// (trace viewer can hardly handle more).
func splitTrace(res trace.ParseResult) []Range {
func splitTrace(res *trace.Parsed) []Range {
params := &traceParams{
parsed: res,
endTime: math.MaxInt64,
}
s, c := splittingTraceConsumer(100 << 20) // 100M
if err := generateTrace(params, c); err != nil {
if err := generateTrace(res, params, c); err != nil {
dief("%v\n", err)
}
return s.Ranges
@ -302,7 +303,7 @@ func splittingTraceConsumer(max int) (*splitter, traceConsumer) {
}
var (
data = ViewerData{Frames: make(map[string]ViewerFrame)}
data = viewerData{Frames: make(map[string]viewerFrame)}
sizes []eventSz
cw countingWriter
@ -314,7 +315,7 @@ func splittingTraceConsumer(max int) (*splitter, traceConsumer) {
consumeTimeUnit: func(unit string) {
data.TimeUnit = unit
},
consumeViewerEvent: func(v *ViewerEvent, required bool) {
consumeViewerEvent: func(v *viewerEvent, required bool) {
if required {
// Store required events inside data
// so flush can include them in the required
@ -327,7 +328,7 @@ func splittingTraceConsumer(max int) (*splitter, traceConsumer) {
sizes = append(sizes, eventSz{v.Time, cw.size + 1}) // +1 for ",".
cw.size = 0
},
consumeViewerFrame: func(k string, v ViewerFrame) {
consumeViewerFrame: func(k string, v viewerFrame) {
data.Frames[k] = v
},
flush: func() {
@ -382,7 +383,7 @@ func (cw *countingWriter) Write(data []byte) (int, error) {
}
type traceParams struct {
parsed trace.ParseResult
parsed *trace.Parsed
mode traceviewMode
startTime int64
endTime int64
@ -399,6 +400,7 @@ const (
)
type traceContext struct {
res *trace.Parsed
*traceParams
consumer traceConsumer
frameTree frameNode
@ -449,16 +451,16 @@ type gInfo struct {
markAssist *trace.Event // if non-nil, the mark assist currently running.
}
type ViewerData struct {
Events []*ViewerEvent `json:"traceEvents"`
Frames map[string]ViewerFrame `json:"stackFrames"`
type viewerData struct {
Events []*viewerEvent `json:"traceEvents"`
Frames map[string]viewerFrame `json:"stackFrames"`
TimeUnit string `json:"displayTimeUnit"`
// This is where mandatory part of the trace starts (e.g. thread names)
footer int
}
type ViewerEvent struct {
type viewerEvent struct {
Name string `json:"name,omitempty"`
Phase string `json:"ph"`
Scope string `json:"s,omitempty"`
@ -474,33 +476,33 @@ type ViewerEvent struct {
Category string `json:"cat,omitempty"`
}
type ViewerFrame struct {
type viewerFrame struct {
Name string `json:"name"`
Parent int `json:"parent,omitempty"`
}
type NameArg struct {
type nameArg struct {
Name string `json:"name"`
}
type TaskArg struct {
type taskArg struct {
ID uint64 `json:"id"`
StartG uint64 `json:"start_g,omitempty"`
EndG uint64 `json:"end_g,omitempty"`
}
type RegionArg struct {
type regionArg struct {
TaskID uint64 `json:"taskid,omitempty"`
}
type SortIndexArg struct {
type sortIndexArg struct {
Index int `json:"sort_index"`
}
type traceConsumer struct {
consumeTimeUnit func(unit string)
consumeViewerEvent func(v *ViewerEvent, required bool)
consumeViewerFrame func(key string, f ViewerFrame)
consumeViewerEvent func(v *viewerEvent, required bool)
consumeViewerFrame func(key string, f viewerFrame)
flush func()
}
@ -517,15 +519,15 @@ const (
// If mode==goroutineMode, generate trace for goroutine goid, otherwise whole trace.
// startTime, endTime determine part of the trace that we are interested in.
// gset restricts goroutines that are included in the resulting trace.
func generateTrace(params *traceParams, consumer traceConsumer) error {
func generateTrace(res *trace.Parsed, params *traceParams, consumer traceConsumer) error {
defer consumer.flush()
ctx := &traceContext{traceParams: params}
ctx := &traceContext{res: res, traceParams: params}
ctx.frameTree.children = make(map[uint64]frameNode)
ctx.consumer = consumer
ctx.consumer.consumeTimeUnit("ns")
maxProc := 0
maxProc := int32(0)
ginfos := make(map[uint64]*gInfo)
stacks := params.parsed.Stacks
@ -570,12 +572,12 @@ func generateTrace(params *traceParams, consumer traceConsumer) error {
newG := ev.Args[0]
info := getGInfo(newG)
if info.name != "" {
return fmt.Errorf("duplicate go create event for go id=%d detected at offset %d", newG, ev.Off)
return fmt.Errorf("duplicate go create event for go id=%d detected at time %d", newG, ev.Ts)
}
stk, ok := stacks[ev.Args[1]]
stk, ok := stacks[uint32(ev.Args[1])]
if !ok || len(stk) == 0 {
return fmt.Errorf("invalid go create event: missing stack information for go id=%d at offset %d", newG, ev.Off)
return fmt.Errorf("invalid go create event: missing stack information for go id=%d at time %d", newG, ev.Ts)
}
fname := stk[0].Fn
@ -746,23 +748,23 @@ func generateTrace(params *traceParams, consumer traceConsumer) error {
ctx.emitSectionFooter(procsSection, "PROCS", 2)
}
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.GCP, Arg: &NameArg{"GC"}})
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.GCP, Arg: &SortIndexArg{-6}})
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.GCP, Arg: &nameArg{"GC"}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.GCP, Arg: &sortIndexArg{-6}})
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.NetpollP, Arg: &NameArg{"Network"}})
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.NetpollP, Arg: &SortIndexArg{-5}})
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.NetpollP, Arg: &nameArg{"Network"}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.NetpollP, Arg: &sortIndexArg{-5}})
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.TimerP, Arg: &NameArg{"Timers"}})
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.TimerP, Arg: &SortIndexArg{-4}})
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.TimerP, Arg: &nameArg{"Timers"}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.TimerP, Arg: &sortIndexArg{-4}})
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.SyscallP, Arg: &NameArg{"Syscalls"}})
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.SyscallP, Arg: &SortIndexArg{-3}})
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: trace.SyscallP, Arg: &nameArg{"Syscalls"}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: trace.SyscallP, Arg: &sortIndexArg{-3}})
// Display rows for Ps if we are in the default trace view mode (not goroutine-oriented presentation)
if ctx.mode&modeGoroutineOriented == 0 {
for i := 0; i <= maxProc; i++ {
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: uint64(i), Arg: &NameArg{fmt.Sprintf("Proc %v", i)}})
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: uint64(i), Arg: &SortIndexArg{i}})
for i := 0; i <= int(maxProc); i++ {
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: uint64(i), Arg: &nameArg{fmt.Sprintf("Proc %v", i)}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: uint64(i), Arg: &sortIndexArg{i}})
}
}
@ -800,27 +802,27 @@ func generateTrace(params *traceParams, consumer traceConsumer) error {
if !ctx.gs[k] {
continue
}
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: k, Arg: &NameArg{v.name}})
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: procsSection, Tid: k, Arg: &nameArg{v.name}})
}
// Row for the main goroutine (maing)
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: ctx.maing, Arg: &SortIndexArg{-2}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: ctx.maing, Arg: &sortIndexArg{-2}})
// Row for GC or global state (specified with G=0)
ctx.emitFooter(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: 0, Arg: &SortIndexArg{-1}})
ctx.emitFooter(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: procsSection, Tid: 0, Arg: &sortIndexArg{-1}})
}
return nil
}
func (ctx *traceContext) emit(e *ViewerEvent) {
func (ctx *traceContext) emit(e *viewerEvent) {
ctx.consumer.consumeViewerEvent(e, false)
}
func (ctx *traceContext) emitFooter(e *ViewerEvent) {
func (ctx *traceContext) emitFooter(e *viewerEvent) {
ctx.consumer.consumeViewerEvent(e, true)
}
func (ctx *traceContext) emitSectionFooter(sectionID uint64, name string, priority int) {
ctx.emitFooter(&ViewerEvent{Name: "process_name", Phase: "M", Pid: sectionID, Arg: &NameArg{name}})
ctx.emitFooter(&ViewerEvent{Name: "process_sort_index", Phase: "M", Pid: sectionID, Arg: &SortIndexArg{priority}})
ctx.emitFooter(&viewerEvent{Name: "process_name", Phase: "M", Pid: sectionID, Arg: &nameArg{name}})
ctx.emitFooter(&viewerEvent{Name: "process_sort_index", Phase: "M", Pid: sectionID, Arg: &sortIndexArg{priority}})
}
func (ctx *traceContext) time(ev *trace.Event) float64 {
@ -842,31 +844,30 @@ func tsWithinRange(ts, s, e int64) bool {
func (ctx *traceContext) proc(ev *trace.Event) uint64 {
if ctx.mode&modeGoroutineOriented != 0 && ev.P < trace.FakeP {
return ev.G
} else {
return uint64(ev.P)
}
return uint64(ev.P)
}
func (ctx *traceContext) emitSlice(ev *trace.Event, name string) {
ctx.emit(ctx.makeSlice(ev, name))
}
func (ctx *traceContext) makeSlice(ev *trace.Event, name string) *ViewerEvent {
// If ViewerEvent.Dur is not a positive value,
func (ctx *traceContext) makeSlice(ev *trace.Event, name string) *viewerEvent {
// If viewerEvent.Dur is not a positive value,
// trace viewer handles it as a non-terminating time interval.
// Avoid it by setting the field with a small value.
durationUsec := ctx.time(ev.Link) - ctx.time(ev)
if ev.Link.Ts-ev.Ts <= 0 {
if ev.Link == nil || ev.Link.Ts-ev.Ts <= 0 {
durationUsec = 0.0001 // 0.1 nanoseconds
}
sl := &ViewerEvent{
sl := &viewerEvent{
Name: name,
Phase: "X",
Time: ctx.time(ev),
Dur: durationUsec,
Tid: ctx.proc(ev),
Stack: ctx.stack(ev.Stk),
EndStack: ctx.stack(ev.Link.Stk),
Stack: ctx.stack(ctx.res.Stacks[ev.StkID]),
EndStack: ctx.stack(ctx.res.Stacks[ev.Link.StkID]),
}
// grey out non-overlapping events if the event is not a global event (ev.G == 0)
@ -876,7 +877,7 @@ func (ctx *traceContext) makeSlice(ev *trace.Event, name string) *ViewerEvent {
type Arg struct {
P int
}
sl.Arg = &Arg{P: ev.P}
sl.Arg = &Arg{P: int(ev.P)}
}
// grey out non-overlapping events.
overlapping := false
@ -898,10 +899,10 @@ func (ctx *traceContext) emitTask(task *taskDesc, sortIndex int) {
taskName := task.name
durationUsec := float64(task.lastTimestamp()-task.firstTimestamp()) / 1e3
ctx.emitFooter(&ViewerEvent{Name: "thread_name", Phase: "M", Pid: tasksSection, Tid: taskRow, Arg: &NameArg{fmt.Sprintf("T%d %s", task.id, taskName)}})
ctx.emit(&ViewerEvent{Name: "thread_sort_index", Phase: "M", Pid: tasksSection, Tid: taskRow, Arg: &SortIndexArg{sortIndex}})
ctx.emitFooter(&viewerEvent{Name: "thread_name", Phase: "M", Pid: tasksSection, Tid: taskRow, Arg: &nameArg{fmt.Sprintf("T%d %s", task.id, taskName)}})
ctx.emit(&viewerEvent{Name: "thread_sort_index", Phase: "M", Pid: tasksSection, Tid: taskRow, Arg: &sortIndexArg{sortIndex}})
ts := float64(task.firstTimestamp()) / 1e3
sl := &ViewerEvent{
sl := &viewerEvent{
Name: taskName,
Phase: "X",
Time: ts,
@ -910,13 +911,13 @@ func (ctx *traceContext) emitTask(task *taskDesc, sortIndex int) {
Tid: taskRow,
Cname: pickTaskColor(task.id),
}
targ := TaskArg{ID: task.id}
targ := taskArg{ID: task.id}
if task.create != nil {
sl.Stack = ctx.stack(task.create.Stk)
sl.Stack = ctx.stack(ctx.res.Stacks[task.create.StkID])
targ.StartG = task.create.G
}
if task.end != nil {
sl.EndStack = ctx.stack(task.end.Stk)
sl.EndStack = ctx.stack(ctx.res.Stacks[task.end.StkID])
targ.EndG = task.end.G
}
sl.Arg = targ
@ -924,8 +925,8 @@ func (ctx *traceContext) emitTask(task *taskDesc, sortIndex int) {
if task.create != nil && task.create.Type == trace.EvUserTaskCreate && task.create.Args[1] != 0 {
ctx.arrowSeq++
ctx.emit(&ViewerEvent{Name: "newTask", Phase: "s", Tid: task.create.Args[1], ID: ctx.arrowSeq, Time: ts, Pid: tasksSection})
ctx.emit(&ViewerEvent{Name: "newTask", Phase: "t", Tid: taskRow, ID: ctx.arrowSeq, Time: ts, Pid: tasksSection})
ctx.emit(&viewerEvent{Name: "newTask", Phase: "s", Tid: task.create.Args[1], ID: ctx.arrowSeq, Time: ts, Pid: tasksSection})
ctx.emit(&viewerEvent{Name: "newTask", Phase: "t", Tid: taskRow, ID: ctx.arrowSeq, Time: ts, Pid: tasksSection})
}
}
@ -946,7 +947,7 @@ func (ctx *traceContext) emitRegion(s regionDesc) {
scopeID := fmt.Sprintf("%x", id)
name := s.Name
sl0 := &ViewerEvent{
sl0 := &viewerEvent{
Category: "Region",
Name: name,
Phase: "b",
@ -957,11 +958,11 @@ func (ctx *traceContext) emitRegion(s regionDesc) {
Cname: pickTaskColor(s.TaskID),
}
if s.Start != nil {
sl0.Stack = ctx.stack(s.Start.Stk)
sl0.Stack = ctx.stack(ctx.res.Stacks[s.Start.StkID])
}
ctx.emit(sl0)
sl1 := &ViewerEvent{
sl1 := &viewerEvent{
Category: "Region",
Name: name,
Phase: "e",
@ -970,10 +971,10 @@ func (ctx *traceContext) emitRegion(s regionDesc) {
ID: uint64(regionID),
Scope: scopeID,
Cname: pickTaskColor(s.TaskID),
Arg: RegionArg{TaskID: s.TaskID},
Arg: regionArg{TaskID: s.TaskID},
}
if s.End != nil {
sl1.Stack = ctx.stack(s.End.Stk)
sl1.Stack = ctx.stack(ctx.res.Stacks[s.End.StkID])
}
ctx.emit(sl1)
}
@ -992,7 +993,7 @@ func (ctx *traceContext) emitHeapCounters(ev *trace.Event) {
diff = ctx.heapStats.nextGC - ctx.heapStats.heapAlloc
}
if tsWithinRange(ev.Ts, ctx.startTime, ctx.endTime) {
ctx.emit(&ViewerEvent{Name: "Heap", Phase: "C", Time: ctx.time(ev), Pid: 1, Arg: &heapCountersArg{ctx.heapStats.heapAlloc, diff}})
ctx.emit(&viewerEvent{Name: "Heap", Phase: "C", Time: ctx.time(ev), Pid: 1, Arg: &heapCountersArg{ctx.heapStats.heapAlloc, diff}})
}
ctx.prevHeapStats = ctx.heapStats
}
@ -1008,7 +1009,7 @@ func (ctx *traceContext) emitGoroutineCounters(ev *trace.Event) {
return
}
if tsWithinRange(ev.Ts, ctx.startTime, ctx.endTime) {
ctx.emit(&ViewerEvent{Name: "Goroutines", Phase: "C", Time: ctx.time(ev), Pid: 1, Arg: &goroutineCountersArg{uint64(ctx.gstates[gRunning]), uint64(ctx.gstates[gRunnable]), uint64(ctx.gstates[gWaitingGC])}})
ctx.emit(&viewerEvent{Name: "Goroutines", Phase: "C", Time: ctx.time(ev), Pid: 1, Arg: &goroutineCountersArg{uint64(ctx.gstates[gRunning]), uint64(ctx.gstates[gRunnable]), uint64(ctx.gstates[gWaitingGC])}})
}
ctx.prevGstates = ctx.gstates
}
@ -1023,7 +1024,7 @@ func (ctx *traceContext) emitThreadCounters(ev *trace.Event) {
return
}
if tsWithinRange(ev.Ts, ctx.startTime, ctx.endTime) {
ctx.emit(&ViewerEvent{Name: "Threads", Phase: "C", Time: ctx.time(ev), Pid: 1, Arg: &threadCountersArg{
ctx.emit(&viewerEvent{Name: "Threads", Phase: "C", Time: ctx.time(ev), Pid: 1, Arg: &threadCountersArg{
Running: ctx.threadStats.prunning,
InSyscall: ctx.threadStats.insyscall}})
}
@ -1061,14 +1062,14 @@ func (ctx *traceContext) emitInstant(ev *trace.Event, name, category string) {
}
arg = &Arg{ev.Args[0]}
}
ctx.emit(&ViewerEvent{
ctx.emit(&viewerEvent{
Name: name,
Category: category,
Phase: "I",
Scope: "t",
Time: ctx.time(ev),
Tid: ctx.proc(ev),
Stack: ctx.stack(ev.Stk),
Stack: ctx.stack(ctx.res.Stacks[ev.StkID]),
Cname: cname,
Arg: arg})
}
@ -1105,8 +1106,11 @@ func (ctx *traceContext) emitArrow(ev *trace.Event, name string) {
}
ctx.arrowSeq++
ctx.emit(&ViewerEvent{Name: name, Phase: "s", Tid: ctx.proc(ev), ID: ctx.arrowSeq, Time: ctx.time(ev), Stack: ctx.stack(ev.Stk), Cname: color})
ctx.emit(&ViewerEvent{Name: name, Phase: "t", Tid: ctx.proc(ev.Link), ID: ctx.arrowSeq, Time: ctx.time(ev.Link), Cname: color})
ctx.emit(&viewerEvent{Name: name, Phase: "s", Tid: ctx.proc(ev),
ID: ctx.arrowSeq, Time: ctx.time(ev),
Stack: ctx.stack(ctx.res.Stacks[ev.StkID]), Cname: color})
ctx.emit(&viewerEvent{Name: name, Phase: "t", Tid: ctx.proc(ev.Link),
ID: ctx.arrowSeq, Time: ctx.time(ev.Link), Cname: color})
}
func (ctx *traceContext) stack(stk []*trace.Frame) int {
@ -1128,7 +1132,7 @@ func (ctx *traceContext) buildBranch(parent frameNode, stk []*trace.Frame) int {
node.id = ctx.frameSeq
node.children = make(map[uint64]frameNode)
parent.children[frame.PC] = node
ctx.consumer.consumeViewerFrame(strconv.Itoa(node.id), ViewerFrame{fmt.Sprintf("%v:%v", frame.Fn, frame.Line), parent.id})
ctx.consumer.consumeViewerFrame(strconv.Itoa(node.id), viewerFrame{fmt.Sprintf("%v:%v", frame.Fn, frame.Line), parent.id})
}
return ctx.buildBranch(node, stk)
}
@ -1163,7 +1167,7 @@ type jsonWriter struct {
}
func viewerDataTraceConsumer(w io.Writer, start, end int64) traceConsumer {
frames := make(map[string]ViewerFrame)
frames := make(map[string]viewerFrame)
enc := json.NewEncoder(w)
written := 0
index := int64(-1)
@ -1175,7 +1179,7 @@ func viewerDataTraceConsumer(w io.Writer, start, end int64) traceConsumer {
enc.Encode(unit)
io.WriteString(w, ",")
},
consumeViewerEvent: func(v *ViewerEvent, required bool) {
consumeViewerEvent: func(v *viewerEvent, required bool) {
index++
if !required && (index < start || index > end) {
// not in the range. Skip!
@ -1192,7 +1196,7 @@ func viewerDataTraceConsumer(w io.Writer, start, end int64) traceConsumer {
// Same should be applied to splittingTraceConsumer.
written++
},
consumeViewerFrame: func(k string, v ViewerFrame) {
consumeViewerFrame: func(k string, v viewerFrame) {
frames[k] = v
},
flush: func() {

46
src/cmd/trace/trace_test.go
View file

@ -8,26 +8,27 @@ package main
import (
"context"
"internal/trace"
"io/ioutil"
rtrace "runtime/trace"
"strings"
"testing"
trace "internal/traceparser"
)
// stacks is a fake stack map populated for test.
type stacks map[uint64][]*trace.Frame
type stacks map[uint32][]*trace.Frame
// add adds a stack with a single frame whose Fn field is
// set to the provided fname and returns a unique stack id.
func (s *stacks) add(fname string) uint64 {
if *s == nil {
*s = make(map[uint64][]*trace.Frame)
*s = make(map[uint32][]*trace.Frame)
}
id := uint64(len(*s))
id := uint32(len(*s))
(*s)[id] = []*trace.Frame{{Fn: fname}}
return id
return uint64(id)
}
// TestGoroutineCount tests runnable/running goroutine counts computed by generateTrace
@ -36,8 +37,7 @@ func (s *stacks) add(fname string) uint64 {
// - the counts must not include goroutines blocked waiting on channels or in syscall.
func TestGoroutineCount(t *testing.T) {
w := trace.NewWriter()
w.Emit(trace.EvBatch, 0, 0) // start of per-P batch event [pid, timestamp]
w.Emit(trace.EvFrequency, 1) // [ticks per second]
w.Emit(trace.EvBatch, 0, 0) // start of per-P batch event [pid, timestamp]
var s stacks
@ -61,8 +61,9 @@ func TestGoroutineCount(t *testing.T) {
w.Emit(trace.EvGoCreate, 1, 40, s.add("pkg.f4"), s.add("main.f4"))
w.Emit(trace.EvGoStartLocal, 1, 40) // [timestamp, goroutine id]
w.Emit(trace.EvGoSched, 1, s.add("main.f4")) // [timestamp, stack]
w.Emit(trace.EvFrequency, 1) // [ticks per second]
res, err := trace.Parse(w, "")
res, err := trace.ParseBuffer(w)
if err != nil {
t.Fatalf("failed to parse test trace: %v", err)
}
@ -74,9 +75,9 @@ func TestGoroutineCount(t *testing.T) {
}
// Use the default viewerDataTraceConsumer but replace
// consumeViewerEvent to intercept the ViewerEvents for testing.
// consumeViewerEvent to intercept the viewerEvents for testing.
c := viewerDataTraceConsumer(ioutil.Discard, 0, 1<<63-1)
c.consumeViewerEvent = func(ev *ViewerEvent, _ bool) {
c.consumeViewerEvent = func(ev *viewerEvent, _ bool) {
if ev.Name == "Goroutines" {
cnt := ev.Arg.(*goroutineCountersArg)
if cnt.Runnable+cnt.Running > 2 {
@ -87,7 +88,7 @@ func TestGoroutineCount(t *testing.T) {
}
// If the counts drop below 0, generateTrace will return an error.
if err := generateTrace(params, c); err != nil {
if err := generateTrace(res, params, c); err != nil {
t.Fatalf("generateTrace failed: %v", err)
}
}
@ -99,8 +100,7 @@ func TestGoroutineFilter(t *testing.T) {
var s stacks
w := trace.NewWriter()
w.Emit(trace.EvBatch, 0, 0) // start of per-P batch event [pid, timestamp]
w.Emit(trace.EvFrequency, 1) // [ticks per second]
w.Emit(trace.EvBatch, 0, 0) // start of per-P batch event [pid, timestamp]
// goroutine 10: blocked
w.Emit(trace.EvGoCreate, 1, 10, s.add("pkg.f1"), s.add("main.f1")) // [timestamp, new goroutine id, new stack id, stack id]
@ -115,8 +115,9 @@ func TestGoroutineFilter(t *testing.T) {
// goroutine 10: runnable->running->block
w.Emit(trace.EvGoStartLocal, 1, 10) // [timestamp, goroutine id]
w.Emit(trace.EvGoBlock, 1, s.add("pkg.f3")) // [timestamp, stack]
w.Emit(trace.EvFrequency, 1) // [ticks per second]
res, err := trace.Parse(w, "")
res, err := trace.ParseBuffer(w)
if err != nil {
t.Fatalf("failed to parse test trace: %v", err)
}
@ -129,15 +130,14 @@ func TestGoroutineFilter(t *testing.T) {
}
c := viewerDataTraceConsumer(ioutil.Discard, 0, 1<<63-1)
if err := generateTrace(params, c); err != nil {
if err := generateTrace(res, params, c); err != nil {
t.Fatalf("generateTrace failed: %v", err)
}
}
func TestPreemptedMarkAssist(t *testing.T) {
w := trace.NewWriter()
w.Emit(trace.EvBatch, 0, 0) // start of per-P batch event [pid, timestamp]
w.Emit(trace.EvFrequency, 1) // [ticks per second]
w.Emit(trace.EvBatch, 0, 0) // start of per-P batch event [pid, timestamp]
var s stacks
// goroutine 9999: running -> mark assisting -> preempted -> assisting -> running -> block
@ -148,11 +148,13 @@ func TestPreemptedMarkAssist(t *testing.T) {
w.Emit(trace.EvGoStartLocal, 1, 9999) // [timestamp, goroutine id]
w.Emit(trace.EvGCMarkAssistDone, 1) // [timestamp]
w.Emit(trace.EvGoBlock, 1, s.add("main.f2")) // [timestamp, stack]
w.Emit(trace.EvFrequency, 1) // [ticks per second]
res, err := trace.Parse(w, "")
res, err := trace.ParseBuffer(w)
if err != nil {
t.Fatalf("failed to parse test trace: %v", err)
}
t.Logf("%+v", *res)
res.Stacks = s // use fake stacks
params := &traceParams{
@ -163,12 +165,12 @@ func TestPreemptedMarkAssist(t *testing.T) {
c := viewerDataTraceConsumer(ioutil.Discard, 0, 1<<63-1)
marks := 0
c.consumeViewerEvent = func(ev *ViewerEvent, _ bool) {
c.consumeViewerEvent = func(ev *viewerEvent, _ bool) {
if strings.Contains(ev.Name, "MARK ASSIST") {
marks++
}
}
if err := generateTrace(params, c); err != nil {
if err := generateTrace(res, params, c); err != nil {
t.Fatalf("generateTrace failed: %v", err)
}
@ -214,7 +216,7 @@ func TestFoo(t *testing.T) {
c := viewerDataTraceConsumer(ioutil.Discard, 0, 1<<63-1)
var logBeforeTaskEnd, logAfterTaskEnd bool
c.consumeViewerEvent = func(ev *ViewerEvent, _ bool) {
c.consumeViewerEvent = func(ev *viewerEvent, _ bool) {
if ev.Name == "log before task ends" {
logBeforeTaskEnd = true
}
@ -222,7 +224,7 @@ func TestFoo(t *testing.T) {
logAfterTaskEnd = true
}
}
if err := generateTrace(params, c); err != nil {
if err := generateTrace(res, params, c); err != nil {
t.Fatalf("generateTrace failed: %v", err)
}
if !logBeforeTaskEnd {

12
src/cmd/trace/trace_unix_test.go
View file

@ -8,7 +8,7 @@ package main
import (
"bytes"
traceparser "internal/trace"
"internal/traceparser"
"io/ioutil"
"runtime"
"runtime/trace"
@ -73,17 +73,15 @@ func TestGoroutineInSyscall(t *testing.T) {
}
trace.Stop()
res, err := traceparser.Parse(buf, "")
if err == traceparser.ErrTimeOrder {
t.Skipf("skipping due to golang.org/issue/16755 (timestamps are unreliable): %v", err)
} else if err != nil {
res, err := traceparser.ParseBuffer(buf)
if err != nil {
t.Fatalf("failed to parse trace: %v", err)
}
// Check only one thread for the pipe read goroutine is
// considered in-syscall.
c := viewerDataTraceConsumer(ioutil.Discard, 0, 1<<63-1)
c.consumeViewerEvent = func(ev *ViewerEvent, _ bool) {
c.consumeViewerEvent = func(ev *viewerEvent, _ bool) {
if ev.Name == "Threads" {
arg := ev.Arg.(*threadCountersArg)
if arg.InSyscall > 1 {
@ -96,7 +94,7 @@ func TestGoroutineInSyscall(t *testing.T) {
parsed: res,
endTime: int64(1<<63 - 1),
}
if err := generateTrace(param, c); err != nil {
if err := generateTrace(res, param, c); err != nil {
t.Fatalf("failed to generate ViewerData: %v", err)
}
}

43
src/crypto/cipher/example_test.go
View file

@ -5,6 +5,7 @@
package cipher_test
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/rand"
@ -298,11 +299,8 @@ func ExampleStreamReader() {
// package like bcrypt or scrypt.
key, _ := hex.DecodeString("6368616e676520746869732070617373")
inFile, err := os.Open("encrypted-file")
if err != nil {
panic(err)
}
defer inFile.Close()
encrypted, _ := hex.DecodeString("cf0495cc6f75dafc23948538e79904a9")
bReader := bytes.NewReader(encrypted)
block, err := aes.NewCipher(key)
if err != nil {
@ -314,15 +312,9 @@ func ExampleStreamReader() {
var iv [aes.BlockSize]byte
stream := cipher.NewOFB(block, iv[:])
outFile, err := os.OpenFile("decrypted-file", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
panic(err)
}
defer outFile.Close()
reader := &cipher.StreamReader{S: stream, R: inFile}
// Copy the input file to the output file, decrypting as we go.
if _, err := io.Copy(outFile, reader); err != nil {
reader := &cipher.StreamReader{S: stream, R: bReader}
// Copy the input to the output stream, decrypting as we go.
if _, err := io.Copy(os.Stdout, reader); err != nil {
panic(err)
}
@ -330,6 +322,8 @@ func ExampleStreamReader() {
// authentication of the encrypted data. If you were actually to use
// StreamReader in this manner, an attacker could flip arbitrary bits in
// the output.
// Output: some secret text
}
func ExampleStreamWriter() {
@ -339,11 +333,7 @@ func ExampleStreamWriter() {
// package like bcrypt or scrypt.
key, _ := hex.DecodeString("6368616e676520746869732070617373")
inFile, err := os.Open("plaintext-file")
if err != nil {
panic(err)
}
defer inFile.Close()
bReader := bytes.NewReader([]byte("some secret text"))
block, err := aes.NewCipher(key)
if err != nil {
@ -355,15 +345,11 @@ func ExampleStreamWriter() {
var iv [aes.BlockSize]byte
stream := cipher.NewOFB(block, iv[:])
outFile, err := os.OpenFile("encrypted-file", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
panic(err)
}
defer outFile.Close()
var out bytes.Buffer
writer := &cipher.StreamWriter{S: stream, W: outFile}
// Copy the input file to the output file, encrypting as we go.
if _, err := io.Copy(writer, inFile); err != nil {
writer := &cipher.StreamWriter{S: stream, W: &out}
// Copy the input to the output buffer, encrypting as we go.
if _, err := io.Copy(writer, bReader); err != nil {
panic(err)
}
@ -371,4 +357,7 @@ func ExampleStreamWriter() {
// authentication of the encrypted data. If you were actually to use
// StreamReader in this manner, an attacker could flip arbitrary bits in
// the decrypted result.
fmt.Printf("%x\n", out.Bytes())
// Output: cf0495cc6f75dafc23948538e79904a9
}

8
src/crypto/cipher/export_test.go Normal file
View file

@ -0,0 +1,8 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package cipher
// Export internal functions for testing.
var XorBytes = xorBytes

27
src/crypto/cipher/xor_amd64.go Normal file
View file

@ -0,0 +1,27 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package cipher
// xorBytes xors the bytes in a and b. The destination should have enough
// space, otherwise xorBytes will panic. Returns the number of bytes xor'd.
func xorBytes(dst, a, b []byte) int {
n := len(a)
if len(b) < n {
n = len(b)
}
if n == 0 {
return 0
}
_ = dst[n-1]
xorBytesSSE2(&dst[0], &a[0], &b[0], n) // amd64 must have SSE2
return n
}
func xorWords(dst, a, b []byte) {
xorBytes(dst, a, b)
}
//go:noescape
func xorBytesSSE2(dst, a, b *byte, n int)

54
src/crypto/cipher/xor_amd64.s Normal file
View file

@ -0,0 +1,54 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
#include "textflag.h"
// func xorBytesSSE2(dst, a, b *byte, n int)
TEXT ·xorBytesSSE2(SB), NOSPLIT, $0
MOVQ dst+0(FP), BX
MOVQ a+8(FP), SI
MOVQ b+16(FP), CX
MOVQ n+24(FP), DX
TESTQ $15, DX // AND 15 & len, if not zero jump to not_aligned.
JNZ not_aligned
aligned:
MOVQ $0, AX // position in slices
loop16b:
MOVOU (SI)(AX*1), X0 // XOR 16byte forwards.
MOVOU (CX)(AX*1), X1
PXOR X1, X0
MOVOU X0, (BX)(AX*1)
ADDQ $16, AX
CMPQ DX, AX
JNE loop16b
RET
loop_1b:
SUBQ $1, DX // XOR 1byte backwards.
MOVB (SI)(DX*1), DI
MOVB (CX)(DX*1), AX
XORB AX, DI
MOVB DI, (BX)(DX*1)
TESTQ $7, DX // AND 7 & len, if not zero jump to loop_1b.
JNZ loop_1b
CMPQ DX, $0 // if len is 0, ret.
JE ret
TESTQ $15, DX // AND 15 & len, if zero jump to aligned.
JZ aligned
not_aligned:
TESTQ $7, DX // AND $7 & len, if not zero jump to loop_1b.
JNE loop_1b
SUBQ $8, DX // XOR 8bytes backwards.
MOVQ (SI)(DX*1), DI
MOVQ (CX)(DX*1), AX
XORQ AX, DI
MOVQ DI, (BX)(DX*1)
CMPQ DX, $16 // if len is greater or equal 16 here, it must be aligned.
JGE aligned
ret:
RET

62
src/crypto/cipher/xor.go → src/crypto/cipher/xor_generic.go
View file

@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !amd64,!ppc64,!ppc64le
package cipher
import (
@ -9,12 +11,9 @@ import (
"unsafe"
)
const wordSize = int(unsafe.Sizeof(uintptr(0)))
const supportsUnaligned = runtime.GOARCH == "386" || runtime.GOARCH == "amd64" || runtime.GOARCH == "ppc64" || runtime.GOARCH == "ppc64le" || runtime.GOARCH == "s390x"
// fastXORBytes xors in bulk. It only works on architectures that
// support unaligned read/writes.
func fastXORBytes(dst, a, b []byte) int {
// xorBytes xors the bytes in a and b. The destination should have enough
// space, otherwise xorBytes will panic. Returns the number of bytes xor'd.
func xorBytes(dst, a, b []byte) int {
n := len(a)
if len(b) < n {
n = len(b)
@ -22,6 +21,28 @@ func fastXORBytes(dst, a, b []byte) int {
if n == 0 {
return 0
}
switch {
case supportsUnaligned:
fastXORBytes(dst, a, b, n)
default:
// TODO(hanwen): if (dst, a, b) have common alignment
// we could still try fastXORBytes. It is not clear
// how often this happens, and it's only worth it if
// the block encryption itself is hardware
// accelerated.
safeXORBytes(dst, a, b, n)
}
return n
}
const wordSize = int(unsafe.Sizeof(uintptr(0)))
const supportsUnaligned = runtime.GOARCH == "386" || runtime.GOARCH == "ppc64" || runtime.GOARCH == "ppc64le" || runtime.GOARCH == "s390x"
// fastXORBytes xors in bulk. It only works on architectures that
// support unaligned read/writes.
// n needs to be smaller or equal than the length of a and b.
func fastXORBytes(dst, a, b []byte, n int) {
// Assert dst has enough space
_ = dst[n-1]
@ -38,34 +59,13 @@ func fastXORBytes(dst, a, b []byte) int {
for i := (n - n%wordSize); i < n; i++ {
dst[i] = a[i] ^ b[i]
}
return n
}
func safeXORBytes(dst, a, b []byte) int {
n := len(a)
if len(b) < n {
n = len(b)
}
// n needs to be smaller or equal than the length of a and b.
func safeXORBytes(dst, a, b []byte, n int) {
for i := 0; i < n; i++ {
dst[i] = a[i] ^ b[i]
}
return n
}
// xorBytes xors the bytes in a and b. The destination should have enough
// space, otherwise xorBytes will panic. Returns the number of bytes xor'd.
func xorBytes(dst, a, b []byte) int {
if supportsUnaligned {
return fastXORBytes(dst, a, b)
} else {
// TODO(hanwen): if (dst, a, b) have common alignment
// we could still try fastXORBytes. It is not clear
// how often this happens, and it's only worth it if
// the block encryption itself is hardware
// accelerated.
return safeXORBytes(dst, a, b)
}
}
// fastXORWords XORs multiples of 4 or 8 bytes (depending on architecture.)
@ -80,10 +80,12 @@ func fastXORWords(dst, a, b []byte) {
}
}
// fastXORWords XORs multiples of 4 or 8 bytes (depending on architecture.)
// The slice arguments a and b are assumed to be of equal length.
func xorWords(dst, a, b []byte) {
if supportsUnaligned {
fastXORWords(dst, a, b)
} else {
safeXORBytes(dst, a, b)
safeXORBytes(dst, a, b, len(b))
}
}

29
src/crypto/cipher/xor_ppc64x.go Normal file
View file

@ -0,0 +1,29 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ppc64 ppc64le
package cipher
// xorBytes xors the bytes in a and b. The destination should have enough
// space, otherwise xorBytes will panic. Returns the number of bytes xor'd.
func xorBytes(dst, a, b []byte) int {
n := len(a)
if len(b) < n {
n = len(b)
}
if n == 0 {
return 0
}
_ = dst[n-1]
xorBytesVSX(&dst[0], &a[0], &b[0], n)
return n
}
func xorWords(dst, a, b []byte) {
xorBytes(dst, a, b)
}
//go:noescape
func xorBytesVSX(dst, a, b *byte, n int)

66
src/crypto/cipher/xor_ppc64x.s Normal file
View file

@ -0,0 +1,66 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ppc64 ppc64le
#include "textflag.h"
// func xorBytesVSX(dst, a, b *byte, n int)
TEXT ·xorBytesVSX(SB), NOSPLIT, $0
MOVD dst+0(FP), R3 // R3 = dst
MOVD a+8(FP), R4 // R4 = a
MOVD b+16(FP), R5 // R5 = b
MOVD n+24(FP), R6 // R6 = n
CMPU R6, $16, CR7 // Check if n ≥ 16 bytes
MOVD R0, R8 // R8 = index
CMPU R6, $8, CR6 // Check if 8 ≤ n < 16 bytes
BGE CR7, preloop16
BLT CR6, small
// Case for 8 ≤ n < 16 bytes
MOVD (R4)(R8), R14 // R14 = a[i,...,i+7]
MOVD (R5)(R8), R15 // R15 = b[i,...,i+7]
XOR R14, R15, R16 // R16 = a[] ^ b[]
SUB $8, R6 // n = n - 8
MOVD R16, (R3)(R8) // Store to dst
ADD $8, R8
// Check if we're finished
CMP R6, R0
BGT small
JMP done
// Case for n ≥ 16 bytes
preloop16:
SRD $4, R6, R7 // Setup loop counter
MOVD R7, CTR
ANDCC $15, R6, R9 // Check for tailing bytes for later
loop16:
LXVD2X (R4)(R8), VS32 // VS32 = a[i,...,i+15]
LXVD2X (R5)(R8), VS33 // VS33 = b[i,...,i+15]
XXLXOR VS32, VS33, VS34 // VS34 = a[] ^ b[]
STXVD2X VS34, (R3)(R8) // Store to dst
ADD $16, R8 // Update index
BC 16, 0, loop16 // bdnz loop16
BEQ CR0, done
SLD $4, R7
SUB R7, R6 // R6 = n - (R7 * 16)
// Case for n < 8 bytes and tailing bytes from the
// previous cases.
small:
MOVD R6, CTR // Setup loop counter
loop:
MOVBZ (R4)(R8), R14 // R14 = a[i]
MOVBZ (R5)(R8), R15 // R15 = b[i]
XOR R14, R15, R16 // R16 = a[i] ^ b[i]
MOVB R16, (R3)(R8) // Store to dst
ADD $1, R8
BC 16, 0, loop // bdnz loop
done:
RET

68
src/crypto/cipher/xor_test.go
View file

@ -2,27 +2,71 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package cipher
package cipher_test
import (
"bytes"
"crypto/cipher"
"crypto/rand"
"fmt"
"io"
"testing"
)
func TestXOR(t *testing.T) {
for alignP := 0; alignP < 2; alignP++ {
for alignQ := 0; alignQ < 2; alignQ++ {
for alignD := 0; alignD < 2; alignD++ {
p := make([]byte, 1024)[alignP:]
q := make([]byte, 1024)[alignQ:]
d1 := make([]byte, 1024+alignD)[alignD:]
d2 := make([]byte, 1024+alignD)[alignD:]
xorBytes(d1, p, q)
safeXORBytes(d2, p, q)
if !bytes.Equal(d1, d2) {
t.Error("not equal")
for j := 1; j <= 1024; j++ {
for alignP := 0; alignP < 2; alignP++ {
for alignQ := 0; alignQ < 2; alignQ++ {
for alignD := 0; alignD < 2; alignD++ {
p := make([]byte, j)[alignP:]
q := make([]byte, j)[alignQ:]
d1 := make([]byte, j+alignD)[alignD:]
d2 := make([]byte, j+alignD)[alignD:]
if _, err := io.ReadFull(rand.Reader, p); err != nil {
t.Fatal(err)
}
if _, err := io.ReadFull(rand.Reader, q); err != nil {
t.Fatal(err)
}
cipher.XorBytes(d1, p, q)
n := min(p, q)
for i := 0; i < n; i++ {
d2[i] = p[i] ^ q[i]
}
if !bytes.Equal(d1, d2) {
t.Logf("p: %#v", p)
t.Logf("q: %#v", q)
t.Logf("expect: %#v", d2)
t.Logf("result: %#v", d1)
t.Fatal("not equal")
}
}
}
}
}
}
func min(a, b []byte) int {
n := len(a)
if len(b) < n {
n = len(b)
}
return n
}
func BenchmarkXORBytes(b *testing.B) {
dst := make([]byte, 1<<15)
data0 := make([]byte, 1<<15)
data1 := make([]byte, 1<<15)
sizes := []int64{1 << 3, 1 << 7, 1 << 11, 1 << 15}
for _, size := range sizes {
b.Run(fmt.Sprintf("%dBytes", size), func(b *testing.B) {
s0 := data0[:size]
s1 := data1[:size]
b.SetBytes(int64(size))
for i := 0; i < b.N; i++ {
cipher.XorBytes(dst, s0, s1)
}
})
}
}

205
src/crypto/md5/gen.go
View file

@ -7,10 +7,7 @@
// This program generates md5block.go
// Invoke as
//
// go run gen.go [-full] -output md5block.go
//
// The -full flag causes the generated code to do a full
// (16x) unrolling instead of a 4x unrolling.
// go run gen.go -output md5block.go
package main
@ -56,13 +53,14 @@ type Data struct {
Table2 []uint32
Table3 []uint32
Table4 []uint32
Full bool
}
var funcs = template.FuncMap{
"dup": dup,
"relabel": relabel,
"rotate": rotate,
"idx": idx,
"seq": seq,
}
func dup(count int, x []int) []int {
@ -74,7 +72,7 @@ func dup(count int, x []int) []int {
}
func relabel(s string) string {
return strings.NewReplacer("a", data.a, "b", data.b, "c", data.c, "d", data.d).Replace(s)
return strings.NewReplacer("arg0", data.a, "arg1", data.b, "arg2", data.c, "arg3", data.d).Replace(s)
}
func rotate() string {
@ -82,8 +80,27 @@ func rotate() string {
return "" // no output
}
func init() {
flag.BoolVar(&data.Full, "full", false, "complete unrolling")
func idx(round, index int) int {
v := 0
switch round {
case 1:
v = index
case 2:
v = (1 + 5*index) & 15
case 3:
v = (5 + 3*index) & 15
case 4:
v = (7 * index) & 15
}
return v
}
func seq(i int) []int {
s := make([]int, i)
for i := range s {
s[i] = i
}
return s
}
var data = Data{
@ -179,152 +196,64 @@ var program = `// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Code generated by go run gen.go{{if .Full}} -full{{end}} -output md5block.go; DO NOT EDIT.
// Code generated by go run gen.go -output md5block.go; DO NOT EDIT.
package md5
import (
"unsafe"
"runtime"
"encoding/binary"
"math/bits"
)
{{if not .Full}}
var t1 = [...]uint32{
{{range .Table1}}{{printf "\t%#x,\n" .}}{{end}}
}
var t2 = [...]uint32{
{{range .Table2}}{{printf "\t%#x,\n" .}}{{end}}
}
var t3 = [...]uint32{
{{range .Table3}}{{printf "\t%#x,\n" .}}{{end}}
}
var t4 = [...]uint32{
{{range .Table4}}{{printf "\t%#x,\n" .}}{{end}}
}
{{end}}
const x86 = runtime.GOARCH == "amd64" || runtime.GOARCH == "386"
var littleEndian bool
func init() {
x := uint32(0x04030201)
y := [4]byte{0x1, 0x2, 0x3, 0x4}
littleEndian = *(*[4]byte)(unsafe.Pointer(&x)) == y
}
func blockGeneric(dig *digest, p []byte) {
a := dig.s[0]
b := dig.s[1]
c := dig.s[2]
d := dig.s[3]
var X *[16]uint32
var xbuf [16]uint32
for len(p) >= chunk {
// load state
a, b, c, d := dig.s[0], dig.s[1], dig.s[2], dig.s[3]
for i := 0; i <= len(p)-BlockSize; i += BlockSize {
// eliminate bounds checks on p
q := p[i:]
q = q[:BlockSize:BlockSize]
// save current state
aa, bb, cc, dd := a, b, c, d
// This is a constant condition - it is not evaluated on each iteration.
if x86 {
// MD5 was designed so that x86 processors can just iterate
// over the block data directly as uint32s, and we generate
// less code and run 1.3x faster if we take advantage of that.
// My apologies.
X = (*[16]uint32)(unsafe.Pointer(&p[0]))
} else if littleEndian && uintptr(unsafe.Pointer(&p[0]))&(unsafe.Alignof(uint32(0))-1) == 0 {
X = (*[16]uint32)(unsafe.Pointer(&p[0]))
} else {
X = &xbuf
j := 0
for i := 0; i < 16; i++ {
X[i&15] = uint32(p[j]) | uint32(p[j+1])<<8 | uint32(p[j+2])<<16 | uint32(p[j+3])<<24
j += 4
}
}
{{if .Full}}
// Round 1.
{{range $i, $s := dup 4 .Shift1}}
{{index $.Table1 $i | printf "a += (((c^d)&b)^d) + X[%d] + %d" $i | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
{{rotate}}
{{end}}
// Round 2.
{{range $i, $s := dup 4 .Shift2}}
{{index $.Table2 $i | printf "a += (((b^c)&d)^c) + X[(1+5*%d)&15] + %d" $i | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
{{rotate}}
{{end}}
// Round 3.
{{range $i, $s := dup 4 .Shift3}}
{{index $.Table3 $i | printf "a += (b^c^d) + X[(5+3*%d)&15] + %d" $i | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
{{rotate}}
{{end}}
// Round 4.
{{range $i, $s := dup 4 .Shift4}}
{{index $.Table4 $i | printf "a += (c^(b|^d)) + X[(7*%d)&15] + %d" $i | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
{{rotate}}
{{end}}
{{else}}
// Round 1.
for i := uint(0); i < 16; {
{{range $s := .Shift1}}
{{printf "a += (((c^d)&b)^d) + X[i&15] + t1[i&15]" | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
i++
{{rotate}}
{{end}}
}
// Round 2.
for i := uint(0); i < 16; {
{{range $s := .Shift2}}
{{printf "a += (((b^c)&d)^c) + X[(1+5*i)&15] + t2[i&15]" | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
i++
{{rotate}}
{{end}}
}
// Round 3.
for i := uint(0); i < 16; {
{{range $s := .Shift3}}
{{printf "a += (b^c^d) + X[(5+3*i)&15] + t3[i&15]" | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
i++
{{rotate}}
{{end}}
}
// Round 4.
for i := uint(0); i < 16; {
{{range $s := .Shift4}}
{{printf "a += (c^(b|^d)) + X[(7*i)&15] + t4[i&15]" | relabel}}
{{printf "a = a<<%d | a>>(32-%d) + b" $s $s | relabel}}
i++
{{rotate}}
{{end}}
}
// load input block
{{range $i := seq 16 -}}
{{printf "x%x := binary.LittleEndian.Uint32(q[4*%#x:])" $i $i}}
{{end}}
// round 1
{{range $i, $s := dup 4 .Shift1 -}}
{{printf "arg0 = arg1 + bits.RotateLeft32((((arg2^arg3)&arg1)^arg3)+arg0+x%x+%#08x, %d)" (idx 1 $i) (index $.Table1 $i) $s | relabel}}
{{rotate -}}
{{end}}
// round 2
{{range $i, $s := dup 4 .Shift2 -}}
{{printf "arg0 = arg1 + bits.RotateLeft32((((arg1^arg2)&arg3)^arg2)+arg0+x%x+%#08x, %d)" (idx 2 $i) (index $.Table2 $i) $s | relabel}}
{{rotate -}}
{{end}}
// round 3
{{range $i, $s := dup 4 .Shift3 -}}
{{printf "arg0 = arg1 + bits.RotateLeft32((arg1^arg2^arg3)+arg0+x%x+%#08x, %d)" (idx 3 $i) (index $.Table3 $i) $s | relabel}}
{{rotate -}}
{{end}}
// round 4
{{range $i, $s := dup 4 .Shift4 -}}
{{printf "arg0 = arg1 + bits.RotateLeft32((arg2^(arg1|^arg3))+arg0+x%x+%#08x, %d)" (idx 4 $i) (index $.Table4 $i) $s | relabel}}
{{rotate -}}
{{end}}
// add saved state
a += aa
b += bb
c += cc
d += dd
p = p[chunk:]
}
dig.s[0] = a
dig.s[1] = b
dig.s[2] = c
dig.s[3] = d
// save state
dig.s[0], dig.s[1], dig.s[2], dig.s[3] = a, b, c, d
}
`

98
src/crypto/md5/md5.go
View file

@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:generate go run gen.go -full -output md5block.go
//go:generate go run gen.go -output md5block.go
// Package md5 implements the MD5 hash algorithm as defined in RFC 1321.
//
@ -12,6 +12,7 @@ package md5
import (
"crypto"
"encoding/binary"
"errors"
"hash"
)
@ -27,7 +28,6 @@ const Size = 16
const BlockSize = 64
const (
chunk = 64
init0 = 0x67452301
init1 = 0xEFCDAB89
init2 = 0x98BADCFE
@ -37,7 +37,7 @@ const (
// digest represents the partial evaluation of a checksum.
type digest struct {
s [4]uint32
x [chunk]byte
x [BlockSize]byte
nx int
len uint64
}
@ -53,7 +53,7 @@ func (d *digest) Reset() {
const (
magic = "md5\x01"
marshaledSize = len(magic) + 4*4 + chunk + 8
marshaledSize = len(magic) + 4*4 + BlockSize + 8
)
func (d *digest) MarshalBinary() ([]byte, error) {
@ -83,45 +83,28 @@ func (d *digest) UnmarshalBinary(b []byte) error {
b, d.s[3] = consumeUint32(b)
b = b[copy(d.x[:], b):]
b, d.len = consumeUint64(b)
d.nx = int(d.len) % chunk
d.nx = int(d.len) % BlockSize
return nil
}
func appendUint64(b []byte, x uint64) []byte {
a := [8]byte{
byte(x >> 56),
byte(x >> 48),
byte(x >> 40),
byte(x >> 32),
byte(x >> 24),
byte(x >> 16),
byte(x >> 8),
byte(x),
}
var a [8]byte
binary.BigEndian.PutUint64(a[:], x)
return append(b, a[:]...)
}
func appendUint32(b []byte, x uint32) []byte {
a := [4]byte{
byte(x >> 24),
byte(x >> 16),
byte(x >> 8),
byte(x),
}
var a [4]byte
binary.BigEndian.PutUint32(a[:], x)
return append(b, a[:]...)
}
func consumeUint64(b []byte) ([]byte, uint64) {
_ = b[7]
x := uint64(b[7]) | uint64(b[6])<<8 | uint64(b[5])<<16 | uint64(b[4])<<24 |
uint64(b[3])<<32 | uint64(b[2])<<40 | uint64(b[1])<<48 | uint64(b[0])<<56
return b[8:], x
return b[8:], binary.BigEndian.Uint64(b[0:8])
}
func consumeUint32(b []byte) ([]byte, uint32) {
_ = b[3]
x := uint32(b[3]) | uint32(b[2])<<8 | uint32(b[1])<<16 | uint32(b[0])<<24
return b[4:], x
return b[4:], binary.BigEndian.Uint32(b[0:4])
}
// New returns a new hash.Hash computing the MD5 checksum. The Hash also
@ -138,20 +121,31 @@ func (d *digest) Size() int { return Size }
func (d *digest) BlockSize() int { return BlockSize }
func (d *digest) Write(p []byte) (nn int, err error) {
// Note that we currently call block or blockGeneric
// directly (guarded using haveAsm) because this allows
// escape analysis to see that p and d don't escape.
nn = len(p)
d.len += uint64(nn)
if d.nx > 0 {
n := copy(d.x[d.nx:], p)
d.nx += n
if d.nx == chunk {
block(d, d.x[:])
if d.nx == BlockSize {
if haveAsm {
block(d, d.x[:])
} else {
blockGeneric(d, d.x[:])
}
d.nx = 0
}
p = p[n:]
}
if len(p) >= chunk {
n := len(p) &^ (chunk - 1)
block(d, p[:n])
if len(p) >= BlockSize {
n := len(p) &^ (BlockSize - 1)
if haveAsm {
block(d, p[:n])
} else {
blockGeneric(d, p[:n])
}
p = p[n:]
}
if len(p) > 0 {
@ -168,35 +162,27 @@ func (d *digest) Sum(in []byte) []byte {
}
func (d *digest) checkSum() [Size]byte {
// Padding. Add a 1 bit and 0 bits until 56 bytes mod 64.
len := d.len
var tmp [64]byte
tmp[0] = 0x80
if len%64 < 56 {
d.Write(tmp[0 : 56-len%64])
} else {
d.Write(tmp[0 : 64+56-len%64])
}
// Length in bits.
len <<= 3
for i := uint(0); i < 8; i++ {
tmp[i] = byte(len >> (8 * i))
}
d.Write(tmp[0:8])
// Append 0x80 to the end of the message and then append zeros
// until the length is a multiple of 56 bytes. Finally append
// 8 bytes representing the message length in bits.
//
// 1 byte end marker :: 0-63 padding bytes :: 8 byte length
tmp := [1 + 63 + 8]byte{0x80}
pad := (55 - d.len) % 64 // calculate number of padding bytes
binary.LittleEndian.PutUint64(tmp[1+pad:], d.len<<3) // append length in bits
d.Write(tmp[:1+pad+8])
// The previous write ensures that a whole number of
// blocks (i.e. a multiple of 64 bytes) have been hashed.
if d.nx != 0 {
panic("d.nx != 0")
}
var digest [Size]byte
for i, s := range d.s {
digest[i*4] = byte(s)
digest[i*4+1] = byte(s >> 8)
digest[i*4+2] = byte(s >> 16)
digest[i*4+3] = byte(s >> 24)
}
binary.LittleEndian.PutUint32(digest[0:], d.s[0])
binary.LittleEndian.PutUint32(digest[4:], d.s[1])
binary.LittleEndian.PutUint32(digest[8:], d.s[2])
binary.LittleEndian.PutUint32(digest[12:], d.s[3])
return digest
}

349
src/crypto/md5/md5block.go
View file

@ -2,263 +2,124 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Code generated by go run gen.go -full -output md5block.go; DO NOT EDIT.
// Code generated by go run gen.go -output md5block.go; DO NOT EDIT.
package md5
import (
"runtime"
"unsafe"
"encoding/binary"
"math/bits"
)
const x86 = runtime.GOARCH == "amd64" || runtime.GOARCH == "386"
var littleEndian bool
func init() {
x := uint32(0x04030201)
y := [4]byte{0x1, 0x2, 0x3, 0x4}
littleEndian = *(*[4]byte)(unsafe.Pointer(&x)) == y
}
func blockGeneric(dig *digest, p []byte) {
a := dig.s[0]
b := dig.s[1]
c := dig.s[2]
d := dig.s[3]
var X *[16]uint32
var xbuf [16]uint32
for len(p) >= chunk {
// load state
a, b, c, d := dig.s[0], dig.s[1], dig.s[2], dig.s[3]
for i := 0; i <= len(p)-BlockSize; i += BlockSize {
// eliminate bounds checks on p
q := p[i:]
q = q[:BlockSize:BlockSize]
// save current state
aa, bb, cc, dd := a, b, c, d
// This is a constant condition - it is not evaluated on each iteration.
if x86 {
// MD5 was designed so that x86 processors can just iterate
// over the block data directly as uint32s, and we generate
// less code and run 1.3x faster if we take advantage of that.
// My apologies.
X = (*[16]uint32)(unsafe.Pointer(&p[0]))
} else if littleEndian && uintptr(unsafe.Pointer(&p[0]))&(unsafe.Alignof(uint32(0))-1) == 0 {
X = (*[16]uint32)(unsafe.Pointer(&p[0]))
} else {
X = &xbuf
j := 0
for i := 0; i < 16; i++ {
X[i&15] = uint32(p[j]) | uint32(p[j+1])<<8 | uint32(p[j+2])<<16 | uint32(p[j+3])<<24
j += 4
}
}
// Round 1.
a += (((c ^ d) & b) ^ d) + X[0] + 3614090360
a = a<<7 | a>>(32-7) + b
d += (((b ^ c) & a) ^ c) + X[1] + 3905402710
d = d<<12 | d>>(32-12) + a
c += (((a ^ b) & d) ^ b) + X[2] + 606105819
c = c<<17 | c>>(32-17) + d
b += (((d ^ a) & c) ^ a) + X[3] + 3250441966
b = b<<22 | b>>(32-22) + c
a += (((c ^ d) & b) ^ d) + X[4] + 4118548399
a = a<<7 | a>>(32-7) + b
d += (((b ^ c) & a) ^ c) + X[5] + 1200080426
d = d<<12 | d>>(32-12) + a
c += (((a ^ b) & d) ^ b) + X[6] + 2821735955
c = c<<17 | c>>(32-17) + d
b += (((d ^ a) & c) ^ a) + X[7] + 4249261313
b = b<<22 | b>>(32-22) + c
a += (((c ^ d) & b) ^ d) + X[8] + 1770035416
a = a<<7 | a>>(32-7) + b
d += (((b ^ c) & a) ^ c) + X[9] + 2336552879
d = d<<12 | d>>(32-12) + a
c += (((a ^ b) & d) ^ b) + X[10] + 4294925233
c = c<<17 | c>>(32-17) + d
b += (((d ^ a) & c) ^ a) + X[11] + 2304563134
b = b<<22 | b>>(32-22) + c
a += (((c ^ d) & b) ^ d) + X[12] + 1804603682
a = a<<7 | a>>(32-7) + b
d += (((b ^ c) & a) ^ c) + X[13] + 4254626195
d = d<<12 | d>>(32-12) + a
c += (((a ^ b) & d) ^ b) + X[14] + 2792965006
c = c<<17 | c>>(32-17) + d
b += (((d ^ a) & c) ^ a) + X[15] + 1236535329
b = b<<22 | b>>(32-22) + c
// Round 2.
a += (((b ^ c) & d) ^ c) + X[(1+5*0)&15] + 4129170786
a = a<<5 | a>>(32-5) + b
d += (((a ^ b) & c) ^ b) + X[(1+5*1)&15] + 3225465664
d = d<<9 | d>>(32-9) + a
c += (((d ^ a) & b) ^ a) + X[(1+5*2)&15] + 643717713
c = c<<14 | c>>(32-14) + d
b += (((c ^ d) & a) ^ d) + X[(1+5*3)&15] + 3921069994
b = b<<20 | b>>(32-20) + c
a += (((b ^ c) & d) ^ c) + X[(1+5*4)&15] + 3593408605
a = a<<5 | a>>(32-5) + b
d += (((a ^ b) & c) ^ b) + X[(1+5*5)&15] + 38016083
d = d<<9 | d>>(32-9) + a
c += (((d ^ a) & b) ^ a) + X[(1+5*6)&15] + 3634488961
c = c<<14 | c>>(32-14) + d
b += (((c ^ d) & a) ^ d) + X[(1+5*7)&15] + 3889429448
b = b<<20 | b>>(32-20) + c
a += (((b ^ c) & d) ^ c) + X[(1+5*8)&15] + 568446438
a = a<<5 | a>>(32-5) + b
d += (((a ^ b) & c) ^ b) + X[(1+5*9)&15] + 3275163606
d = d<<9 | d>>(32-9) + a
c += (((d ^ a) & b) ^ a) + X[(1+5*10)&15] + 4107603335
c = c<<14 | c>>(32-14) + d
b += (((c ^ d) & a) ^ d) + X[(1+5*11)&15] + 1163531501
b = b<<20 | b>>(32-20) + c
a += (((b ^ c) & d) ^ c) + X[(1+5*12)&15] + 2850285829
a = a<<5 | a>>(32-5) + b
d += (((a ^ b) & c) ^ b) + X[(1+5*13)&15] + 4243563512
d = d<<9 | d>>(32-9) + a
c += (((d ^ a) & b) ^ a) + X[(1+5*14)&15] + 1735328473
c = c<<14 | c>>(32-14) + d
b += (((c ^ d) & a) ^ d) + X[(1+5*15)&15] + 2368359562
b = b<<20 | b>>(32-20) + c
// Round 3.
a += (b ^ c ^ d) + X[(5+3*0)&15] + 4294588738
a = a<<4 | a>>(32-4) + b
d += (a ^ b ^ c) + X[(5+3*1)&15] + 2272392833
d = d<<11 | d>>(32-11) + a
c += (d ^ a ^ b) + X[(5+3*2)&15] + 1839030562
c = c<<16 | c>>(32-16) + d
b += (c ^ d ^ a) + X[(5+3*3)&15] + 4259657740
b = b<<23 | b>>(32-23) + c
a += (b ^ c ^ d) + X[(5+3*4)&15] + 2763975236
a = a<<4 | a>>(32-4) + b
d += (a ^ b ^ c) + X[(5+3*5)&15] + 1272893353
d = d<<11 | d>>(32-11) + a
c += (d ^ a ^ b) + X[(5+3*6)&15] + 4139469664
c = c<<16 | c>>(32-16) + d
b += (c ^ d ^ a) + X[(5+3*7)&15] + 3200236656
b = b<<23 | b>>(32-23) + c
a += (b ^ c ^ d) + X[(5+3*8)&15] + 681279174
a = a<<4 | a>>(32-4) + b
d += (a ^ b ^ c) + X[(5+3*9)&15] + 3936430074
d = d<<11 | d>>(32-11) + a
c += (d ^ a ^ b) + X[(5+3*10)&15] + 3572445317
c = c<<16 | c>>(32-16) + d
b += (c ^ d ^ a) + X[(5+3*11)&15] + 76029189
b = b<<23 | b>>(32-23) + c
a += (b ^ c ^ d) + X[(5+3*12)&15] + 3654602809
a = a<<4 | a>>(32-4) + b
d += (a ^ b ^ c) + X[(5+3*13)&15] + 3873151461
d = d<<11 | d>>(32-11) + a
c += (d ^ a ^ b) + X[(5+3*14)&15] + 530742520
c = c<<16 | c>>(32-16) + d
b += (c ^ d ^ a) + X[(5+3*15)&15] + 3299628645
b = b<<23 | b>>(32-23) + c
// Round 4.
a += (c ^ (b | ^d)) + X[(7*0)&15] + 4096336452
a = a<<6 | a>>(32-6) + b
d += (b ^ (a | ^c)) + X[(7*1)&15] + 1126891415
d = d<<10 | d>>(32-10) + a
c += (a ^ (d | ^b)) + X[(7*2)&15] + 2878612391
c = c<<15 | c>>(32-15) + d
b += (d ^ (c | ^a)) + X[(7*3)&15] + 4237533241
b = b<<21 | b>>(32-21) + c
a += (c ^ (b | ^d)) + X[(7*4)&15] + 1700485571
a = a<<6 | a>>(32-6) + b
d += (b ^ (a | ^c)) + X[(7*5)&15] + 2399980690
d = d<<10 | d>>(32-10) + a
c += (a ^ (d | ^b)) + X[(7*6)&15] + 4293915773
c = c<<15 | c>>(32-15) + d
b += (d ^ (c | ^a)) + X[(7*7)&15] + 2240044497
b = b<<21 | b>>(32-21) + c
a += (c ^ (b | ^d)) + X[(7*8)&15] + 1873313359
a = a<<6 | a>>(32-6) + b
d += (b ^ (a | ^c)) + X[(7*9)&15] + 4264355552
d = d<<10 | d>>(32-10) + a
c += (a ^ (d | ^b)) + X[(7*10)&15] + 2734768916
c = c<<15 | c>>(32-15) + d
b += (d ^ (c | ^a)) + X[(7*11)&15] + 1309151649
b = b<<21 | b>>(32-21) + c
a += (c ^ (b | ^d)) + X[(7*12)&15] + 4149444226
a = a<<6 | a>>(32-6) + b
d += (b ^ (a | ^c)) + X[(7*13)&15] + 3174756917
d = d<<10 | d>>(32-10) + a
c += (a ^ (d | ^b)) + X[(7*14)&15] + 718787259
c = c<<15 | c>>(32-15) + d
b += (d ^ (c | ^a)) + X[(7*15)&15] + 3951481745
b = b<<21 | b>>(32-21) + c
// load input block
x0 := binary.LittleEndian.Uint32(q[4*0x0:])
x1 := binary.LittleEndian.Uint32(q[4*0x1:])
x2 := binary.LittleEndian.Uint32(q[4*0x2:])
x3 := binary.LittleEndian.Uint32(q[4*0x3:])
x4 := binary.LittleEndian.Uint32(q[4*0x4:])
x5 := binary.LittleEndian.Uint32(q[4*0x5:])
x6 := binary.LittleEndian.Uint32(q[4*0x6:])
x7 := binary.LittleEndian.Uint32(q[4*0x7:])
x8 := binary.LittleEndian.Uint32(q[4*0x8:])
x9 := binary.LittleEndian.Uint32(q[4*0x9:])
xa := binary.LittleEndian.Uint32(q[4*0xa:])
xb := binary.LittleEndian.Uint32(q[4*0xb:])
xc := binary.LittleEndian.Uint32(q[4*0xc:])
xd := binary.LittleEndian.Uint32(q[4*0xd:])
xe := binary.LittleEndian.Uint32(q[4*0xe:])
xf := binary.LittleEndian.Uint32(q[4*0xf:])
// round 1
a = b + bits.RotateLeft32((((c^d)&b)^d)+a+x0+0xd76aa478, 7)
d = a + bits.RotateLeft32((((b^c)&a)^c)+d+x1+0xe8c7b756, 12)
c = d + bits.RotateLeft32((((a^b)&d)^b)+c+x2+0x242070db, 17)
b = c + bits.RotateLeft32((((d^a)&c)^a)+b+x3+0xc1bdceee, 22)
a = b + bits.RotateLeft32((((c^d)&b)^d)+a+x4+0xf57c0faf, 7)
d = a + bits.RotateLeft32((((b^c)&a)^c)+d+x5+0x4787c62a, 12)
c = d + bits.RotateLeft32((((a^b)&d)^b)+c+x6+0xa8304613, 17)
b = c + bits.RotateLeft32((((d^a)&c)^a)+b+x7+0xfd469501, 22)
a = b + bits.RotateLeft32((((c^d)&b)^d)+a+x8+0x698098d8, 7)
d = a + bits.RotateLeft32((((b^c)&a)^c)+d+x9+0x8b44f7af, 12)
c = d + bits.RotateLeft32((((a^b)&d)^b)+c+xa+0xffff5bb1, 17)
b = c + bits.RotateLeft32((((d^a)&c)^a)+b+xb+0x895cd7be, 22)
a = b + bits.RotateLeft32((((c^d)&b)^d)+a+xc+0x6b901122, 7)
d = a + bits.RotateLeft32((((b^c)&a)^c)+d+xd+0xfd987193, 12)
c = d + bits.RotateLeft32((((a^b)&d)^b)+c+xe+0xa679438e, 17)
b = c + bits.RotateLeft32((((d^a)&c)^a)+b+xf+0x49b40821, 22)
// round 2
a = b + bits.RotateLeft32((((b^c)&d)^c)+a+x1+0xf61e2562, 5)
d = a + bits.RotateLeft32((((a^b)&c)^b)+d+x6+0xc040b340, 9)
c = d + bits.RotateLeft32((((d^a)&b)^a)+c+xb+0x265e5a51, 14)
b = c + bits.RotateLeft32((((c^d)&a)^d)+b+x0+0xe9b6c7aa, 20)
a = b + bits.RotateLeft32((((b^c)&d)^c)+a+x5+0xd62f105d, 5)
d = a + bits.RotateLeft32((((a^b)&c)^b)+d+xa+0x02441453, 9)
c = d + bits.RotateLeft32((((d^a)&b)^a)+c+xf+0xd8a1e681, 14)
b = c + bits.RotateLeft32((((c^d)&a)^d)+b+x4+0xe7d3fbc8, 20)
a = b + bits.RotateLeft32((((b^c)&d)^c)+a+x9+0x21e1cde6, 5)
d = a + bits.RotateLeft32((((a^b)&c)^b)+d+xe+0xc33707d6, 9)
c = d + bits.RotateLeft32((((d^a)&b)^a)+c+x3+0xf4d50d87, 14)
b = c + bits.RotateLeft32((((c^d)&a)^d)+b+x8+0x455a14ed, 20)
a = b + bits.RotateLeft32((((b^c)&d)^c)+a+xd+0xa9e3e905, 5)
d = a + bits.RotateLeft32((((a^b)&c)^b)+d+x2+0xfcefa3f8, 9)
c = d + bits.RotateLeft32((((d^a)&b)^a)+c+x7+0x676f02d9, 14)
b = c + bits.RotateLeft32((((c^d)&a)^d)+b+xc+0x8d2a4c8a, 20)
// round 3
a = b + bits.RotateLeft32((b^c^d)+a+x5+0xfffa3942, 4)
d = a + bits.RotateLeft32((a^b^c)+d+x8+0x8771f681, 11)
c = d + bits.RotateLeft32((d^a^b)+c+xb+0x6d9d6122, 16)
b = c + bits.RotateLeft32((c^d^a)+b+xe+0xfde5380c, 23)
a = b + bits.RotateLeft32((b^c^d)+a+x1+0xa4beea44, 4)
d = a + bits.RotateLeft32((a^b^c)+d+x4+0x4bdecfa9, 11)
c = d + bits.RotateLeft32((d^a^b)+c+x7+0xf6bb4b60, 16)
b = c + bits.RotateLeft32((c^d^a)+b+xa+0xbebfbc70, 23)
a = b + bits.RotateLeft32((b^c^d)+a+xd+0x289b7ec6, 4)
d = a + bits.RotateLeft32((a^b^c)+d+x0+0xeaa127fa, 11)
c = d + bits.RotateLeft32((d^a^b)+c+x3+0xd4ef3085, 16)
b = c + bits.RotateLeft32((c^d^a)+b+x6+0x04881d05, 23)
a = b + bits.RotateLeft32((b^c^d)+a+x9+0xd9d4d039, 4)
d = a + bits.RotateLeft32((a^b^c)+d+xc+0xe6db99e5, 11)
c = d + bits.RotateLeft32((d^a^b)+c+xf+0x1fa27cf8, 16)
b = c + bits.RotateLeft32((c^d^a)+b+x2+0xc4ac5665, 23)
// round 4
a = b + bits.RotateLeft32((c^(b|^d))+a+x0+0xf4292244, 6)
d = a + bits.RotateLeft32((b^(a|^c))+d+x7+0x432aff97, 10)
c = d + bits.RotateLeft32((a^(d|^b))+c+xe+0xab9423a7, 15)
b = c + bits.RotateLeft32((d^(c|^a))+b+x5+0xfc93a039, 21)
a = b + bits.RotateLeft32((c^(b|^d))+a+xc+0x655b59c3, 6)
d = a + bits.RotateLeft32((b^(a|^c))+d+x3+0x8f0ccc92, 10)
c = d + bits.RotateLeft32((a^(d|^b))+c+xa+0xffeff47d, 15)
b = c + bits.RotateLeft32((d^(c|^a))+b+x1+0x85845dd1, 21)
a = b + bits.RotateLeft32((c^(b|^d))+a+x8+0x6fa87e4f, 6)
d = a + bits.RotateLeft32((b^(a|^c))+d+xf+0xfe2ce6e0, 10)
c = d + bits.RotateLeft32((a^(d|^b))+c+x6+0xa3014314, 15)
b = c + bits.RotateLeft32((d^(c|^a))+b+xd+0x4e0811a1, 21)
a = b + bits.RotateLeft32((c^(b|^d))+a+x4+0xf7537e82, 6)
d = a + bits.RotateLeft32((b^(a|^c))+d+xb+0xbd3af235, 10)
c = d + bits.RotateLeft32((a^(d|^b))+c+x2+0x2ad7d2bb, 15)
b = c + bits.RotateLeft32((d^(c|^a))+b+x9+0xeb86d391, 21)
// add saved state
a += aa
b += bb
c += cc
d += dd
p = p[chunk:]
}
dig.s[0] = a
dig.s[1] = b
dig.s[2] = c
dig.s[3] = d
// save state
dig.s[0], dig.s[1], dig.s[2], dig.s[3] = a, b, c, d
}

2
src/crypto/md5/md5block_decl.go
View file

@ -6,6 +6,8 @@
package md5
const haveAsm = true
//go:noescape
func block(dig *digest, p []byte)

2
src/crypto/md5/md5block_generic.go
View file

@ -6,4 +6,6 @@
package md5
const haveAsm = false
var block = blockGeneric

92
src/crypto/tls/cipher_suites.go
View file

@ -5,6 +5,7 @@
package tls
import (
"crypto"
"crypto/aes"
"crypto/cipher"
"crypto/des"
@ -59,8 +60,7 @@ const (
suiteDefaultOff
)
// A cipherSuite is a specific combination of key agreement, cipher and MAC
// function. All cipher suites currently assume RSA key agreement.
// A cipherSuite is a specific combination of key agreement, cipher and MAC function.
type cipherSuite struct {
id uint16
// the lengths, in bytes, of the key material needed for each component.
@ -72,7 +72,7 @@ type cipherSuite struct {
flags int
cipher func(key, iv []byte, isRead bool) interface{}
mac func(version uint16, macKey []byte) macFunction
aead func(key, fixedNonce []byte) cipher.AEAD
aead func(key, fixedNonce []byte) aead
}
var cipherSuites = []*cipherSuite{
@ -104,6 +104,21 @@ var cipherSuites = []*cipherSuite{
{TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, 16, 20, 0, ecdheECDSAKA, suiteECDHE | suiteECDSA | suiteDefaultOff, cipherRC4, macSHA1, nil},
}
// A cipherSuiteTLS13 defines only the pair of the AEAD algorithm and hash
// algorithm to be used with HKDF. See RFC 8446, Appendix B.4.
type cipherSuiteTLS13 struct {
id uint16
keyLen int
aead func(key, fixedNonce []byte) aead
hash crypto.Hash
}
var cipherSuitesTLS13 = []*cipherSuiteTLS13{
{TLS_AES_128_GCM_SHA256, 16, aeadAESGCMTLS13, crypto.SHA256},
{TLS_CHACHA20_POLY1305_SHA256, 32, aeadChaCha20Poly1305, crypto.SHA256},
{TLS_AES_256_GCM_SHA384, 32, aeadAESGCMTLS13, crypto.SHA384},
}
func cipherRC4(key, iv []byte, isRead bool) interface{} {
cipher, _ := rc4.NewCipher(key)
return cipher
@ -166,36 +181,41 @@ type aead interface {
explicitNonceLen() int
}
// fixedNonceAEAD wraps an AEAD and prefixes a fixed portion of the nonce to
const (
aeadNonceLength = 12
noncePrefixLength = 4
)
// prefixNonceAEAD wraps an AEAD and prefixes a fixed portion of the nonce to
// each call.
type fixedNonceAEAD struct {
type prefixNonceAEAD struct {
// nonce contains the fixed part of the nonce in the first four bytes.
nonce [12]byte
nonce [aeadNonceLength]byte
aead cipher.AEAD
}
func (f *fixedNonceAEAD) NonceSize() int { return 8 }
func (f *fixedNonceAEAD) Overhead() int { return f.aead.Overhead() }
func (f *fixedNonceAEAD) explicitNonceLen() int { return 8 }
func (f *prefixNonceAEAD) NonceSize() int { return aeadNonceLength - noncePrefixLength }
func (f *prefixNonceAEAD) Overhead() int { return f.aead.Overhead() }
func (f *prefixNonceAEAD) explicitNonceLen() int { return f.NonceSize() }
func (f *fixedNonceAEAD) Seal(out, nonce, plaintext, additionalData []byte) []byte {
func (f *prefixNonceAEAD) Seal(out, nonce, plaintext, additionalData []byte) []byte {
copy(f.nonce[4:], nonce)
return f.aead.Seal(out, f.nonce[:], plaintext, additionalData)
}
func (f *fixedNonceAEAD) Open(out, nonce, plaintext, additionalData []byte) ([]byte, error) {
func (f *prefixNonceAEAD) Open(out, nonce, ciphertext, additionalData []byte) ([]byte, error) {
copy(f.nonce[4:], nonce)
return f.aead.Open(out, f.nonce[:], plaintext, additionalData)
return f.aead.Open(out, f.nonce[:], ciphertext, additionalData)
}
// xoredNonceAEAD wraps an AEAD by XORing in a fixed pattern to the nonce
// before each call.
type xorNonceAEAD struct {
nonceMask [12]byte
nonceMask [aeadNonceLength]byte
aead cipher.AEAD
}
func (f *xorNonceAEAD) NonceSize() int { return 8 }
func (f *xorNonceAEAD) NonceSize() int { return 8 } // 64-bit sequence number
func (f *xorNonceAEAD) Overhead() int { return f.aead.Overhead() }
func (f *xorNonceAEAD) explicitNonceLen() int { return 0 }
@ -211,11 +231,11 @@ func (f *xorNonceAEAD) Seal(out, nonce, plaintext, additionalData []byte) []byte
return result
}
func (f *xorNonceAEAD) Open(out, nonce, plaintext, additionalData []byte) ([]byte, error) {
func (f *xorNonceAEAD) Open(out, nonce, ciphertext, additionalData []byte) ([]byte, error) {
for i, b := range nonce {
f.nonceMask[4+i] ^= b
}
result, err := f.aead.Open(out, f.nonceMask[:], plaintext, additionalData)
result, err := f.aead.Open(out, f.nonceMask[:], ciphertext, additionalData)
for i, b := range nonce {
f.nonceMask[4+i] ^= b
}
@ -227,7 +247,10 @@ type gcmtls interface {
NewGCMTLS() (cipher.AEAD, error)
}
func aeadAESGCM(key, fixedNonce []byte) cipher.AEAD {
func aeadAESGCM(key, noncePrefix []byte) aead {
if len(noncePrefix) != noncePrefixLength {
panic("tls: internal error: wrong nonce length")
}
aes, err := aes.NewCipher(key)
if err != nil {
panic(err)
@ -243,19 +266,40 @@ func aeadAESGCM(key, fixedNonce []byte) cipher.AEAD {
panic(err)
}
ret := &fixedNonceAEAD{aead: aead}
copy(ret.nonce[:], fixedNonce)
ret := &prefixNonceAEAD{aead: aead}
copy(ret.nonce[:], noncePrefix)
return ret
}
func aeadChaCha20Poly1305(key, fixedNonce []byte) cipher.AEAD {
func aeadAESGCMTLS13(key, nonceMask []byte) aead {
if len(nonceMask) != aeadNonceLength {
panic("tls: internal error: wrong nonce length")
}
aes, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
aead, err := cipher.NewGCM(aes)
if err != nil {
panic(err)
}
ret := &xorNonceAEAD{aead: aead}
copy(ret.nonceMask[:], nonceMask)
return ret
}
func aeadChaCha20Poly1305(key, nonceMask []byte) aead {
if len(nonceMask) != aeadNonceLength {
panic("tls: internal error: wrong nonce length")
}
aead, err := chacha20poly1305.New(key)
if err != nil {
panic(err)
}
ret := &xorNonceAEAD{aead: aead}
copy(ret.nonceMask[:], fixedNonce)
copy(ret.nonceMask[:], nonceMask)
return ret
}
@ -391,6 +435,7 @@ func mutualCipherSuite(have []uint16, want uint16) *cipherSuite {
//
// Taken from https://www.iana.org/assignments/tls-parameters/tls-parameters.xml
const (
// TLS 1.0 - 1.2 cipher suites.
TLS_RSA_WITH_RC4_128_SHA uint16 = 0x0005
TLS_RSA_WITH_3DES_EDE_CBC_SHA uint16 = 0x000a
TLS_RSA_WITH_AES_128_CBC_SHA uint16 = 0x002f
@ -414,6 +459,11 @@ const (
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 uint16 = 0xcca8
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 uint16 = 0xcca9
// TLS 1.3 cipher suites.
TLS_AES_128_GCM_SHA256 uint16 = 0x1301
TLS_AES_256_GCM_SHA384 uint16 = 0x1302
TLS_CHACHA20_POLY1305_SHA256 uint16 = 0x1303
// TLS_FALLBACK_SCSV isn't a standard cipher suite but an indicator
// that the client is doing version fallback. See RFC 7507.
TLS_FALLBACK_SCSV uint16 = 0x5600

114
src/crypto/tls/common.go
View file

@ -27,14 +27,19 @@ const (
VersionTLS10 = 0x0301
VersionTLS11 = 0x0302
VersionTLS12 = 0x0303
// VersionTLS13 is under development in this library and can't be selected
// nor negotiated yet on either side.
VersionTLS13 = 0x0304
)
const (
maxPlaintext = 16384 // maximum plaintext payload length
maxCiphertext = 16384 + 2048 // maximum ciphertext payload length
recordHeaderLen = 5 // record header length
maxHandshake = 65536 // maximum handshake we support (protocol max is 16 MB)
maxWarnAlertCount = 5 // maximum number of consecutive warning alerts
maxPlaintext = 16384 // maximum plaintext payload length
maxCiphertext = 16384 + 2048 // maximum ciphertext payload length
maxCiphertextTLS13 = 16384 + 256 // maximum ciphertext length in TLS 1.3
recordHeaderLen = 5 // record header length
maxHandshake = 65536 // maximum handshake we support (protocol max is 16 MB)
maxUselessRecords = 5 // maximum number of consecutive non-advancing records
minVersion = VersionTLS10
maxVersion = VersionTLS12
@ -74,16 +79,22 @@ const (
// TLS extension numbers
const (
extensionServerName uint16 = 0
extensionStatusRequest uint16 = 5
extensionSupportedCurves uint16 = 10
extensionSupportedPoints uint16 = 11
extensionSignatureAlgorithms uint16 = 13
extensionALPN uint16 = 16
extensionSCT uint16 = 18 // RFC 6962, Section 6
extensionSessionTicket uint16 = 35
extensionNextProtoNeg uint16 = 13172 // not IANA assigned
extensionRenegotiationInfo uint16 = 0xff01
extensionServerName uint16 = 0
extensionStatusRequest uint16 = 5
extensionSupportedCurves uint16 = 10 // supported_groups in TLS 1.3, see RFC 8446, Section 4.2.7
extensionSupportedPoints uint16 = 11
extensionSignatureAlgorithms uint16 = 13
extensionALPN uint16 = 16
extensionSCT uint16 = 18
extensionSessionTicket uint16 = 35
extensionPreSharedKey uint16 = 41
extensionSupportedVersions uint16 = 43
extensionCookie uint16 = 44
extensionPSKModes uint16 = 45
extensionSignatureAlgorithmsCert uint16 = 50
extensionKeyShare uint16 = 51
extensionNextProtoNeg uint16 = 13172 // not IANA assigned
extensionRenegotiationInfo uint16 = 0xff01
)
// TLS signaling cipher suite values
@ -92,7 +103,10 @@ const (
)
// CurveID is the type of a TLS identifier for an elliptic curve. See
// https://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-8
// https://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-8.
//
// In TLS 1.3, this type is called NamedGroup, but at this time this library
// only supports Elliptic Curve based groups. See RFC 8446, Section 4.2.7.
type CurveID uint16
const (
@ -102,6 +116,25 @@ const (
X25519 CurveID = 29
)
// TLS 1.3 Key Share. See RFC 8446, Section 4.2.8.
type keyShare struct {
group CurveID
data []byte
}
// TLS 1.3 PSK Key Exchange Modes. See RFC 8446, Section 4.2.9.
const (
pskModePlain uint8 = 0
pskModeDHE uint8 = 1
)
// TLS 1.3 PSK Identity. Can be a Session Ticket, or a reference to a saved
// session. See RFC 8446, Section 4.2.11.
type pskIdentity struct {
label []byte
obfuscatedTicketAge uint32
}
// TLS Elliptic Curve Point Formats
// https://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-9
const (
@ -425,7 +458,8 @@ type Config struct {
// If RootCAs is nil, TLS uses the host's root CA set.
RootCAs *x509.CertPool
// NextProtos is a list of supported, application level protocols.
// NextProtos is a list of supported application level protocols, in
// order of preference.
NextProtos []string
// ServerName is used to verify the hostname on the returned
@ -777,10 +811,14 @@ func (c *Config) BuildNameToCertificate() {
c.NameToCertificate = make(map[string]*Certificate)
for i := range c.Certificates {
cert := &c.Certificates[i]
x509Cert, err := x509.ParseCertificate(cert.Certificate[0])
if err != nil {
continue
if cert.Leaf == nil {
x509Cert, err := x509.ParseCertificate(cert.Certificate[0])
if err != nil {
continue
}
cert.Leaf = x509Cert
}
x509Cert := cert.Leaf
if len(x509Cert.Subject.CommonName) > 0 {
c.NameToCertificate[x509Cert.Subject.CommonName] = cert
}
@ -922,8 +960,9 @@ func defaultConfig() *Config {
}
var (
once sync.Once
varDefaultCipherSuites []uint16
once sync.Once
varDefaultCipherSuites []uint16
varDefaultCipherSuitesTLS13 []uint16
)
func defaultCipherSuites() []uint16 {
@ -931,19 +970,24 @@ func defaultCipherSuites() []uint16 {
return varDefaultCipherSuites
}
func defaultCipherSuitesTLS13() []uint16 {
once.Do(initDefaultCipherSuites)
return varDefaultCipherSuitesTLS13
}
func initDefaultCipherSuites() {
var topCipherSuites []uint16
// Check the cpu flags for each platform that has optimized GCM implementations.
// Worst case, these variables will just all be false
hasGCMAsmAMD64 := cpu.X86.HasAES && cpu.X86.HasPCLMULQDQ
// Worst case, these variables will just all be false.
var (
hasGCMAsmAMD64 = cpu.X86.HasAES && cpu.X86.HasPCLMULQDQ
hasGCMAsmARM64 = cpu.ARM64.HasAES && cpu.ARM64.HasPMULL
// Keep in sync with crypto/aes/cipher_s390x.go.
hasGCMAsmS390X = cpu.S390X.HasAES && cpu.S390X.HasAESCBC && cpu.S390X.HasAESCTR && (cpu.S390X.HasGHASH || cpu.S390X.HasAESGCM)
hasGCMAsmARM64 := cpu.ARM64.HasAES && cpu.ARM64.HasPMULL
// Keep in sync with crypto/aes/cipher_s390x.go.
hasGCMAsmS390X := cpu.S390X.HasAES && cpu.S390X.HasAESCBC && cpu.S390X.HasAESCTR && (cpu.S390X.HasGHASH || cpu.S390X.HasAESGCM)
hasGCMAsm := hasGCMAsmAMD64 || hasGCMAsmARM64 || hasGCMAsmS390X
hasGCMAsm = hasGCMAsmAMD64 || hasGCMAsmARM64 || hasGCMAsmS390X
)
if hasGCMAsm || boring.Enabled {
// If BoringCrypto is enabled, always prioritize AES-GCM.
@ -957,6 +1001,11 @@ func initDefaultCipherSuites() {
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
}
varDefaultCipherSuitesTLS13 = []uint16{
TLS_AES_128_GCM_SHA256,
TLS_CHACHA20_POLY1305_SHA256,
TLS_AES_256_GCM_SHA384,
}
} else {
// Without AES-GCM hardware, we put the ChaCha20-Poly1305
// cipher suites first.
@ -968,6 +1017,11 @@ func initDefaultCipherSuites() {
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
}
varDefaultCipherSuitesTLS13 = []uint16{
TLS_CHACHA20_POLY1305_SHA256,
TLS_AES_128_GCM_SHA256,
TLS_AES_256_GCM_SHA384,
}
}
varDefaultCipherSuites = make([]uint16, 0, len(cipherSuites))

162
src/crypto/tls/conn.go
View file

@ -94,9 +94,9 @@ type Conn struct {
bytesSent int64
packetsSent int64
// warnCount counts the number of consecutive warning alerts received
// retryCount counts the number of consecutive warning alerts received
// by Conn.readRecord. Protected by in.Mutex.
warnCount int
retryCount int
// activeCall is an atomic int32; the low bit is whether Close has
// been called. the rest of the bits are the number of goroutines
@ -291,9 +291,17 @@ type cbcMode interface {
// decrypt authenticates and decrypts the record if protection is active at
// this stage. The returned plaintext might overlap with the input.
func (hc *halfConn) decrypt(record []byte) (plaintext []byte, err error) {
func (hc *halfConn) decrypt(record []byte) ([]byte, recordType, error) {
var plaintext []byte
typ := recordType(record[0])
payload := record[recordHeaderLen:]
// In TLS 1.3, change_cipher_spec messages are to be ignored without being
// decrypted. See RFC 8446, Appendix D.4.
if hc.version == VersionTLS13 && typ == recordTypeChangeCipherSpec {
return payload, typ, nil
}
paddingGood := byte(255)
paddingLen := 0
@ -305,7 +313,7 @@ func (hc *halfConn) decrypt(record []byte) (plaintext []byte, err error) {
c.XORKeyStream(payload, payload)
case aead:
if len(payload) < explicitNonceLen {
return nil, alertBadRecordMAC
return nil, 0, alertBadRecordMAC
}
nonce := payload[:explicitNonceLen]
if len(nonce) == 0 {
@ -313,22 +321,27 @@ func (hc *halfConn) decrypt(record []byte) (plaintext []byte, err error) {
}
payload = payload[explicitNonceLen:]
copy(hc.additionalData[:], hc.seq[:])
copy(hc.additionalData[8:], record[:3])
n := len(payload) - c.Overhead()
hc.additionalData[11] = byte(n >> 8)
hc.additionalData[12] = byte(n)
additionalData := hc.additionalData[:]
if hc.version == VersionTLS13 {
additionalData = record[:recordHeaderLen]
} else {
copy(additionalData, hc.seq[:])
copy(additionalData[8:], record[:3])
n := len(payload) - c.Overhead()
additionalData[11] = byte(n >> 8)
additionalData[12] = byte(n)
}
var err error
plaintext, err = c.Open(payload[:0], nonce, payload, hc.additionalData[:])
plaintext, err = c.Open(payload[:0], nonce, payload, additionalData)
if err != nil {
return nil, alertBadRecordMAC
return nil, 0, alertBadRecordMAC
}
case cbcMode:
blockSize := c.BlockSize()
minPayload := explicitNonceLen + roundUp(hc.mac.Size()+1, blockSize) // TODO: vuln?
if len(payload)%blockSize != 0 || len(payload) < minPayload {
return nil, alertBadRecordMAC
return nil, 0, alertBadRecordMAC
}
if explicitNonceLen > 0 {
@ -351,6 +364,26 @@ func (hc *halfConn) decrypt(record []byte) (plaintext []byte, err error) {
default:
panic("unknown cipher type")
}
if hc.version == VersionTLS13 {
if typ != recordTypeApplicationData {
return nil, 0, alertUnexpectedMessage
}
if len(plaintext) > maxPlaintext+1 {
return nil, 0, alertRecordOverflow
}
// Remove padding and find the ContentType scanning from the end.
for i := len(plaintext) - 1; i >= 0; i-- {
if plaintext[i] != 0 {
typ = recordType(plaintext[i])
plaintext = plaintext[:i]
break
}
if i == 0 {
return nil, 0, alertUnexpectedMessage
}
}
}
} else {
plaintext = payload
}
@ -358,7 +391,7 @@ func (hc *halfConn) decrypt(record []byte) (plaintext []byte, err error) {
if hc.mac != nil {
macSize := hc.mac.Size()
if len(payload) < macSize {
return nil, alertBadRecordMAC
return nil, 0, alertBadRecordMAC
}
n := len(payload) - macSize - paddingLen
@ -369,14 +402,14 @@ func (hc *halfConn) decrypt(record []byte) (plaintext []byte, err error) {
localMAC := hc.mac.MAC(hc.seq[0:], record[:recordHeaderLen], payload[:n], payload[n+macSize:])
if subtle.ConstantTimeCompare(localMAC, remoteMAC) != 1 || paddingGood != 255 {
return nil, alertBadRecordMAC
return nil, 0, alertBadRecordMAC
}
plaintext = payload[:n]
}
hc.incSeq()
return plaintext, nil
return plaintext, typ, nil
}
// sliceForAppend extends the input slice by n bytes. head is the full extended
@ -438,12 +471,24 @@ func (hc *halfConn) encrypt(record, payload []byte, rand io.Reader) ([]byte, err
nonce = hc.seq[:]
}
copy(hc.additionalData[:], hc.seq[:])
copy(hc.additionalData[8:], record[:3])
hc.additionalData[11] = byte(len(payload) >> 8)
hc.additionalData[12] = byte(len(payload))
if hc.version == VersionTLS13 {
record = append(record, payload...)
record = c.Seal(record, nonce, payload, hc.additionalData[:])
// Encrypt the actual ContentType and replace the plaintext one.
record = append(record, record[0])
record[0] = byte(recordTypeApplicationData)
n := len(payload) + 1 + c.Overhead()
record[3] = byte(n >> 8)
record[4] = byte(n)
record = c.Seal(record[:recordHeaderLen],
nonce, record[recordHeaderLen:], record[:recordHeaderLen])
} else {
copy(hc.additionalData[:], hc.seq[:])
copy(hc.additionalData[8:], record)
record = c.Seal(record, nonce, payload, hc.additionalData[:])
}
case cbcMode:
blockSize := c.BlockSize()
plaintextLen := len(payload) + len(mac)
@ -546,7 +591,7 @@ func (c *Conn) readRecord(want recordType) error {
vers := uint16(hdr[1])<<8 | uint16(hdr[2])
n := int(hdr[3])<<8 | int(hdr[4])
if c.haveVers && vers != c.vers {
if c.haveVers && c.vers != VersionTLS13 && vers != c.vers {
c.sendAlert(alertProtocolVersion)
msg := fmt.Sprintf("received record with version %x when expecting version %x", vers, c.vers)
return c.in.setErrorLocked(c.newRecordHeaderError(nil, msg))
@ -560,7 +605,7 @@ func (c *Conn) readRecord(want recordType) error {
return c.in.setErrorLocked(c.newRecordHeaderError(c.conn, "first record does not look like a TLS handshake"))
}
}
if n > maxCiphertext {
if c.vers == VersionTLS13 && n > maxCiphertextTLS13 || n > maxCiphertext {
c.sendAlert(alertRecordOverflow)
msg := fmt.Sprintf("oversized record received with length %d", n)
return c.in.setErrorLocked(c.newRecordHeaderError(nil, msg))
@ -574,7 +619,7 @@ func (c *Conn) readRecord(want recordType) error {
// Process message.
record := c.rawInput.Next(recordHeaderLen + n)
data, err := c.in.decrypt(record)
data, typ, err := c.in.decrypt(record)
if err != nil {
return c.in.setErrorLocked(c.sendAlert(err.(alert)))
}
@ -582,9 +627,31 @@ func (c *Conn) readRecord(want recordType) error {
return c.in.setErrorLocked(c.sendAlert(alertRecordOverflow))
}
if typ != recordTypeAlert && len(data) > 0 {
// this is a valid non-alert message: reset the count of alerts
c.warnCount = 0
// Application Data messages are always protected.
if c.in.cipher == nil && typ == recordTypeApplicationData {
return c.in.setErrorLocked(c.sendAlert(alertUnexpectedMessage))
}
if typ != recordTypeAlert && typ != recordTypeChangeCipherSpec && len(data) > 0 {
// This is a state-advancing message: reset the retry count.
c.retryCount = 0
}
// Handshake messages MUST NOT be interleaved with other record types in TLS 1.3.
if c.vers == VersionTLS13 && typ != recordTypeHandshake && c.hand.Len() > 0 {
return c.in.setErrorLocked(c.sendAlert(alertUnexpectedMessage))
}
// In TLS 1.3, change_cipher_spec records are ignored until the Finished.
// See RFC 8446, Appendix D.4. Note that according to Section 5, a server
// can send a ChangeCipherSpec before its ServerHello, when c.vers is still
// unset. That's not useful though and suspicious if the server then selects
// a lower protocol version, so don't allow that.
if c.vers == VersionTLS13 && typ == recordTypeChangeCipherSpec {
if len(data) != 1 || data[0] != 1 || c.handshakeComplete() {
return c.in.setErrorLocked(c.sendAlert(alertUnexpectedMessage))
}
return c.retryReadRecord(want)
}
switch typ {
@ -598,14 +665,12 @@ func (c *Conn) readRecord(want recordType) error {
if alert(data[1]) == alertCloseNotify {
return c.in.setErrorLocked(io.EOF)
}
if c.vers == VersionTLS13 {
return c.in.setErrorLocked(&net.OpError{Op: "remote error", Err: alert(data[1])})
}
switch data[0] {
case alertLevelWarning:
c.warnCount++
if c.warnCount > maxWarnAlertCount {
c.sendAlert(alertUnexpectedMessage)
return c.in.setErrorLocked(errors.New("tls: too many warn alerts"))
}
return c.readRecord(want) // Drop the record on the floor and retry.
return c.retryReadRecord(want) // Drop the record on the floor and retry.
case alertLevelError:
return c.in.setErrorLocked(&net.OpError{Op: "remote error", Err: alert(data[1])})
default:
@ -629,6 +694,9 @@ func (c *Conn) readRecord(want recordType) error {
if typ != want {
return c.in.setErrorLocked(c.sendAlert(alertUnexpectedMessage))
}
if len(data) == 0 {
return c.retryReadRecord(want)
}
// Note that data is owned by c.rawInput, following the Next call above,
// to avoid copying the plaintext. This is safe because c.rawInput is
// not read from or written to until c.input is drained.
@ -636,14 +704,35 @@ func (c *Conn) readRecord(want recordType) error {
return nil
case recordTypeHandshake:
if typ != want && !(c.isClient && c.config.Renegotiation != RenegotiateNever) {
if typ != want && !c.isRenegotiationAcceptable() {
return c.in.setErrorLocked(c.sendAlert(alertNoRenegotiation))
}
if len(data) == 0 {
return c.in.setErrorLocked(c.sendAlert(alertUnexpectedMessage))
}
c.hand.Write(data)
return nil
}
}
// retryReadRecord recurses into readRecord to drop a non-advancing record, like
// a warning alert, empty application_data, or a change_cipher_spec in TLS 1.3.
func (c *Conn) retryReadRecord(want recordType) error {
c.retryCount++
if c.retryCount > maxUselessRecords {
c.sendAlert(alertUnexpectedMessage)
return c.in.setErrorLocked(errors.New("tls: too many ignored records"))
}
return c.readRecord(want)
}
func (c *Conn) isRenegotiationAcceptable() bool {
return c.isClient &&
c.vers != VersionTLS13 &&
c.handshakeComplete() &&
c.config.Renegotiation != RenegotiateNever
}
// atLeastReader reads from R, stopping with EOF once at least N bytes have been
// read. It is different from an io.LimitedReader in that it doesn't cut short
// the last Read call, and in that it considers an early EOF an error.
@ -767,6 +856,9 @@ func (c *Conn) maxPayloadSizeForWrite(typ recordType) int {
panic("unknown cipher type")
}
}
if c.vers == VersionTLS13 {
payloadBytes-- // encrypted ContentType
}
// Allow packet growth in arithmetic progression up to max.
pkt := c.packetsSent
@ -822,6 +914,10 @@ func (c *Conn) writeRecordLocked(typ recordType, data []byte) (int, error) {
// Some TLS servers fail if the record version is
// greater than TLS 1.0 for the initial ClientHello.
vers = VersionTLS10
} else if vers == VersionTLS13 {
// TLS 1.3 froze the record layer version to 1.2.
// See RFC 8446, Section 5.1.
vers = VersionTLS12
}
c.outBuf[1] = byte(vers >> 8)
c.outBuf[2] = byte(vers)

278
src/crypto/tls/handshake_messages.go
View file

@ -49,24 +49,31 @@ func readUint24LengthPrefixed(s *cryptobyte.String, out *[]byte) bool {
}
type clientHelloMsg struct {
raw []byte
vers uint16
random []byte
sessionId []byte
cipherSuites []uint16
compressionMethods []uint8
nextProtoNeg bool
serverName string
ocspStapling bool
scts bool
supportedCurves []CurveID
supportedPoints []uint8
ticketSupported bool
sessionTicket []uint8
supportedSignatureAlgorithms []SignatureScheme
secureRenegotiation []byte
secureRenegotiationSupported bool
alpnProtocols []string
raw []byte
vers uint16
random []byte
sessionId []byte
cipherSuites []uint16
compressionMethods []uint8
nextProtoNeg bool
serverName string
ocspStapling bool
supportedCurves []CurveID
supportedPoints []uint8
ticketSupported bool
sessionTicket []uint8
supportedSignatureAlgorithms []SignatureScheme
supportedSignatureAlgorithmsCert []SignatureScheme
secureRenegotiationSupported bool
secureRenegotiation []byte
alpnProtocols []string
scts bool
supportedVersions []uint16
cookie []byte
keyShares []keyShare
pskModes []uint8
pskIdentities []pskIdentity
pskBinders [][]byte
}
func (m *clientHelloMsg) marshal() []byte {
@ -123,7 +130,7 @@ func (m *clientHelloMsg) marshal() []byte {
})
}
if len(m.supportedCurves) > 0 {
// RFC 4492, Section 5.1.1
// RFC 4492, Section 5.1.1 and RFC 8446, Section 4.2.7
b.AddUint16(extensionSupportedCurves)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
@ -160,6 +167,17 @@ func (m *clientHelloMsg) marshal() []byte {
})
})
}
if len(m.supportedSignatureAlgorithmsCert) > 0 {
// RFC 8446, Section 4.2.3
b.AddUint16(extensionSignatureAlgorithmsCert)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
for _, sigAlgo := range m.supportedSignatureAlgorithmsCert {
b.AddUint16(uint16(sigAlgo))
}
})
})
}
if m.secureRenegotiationSupported {
// RFC 5746, Section 3.2
b.AddUint16(extensionRenegotiationInfo)
@ -187,6 +205,70 @@ func (m *clientHelloMsg) marshal() []byte {
b.AddUint16(extensionSCT)
b.AddUint16(0) // empty extension_data
}
if len(m.supportedVersions) > 0 {
// RFC 8446, Section 4.2.1
b.AddUint16(extensionSupportedVersions)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint8LengthPrefixed(func(b *cryptobyte.Builder) {
for _, vers := range m.supportedVersions {
b.AddUint16(vers)
}
})
})
}
if len(m.cookie) > 0 {
// RFC 8446, Section 4.2.2
b.AddUint16(extensionCookie)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(m.cookie)
})
})
}
if len(m.keyShares) > 0 {
// RFC 8446, Section 4.2.8
b.AddUint16(extensionKeyShare)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
for _, ks := range m.keyShares {
b.AddUint16(uint16(ks.group))
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(ks.data)
})
}
})
})
}
if len(m.pskModes) > 0 {
// RFC 8446, Section 4.2.9
b.AddUint16(extensionPSKModes)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint8LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(m.pskModes)
})
})
}
if len(m.pskIdentities) > 0 { // pre_shared_key must be the last extension
// RFC 8446, Section 4.2.11
b.AddUint16(extensionPreSharedKey)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
for _, psk := range m.pskIdentities {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(psk.label)
})
b.AddUint32(psk.obfuscatedTicketAge)
}
})
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
for _, binder := range m.pskBinders {
b.AddUint8LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(binder)
})
}
})
})
}
extensionsPresent = len(b.BytesOrPanic()) > 2
})
@ -291,7 +373,7 @@ func (m *clientHelloMsg) unmarshal(data []byte) bool {
}
m.ocspStapling = statusType == statusTypeOCSP
case extensionSupportedCurves:
// RFC 4492, Section 5.1.1
// RFC 4492, Section 5.1.1 and RFC 8446, Section 4.2.7
var curves cryptobyte.String
if !extData.ReadUint16LengthPrefixed(&curves) || curves.Empty() {
return false
@ -327,6 +409,20 @@ func (m *clientHelloMsg) unmarshal(data []byte) bool {
m.supportedSignatureAlgorithms = append(
m.supportedSignatureAlgorithms, SignatureScheme(sigAndAlg))
}
case extensionSignatureAlgorithmsCert:
// RFC 8446, Section 4.2.3
var sigAndAlgs cryptobyte.String
if !extData.ReadUint16LengthPrefixed(&sigAndAlgs) || sigAndAlgs.Empty() {
return false
}
for !sigAndAlgs.Empty() {
var sigAndAlg uint16
if !sigAndAlgs.ReadUint16(&sigAndAlg) {
return false
}
m.supportedSignatureAlgorithmsCert = append(
m.supportedSignatureAlgorithmsCert, SignatureScheme(sigAndAlg))
}
case extensionRenegotiationInfo:
// RFC 5746, Section 3.2
if !readUint8LengthPrefixed(&extData, &m.secureRenegotiation) {
@ -349,6 +445,74 @@ func (m *clientHelloMsg) unmarshal(data []byte) bool {
case extensionSCT:
// RFC 6962, Section 3.3.1
m.scts = true
case extensionSupportedVersions:
// RFC 8446, Section 4.2.1
var versList cryptobyte.String
if !extData.ReadUint8LengthPrefixed(&versList) || versList.Empty() {
return false
}
for !versList.Empty() {
var vers uint16
if !versList.ReadUint16(&vers) {
return false
}
m.supportedVersions = append(m.supportedVersions, vers)
}
case extensionCookie:
// RFC 8446, Section 4.2.2
if !readUint16LengthPrefixed(&extData, &m.cookie) {
return false
}
case extensionKeyShare:
// RFC 8446, Section 4.2.8
var clientShares cryptobyte.String
if !extData.ReadUint16LengthPrefixed(&clientShares) {
return false
}
for !clientShares.Empty() {
var ks keyShare
if !clientShares.ReadUint16((*uint16)(&ks.group)) ||
!readUint16LengthPrefixed(&clientShares, &ks.data) ||
len(ks.data) == 0 {
return false
}
m.keyShares = append(m.keyShares, ks)
}
case extensionPSKModes:
// RFC 8446, Section 4.2.9
if !readUint8LengthPrefixed(&extData, &m.pskModes) {
return false
}
case extensionPreSharedKey:
// RFC 8446, Section 4.2.11
if !extensions.Empty() {
return false // pre_shared_key must be the last extension
}
var identities cryptobyte.String
if !extData.ReadUint16LengthPrefixed(&identities) || identities.Empty() {
return false
}
for !identities.Empty() {
var psk pskIdentity
if !readUint16LengthPrefixed(&identities, &psk.label) ||
!identities.ReadUint32(&psk.obfuscatedTicketAge) ||
len(psk.label) == 0 {
return false
}
m.pskIdentities = append(m.pskIdentities, psk)
}
var binders cryptobyte.String
if !extData.ReadUint16LengthPrefixed(&binders) || binders.Empty() {
return false
}
for !binders.Empty() {
var binder []byte
if !readUint8LengthPrefixed(&binders, &binder) ||
len(binder) == 0 {
return false
}
m.pskBinders = append(m.pskBinders, binder)
}
default:
// Ignore unknown extensions.
continue
@ -372,11 +536,19 @@ type serverHelloMsg struct {
nextProtoNeg bool
nextProtos []string
ocspStapling bool
scts [][]byte
ticketSupported bool
secureRenegotiation []byte
secureRenegotiationSupported bool
secureRenegotiation []byte
alpnProtocol string
scts [][]byte
supportedVersion uint16
serverShare keyShare
selectedIdentityPresent bool
selectedIdentity uint16
// HelloRetryRequest extensions
cookie []byte
selectedGroup CurveID
}
func (m *serverHelloMsg) marshal() []byte {
@ -448,6 +620,42 @@ func (m *serverHelloMsg) marshal() []byte {
})
})
}
if m.supportedVersion != 0 {
b.AddUint16(extensionSupportedVersions)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16(m.supportedVersion)
})
}
if m.serverShare.group != 0 {
b.AddUint16(extensionKeyShare)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16(uint16(m.serverShare.group))
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(m.serverShare.data)
})
})
}
if m.selectedIdentityPresent {
b.AddUint16(extensionPreSharedKey)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16(m.selectedIdentity)
})
}
if len(m.cookie) > 0 {
b.AddUint16(extensionCookie)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddBytes(m.cookie)
})
})
}
if m.selectedGroup != 0 {
b.AddUint16(extensionKeyShare)
b.AddUint16LengthPrefixed(func(b *cryptobyte.Builder) {
b.AddUint16(uint16(m.selectedGroup))
})
}
extensionsPresent = len(b.BytesOrPanic()) > 2
})
@ -535,6 +743,32 @@ func (m *serverHelloMsg) unmarshal(data []byte) bool {
}
m.scts = append(m.scts, sct)
}
case extensionSupportedVersions:
if !extData.ReadUint16(&m.supportedVersion) {
return false
}
case extensionCookie:
if !readUint16LengthPrefixed(&extData, &m.cookie) {
return false
}
case extensionKeyShare:
// This extension has different formats in SH and HRR, accept either
// and let the handshake logic decide. See RFC 8446, Section 4.2.8.
if len(extData) == 2 {
if !extData.ReadUint16((*uint16)(&m.selectedGroup)) {
return false
}
} else {
if !extData.ReadUint16((*uint16)(&m.serverShare.group)) ||
!readUint16LengthPrefixed(&extData, &m.serverShare.data) {
return false
}
}
case extensionPreSharedKey:
m.selectedIdentityPresent = true
if !extData.ReadUint16(&m.selectedIdentity) {
return false
}
default:
// Ignore unknown extensions.
continue

51
src/crypto/tls/handshake_messages_test.go
View file

@ -11,6 +11,7 @@ import (
"strings"
"testing"
"testing/quick"
"time"
)
var tests = []interface{}{
@ -31,7 +32,7 @@ var tests = []interface{}{
}
func TestMarshalUnmarshal(t *testing.T) {
rand := rand.New(rand.NewSource(0))
rand := rand.New(rand.NewSource(time.Now().UnixNano()))
for i, iface := range tests {
ty := reflect.ValueOf(iface).Type()
@ -132,7 +133,7 @@ func (*clientHelloMsg) Generate(rand *rand.Rand, size int) reflect.Value {
m.supportedPoints = randomBytes(rand.Intn(5)+1, rand)
m.supportedCurves = make([]CurveID, rand.Intn(5)+1)
for i := range m.supportedCurves {
m.supportedCurves[i] = CurveID(rand.Intn(30000))
m.supportedCurves[i] = CurveID(rand.Intn(30000) + 1)
}
if rand.Intn(10) > 5 {
m.ticketSupported = true
@ -145,6 +146,9 @@ func (*clientHelloMsg) Generate(rand *rand.Rand, size int) reflect.Value {
if rand.Intn(10) > 5 {
m.supportedSignatureAlgorithms = supportedSignatureAlgorithms()
}
if rand.Intn(10) > 5 {
m.supportedSignatureAlgorithmsCert = supportedSignatureAlgorithms()
}
for i := 0; i < rand.Intn(5); i++ {
m.alpnProtocols = append(m.alpnProtocols, randomString(rand.Intn(20)+1, rand))
}
@ -155,6 +159,31 @@ func (*clientHelloMsg) Generate(rand *rand.Rand, size int) reflect.Value {
m.secureRenegotiationSupported = true
m.secureRenegotiation = randomBytes(rand.Intn(50)+1, rand)
}
for i := 0; i < rand.Intn(5); i++ {
m.supportedVersions = append(m.supportedVersions, uint16(rand.Intn(0xffff)+1))
}
if rand.Intn(10) > 5 {
m.cookie = randomBytes(rand.Intn(500)+1, rand)
}
for i := 0; i < rand.Intn(5); i++ {
var ks keyShare
ks.group = CurveID(rand.Intn(30000) + 1)
ks.data = randomBytes(rand.Intn(200)+1, rand)
m.keyShares = append(m.keyShares, ks)
}
switch rand.Intn(3) {
case 1:
m.pskModes = []uint8{pskModeDHE}
case 2:
m.pskModes = []uint8{pskModeDHE, pskModePlain}
}
for i := 0; i < rand.Intn(5); i++ {
var psk pskIdentity
psk.obfuscatedTicketAge = uint32(rand.Intn(500000))
psk.label = randomBytes(rand.Intn(500)+1, rand)
m.pskIdentities = append(m.pskIdentities, psk)
m.pskBinders = append(m.pskBinders, randomBytes(rand.Intn(50)+32, rand))
}
return reflect.ValueOf(m)
}
@ -190,6 +219,24 @@ func (*serverHelloMsg) Generate(rand *rand.Rand, size int) reflect.Value {
m.secureRenegotiationSupported = true
m.secureRenegotiation = randomBytes(rand.Intn(50)+1, rand)
}
if rand.Intn(10) > 5 {
m.supportedVersion = uint16(rand.Intn(0xffff) + 1)
}
if rand.Intn(10) > 5 {
m.cookie = randomBytes(rand.Intn(500)+1, rand)
}
if rand.Intn(10) > 5 {
for i := 0; i < rand.Intn(5); i++ {
m.serverShare.group = CurveID(rand.Intn(30000) + 1)
m.serverShare.data = randomBytes(rand.Intn(200)+1, rand)
}
} else if rand.Intn(10) > 5 {
m.selectedGroup = CurveID(rand.Intn(30000) + 1)
}
if rand.Intn(10) > 5 {
m.selectedIdentityPresent = true
m.selectedIdentity = uint16(rand.Intn(0xffff))
}
return reflect.ValueOf(m)
}

4
src/crypto/tls/tls_test.go
View file

@ -589,7 +589,7 @@ func TestWarningAlertFlood(t *testing.T) {
if err == nil {
return errors.New("unexpected lack of error from server")
}
const expected = "too many warn"
const expected = "too many ignored"
if str := err.Error(); !strings.Contains(str, expected) {
return fmt.Errorf("expected error containing %q, but saw: %s", expected, str)
}
@ -610,7 +610,7 @@ func TestWarningAlertFlood(t *testing.T) {
t.Fatal(err)
}
for i := 0; i < maxWarnAlertCount+1; i++ {
for i := 0; i < maxUselessRecords+1; i++ {
conn.sendAlert(alertNoRenegotiation)
}

1
src/database/sql/sql.go
View file

@ -133,6 +133,7 @@ const (
LevelLinearizable
)
// String returns the name of the transaction isolation level.
func (i IsolationLevel) String() string {
switch i {
case LevelDefault:

2
src/debug/elf/file_test.go
View file

@ -784,7 +784,7 @@ func TestCompressedSection(t *testing.T) {
func TestNoSectionOverlaps(t *testing.T) {
// Ensure cmd/link outputs sections without overlaps.
switch runtime.GOOS {
case "android", "darwin", "js", "nacl", "plan9", "windows":
case "aix", "android", "darwin", "js", "nacl", "plan9", "windows":
t.Skipf("cmd/link doesn't produce ELF binaries on %s", runtime.GOOS)
}
_ = net.ResolveIPAddr // force dynamic linkage

2
src/go/build/build.go
View file

@ -298,7 +298,7 @@ func defaultContext() Context {
// (perhaps it is the stub to use in that case) should say "+build !go1.x".
// NOTE: If you add to this list, also update the doc comment in doc.go.
// NOTE: The last element in ReleaseTags should be the current release.
const version = 11 // go1.11
const version = 12 // go1.12
for i := 1; i <= version; i++ {
c.ReleaseTags = append(c.ReleaseTags, "go1."+strconv.Itoa(i))
}

1
src/go/build/doc.go
View file

@ -108,6 +108,7 @@
// - "go1.9", from Go version 1.9 onward
// - "go1.10", from Go version 1.10 onward
// - "go1.11", from Go version 1.11 onward
// - "go1.12", from Go version 1.12 onward
// - any additional words listed in ctxt.BuildTags
//
// There are no build tags for beta or minor releases. Programs that need the

2
src/go/build/syslist.go
View file

@ -4,5 +4,5 @@
package build
const goosList = "aix android darwin dragonfly freebsd js linux nacl netbsd openbsd plan9 solaris windows zos "
const goosList = "aix android darwin dragonfly freebsd hurd js linux nacl netbsd openbsd plan9 solaris windows zos "
const goarchList = "386 amd64 amd64p32 arm armbe arm64 arm64be ppc64 ppc64le mips mipsle mips64 mips64le mips64p32 mips64p32le ppc riscv riscv64 s390 s390x sparc sparc64 wasm "

17
src/internal/cpu/cpu_ppc64x.go
View file

@ -11,18 +11,19 @@ const CacheLinePadSize = 128
// ppc64x doesn't have a 'cpuid' equivalent, so we rely on HWCAP/HWCAP2.
// These are initialized by archauxv in runtime/os_linux_ppc64x.go.
// These should not be changed after they are initialized.
// On aix/ppc64, these values are initialized early in the runtime in runtime/os_aix.go.
var HWCap uint
var HWCap2 uint
// HWCAP/HWCAP2 bits. These are exposed by the kernel.
const (
// ISA Level
_PPC_FEATURE2_ARCH_2_07 = 0x80000000
_PPC_FEATURE2_ARCH_3_00 = 0x00800000
PPC_FEATURE2_ARCH_2_07 = 0x80000000
PPC_FEATURE2_ARCH_3_00 = 0x00800000
// CPU features
_PPC_FEATURE2_DARN = 0x00200000
_PPC_FEATURE2_SCV = 0x00100000
PPC_FEATURE2_DARN = 0x00200000
PPC_FEATURE2_SCV = 0x00100000
)
func doinit() {
@ -36,10 +37,10 @@ func doinit() {
}
// HWCAP2 feature bits
PPC64.IsPOWER8 = isSet(HWCap2, _PPC_FEATURE2_ARCH_2_07)
PPC64.IsPOWER9 = isSet(HWCap2, _PPC_FEATURE2_ARCH_3_00)
PPC64.HasDARN = isSet(HWCap2, _PPC_FEATURE2_DARN)
PPC64.HasSCV = isSet(HWCap2, _PPC_FEATURE2_SCV)
PPC64.IsPOWER8 = isSet(HWCap2, PPC_FEATURE2_ARCH_2_07)
PPC64.IsPOWER9 = isSet(HWCap2, PPC_FEATURE2_ARCH_3_00)
PPC64.HasDARN = isSet(HWCap2, PPC_FEATURE2_DARN)
PPC64.HasSCV = isSet(HWCap2, PPC_FEATURE2_SCV)
}
func isSet(hwc uint, value uint) bool {

8
src/internal/poll/fd_plan9.go
View file

@ -193,10 +193,10 @@ func isInterrupted(err error) bool {
return err != nil && stringsHasSuffix(err.Error(), "interrupted")
}
// PollDescriptor returns the descriptor being used by the poller,
// or ^uintptr(0) if there isn't one. This is only used for testing.
func PollDescriptor() uintptr {
return ^uintptr(0)
// IsPollDescriptor returns true if fd is the descriptor being used by the poller.
// This is only used for testing.
func IsPollDescriptor(fd uintptr) bool {
return false
}
// RawControl invokes the user-defined function f for a non-IO

8
src/internal/poll/fd_poll_nacljs.go
View file

@ -92,8 +92,8 @@ func setDeadlineImpl(fd *FD, t time.Time, mode int) error {
return nil
}
// PollDescriptor returns the descriptor being used by the poller,
// or ^uintptr(0) if there isn't one. This is only used for testing.
func PollDescriptor() uintptr {
return ^uintptr(0)
// IsPollDescriptor returns true if fd is the descriptor being used by the poller.
// This is only used for testing.
func IsPollDescriptor(fd uintptr) bool {
return false
}

27
src/internal/poll/fd_poll_runtime.go
View file

@ -11,13 +11,14 @@ import (
"sync"
"syscall"
"time"
_ "unsafe" // for go:linkname
)
// runtimeNano returns the current value of the runtime clock in nanoseconds.
//go:linkname runtimeNano runtime.nanotime
func runtimeNano() int64
func runtime_pollServerInit()
func runtime_pollServerDescriptor() uintptr
func runtime_pollOpen(fd uintptr) (uintptr, int)
func runtime_pollClose(ctx uintptr)
func runtime_pollWait(ctx uintptr, mode int) int
@ -25,6 +26,7 @@ func runtime_pollWaitCanceled(ctx uintptr, mode int) int
func runtime_pollReset(ctx uintptr, mode int) int
func runtime_pollSetDeadline(ctx uintptr, d int64, mode int)
func runtime_pollUnblock(ctx uintptr)
func runtime_isPollServerDescriptor(fd uintptr) bool
type pollDesc struct {
runtimeCtx uintptr
@ -134,15 +136,12 @@ func (fd *FD) SetWriteDeadline(t time.Time) error {
}
func setDeadlineImpl(fd *FD, t time.Time, mode int) error {
diff := int64(time.Until(t))
d := runtimeNano() + diff
if d <= 0 && diff > 0 {
// If the user has a deadline in the future, but the delay calculation
// overflows, then set the deadline to the maximum possible value.
d = 1<<63 - 1
}
if t.IsZero() {
d = 0
var d int64
if !t.IsZero() {
d = int64(time.Until(t))
if d == 0 {
d = -1 // don't confuse deadline right now with no deadline
}
}
if err := fd.incref(); err != nil {
return err
@ -155,8 +154,8 @@ func setDeadlineImpl(fd *FD, t time.Time, mode int) error {
return nil
}
// PollDescriptor returns the descriptor being used by the poller,
// or ^uintptr(0) if there isn't one. This is only used for testing.
func PollDescriptor() uintptr {
return runtime_pollServerDescriptor()
// IsPollDescriptor returns true if fd is the descriptor being used by the poller.
// This is only used for testing.
func IsPollDescriptor(fd uintptr) bool {
return runtime_isPollServerDescriptor(fd)
}

10
src/internal/syscall/unix/asm_solaris.s Normal file
View file

@ -0,0 +1,10 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
#include "textflag.h"
// System calls for Solaris are implemented in runtime/syscall_solaris.go
TEXT ·syscall6(SB),NOSPLIT,$0-88
JMP syscall·sysvicall6(SB)

58
src/internal/syscall/unix/at.go Normal file
View file

@ -0,0 +1,58 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build linux darwin openbsd netbsd dragonfly
package unix
import (
"syscall"
"unsafe"
)
func Unlinkat(dirfd int, path string, flags int) error {
var p *byte
p, err := syscall.BytePtrFromString(path)
if err != nil {
return err
}
_, _, errno := syscall.Syscall(unlinkatTrap, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(flags))
if errno != 0 {
return errno
}
return nil
}
func Openat(dirfd int, path string, flags int, perm uint32) (int, error) {
var p *byte
p, err := syscall.BytePtrFromString(path)
if err != nil {
return 0, err
}
fd, _, errno := syscall.Syscall6(openatTrap, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(flags), uintptr(perm), 0, 0)
if errno != 0 {
return 0, errno
}
return int(fd), nil
}
func Fstatat(dirfd int, path string, stat *syscall.Stat_t, flags int) error {
var p *byte
p, err := syscall.BytePtrFromString(path)
if err != nil {
return err
}
_, _, errno := syscall.Syscall6(fstatatTrap, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(unsafe.Pointer(stat)), uintptr(flags), 0, 0)
if errno != 0 {
return errno
}
return nil
}

14
src/internal/syscall/unix/at_aix.go Normal file
View file

@ -0,0 +1,14 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
//go:cgo_import_dynamic libc_fstatat fstatat "libc.a/shr_64.o"
//go:cgo_import_dynamic libc_openat openat "libc.a/shr_64.o"
//go:cgo_import_dynamic libc_unlinkat unlinkat "libc.a/shr_64.o"
const (
AT_REMOVEDIR = 0x1
AT_SYMLINK_NOFOLLOW = 0x1
)

47
src/internal/syscall/unix/at_freebsd.go Normal file
View file

@ -0,0 +1,47 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
import (
"syscall"
"unsafe"
)
const (
AT_REMOVEDIR = 0x800
AT_SYMLINK_NOFOLLOW = 0x200
)
func Unlinkat(dirfd int, path string, flags int) error {
p, err := syscall.BytePtrFromString(path)
if err != nil {
return err
}
_, _, errno := syscall.Syscall(syscall.SYS_UNLINKAT, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(flags))
if errno != 0 {
return errno
}
return nil
}
func Openat(dirfd int, path string, flags int, perm uint32) (int, error) {
p, err := syscall.BytePtrFromString(path)
if err != nil {
return 0, err
}
fd, _, errno := syscall.Syscall6(syscall.SYS_OPENAT, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(flags), uintptr(perm), 0, 0)
if errno != 0 {
return 0, errno
}
return int(fd), nil
}
func Fstatat(dirfd int, path string, stat *syscall.Stat_t, flags int) error {
return syscall.Fstatat(dirfd, path, stat, flags)
}

64
src/internal/syscall/unix/at_libc.go Normal file
View file

@ -0,0 +1,64 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build aix solaris
package unix
import (
"syscall"
"unsafe"
)
//go:linkname procFstatat libc_fstatat
//go:linkname procOpenat libc_openat
//go:linkname procUnlinkat libc_unlinkat
var (
procFstatat,
procOpenat,
procUnlinkat uintptr
)
func Unlinkat(dirfd int, path string, flags int) error {
p, err := syscall.BytePtrFromString(path)
if err != nil {
return err
}
_, _, errno := syscall6(uintptr(unsafe.Pointer(&procUnlinkat)), 3, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(flags), 0, 0, 0)
if errno != 0 {
return errno
}
return nil
}
func Openat(dirfd int, path string, flags int, perm uint32) (int, error) {
p, err := syscall.BytePtrFromString(path)
if err != nil {
return 0, err
}
fd, _, errno := syscall6(uintptr(unsafe.Pointer(&procOpenat)), 4, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(flags), uintptr(perm), 0, 0)
if errno != 0 {
return 0, errno
}
return int(fd), nil
}
func Fstatat(dirfd int, path string, stat *syscall.Stat_t, flags int) error {
p, err := syscall.BytePtrFromString(path)
if err != nil {
return err
}
_, _, errno := syscall6(uintptr(unsafe.Pointer(&procFstatat)), 4, uintptr(dirfd), uintptr(unsafe.Pointer(p)), uintptr(unsafe.Pointer(stat)), uintptr(flags), 0, 0)
if errno != 0 {
return errno
}
return nil
}

19
src/internal/syscall/unix/at_solaris.go Normal file
View file

@ -0,0 +1,19 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
import "syscall"
// Implemented as sysvicall6 in runtime/syscall_solaris.go.
func syscall6(trap, nargs, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2 uintptr, err syscall.Errno)
//go:cgo_import_dynamic libc_fstatat fstatat "libc.so"
//go:cgo_import_dynamic libc_openat openat "libc.so"
//go:cgo_import_dynamic libc_unlinkat unlinkat "libc.so"
const (
AT_REMOVEDIR = 0x1
AT_SYMLINK_NOFOLLOW = 0x1000
)

12
src/internal/syscall/unix/at_sysnum_darwin.go Normal file
View file

@ -0,0 +1,12 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
const unlinkatTrap uintptr = 472
const openatTrap uintptr = 463
const fstatatTrap uintptr = 470
const AT_REMOVEDIR = 0x80
const AT_SYMLINK_NOFOLLOW = 0x0020

14
src/internal/syscall/unix/at_sysnum_dragonfly.go Normal file
View file

@ -0,0 +1,14 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
import "syscall"
const unlinkatTrap uintptr = syscall.SYS_UNLINKAT
const openatTrap uintptr = syscall.SYS_OPENAT
const fstatatTrap uintptr = syscall.SYS_FSTATAT
const AT_REMOVEDIR = 0x2
const AT_SYMLINK_NOFOLLOW = 0x1

11
src/internal/syscall/unix/at_sysnum_fstatat64_linux.go Normal file
View file

@ -0,0 +1,11 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build arm mips mipsle 386
package unix
import "syscall"
const fstatatTrap uintptr = syscall.SYS_FSTATAT64

11
src/internal/syscall/unix/at_sysnum_fstatat_linux.go Normal file
View file

@ -0,0 +1,11 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build arm64
package unix
import "syscall"
const fstatatTrap uintptr = syscall.SYS_FSTATAT

13
src/internal/syscall/unix/at_sysnum_linux.go Normal file
View file

@ -0,0 +1,13 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
import "syscall"
const unlinkatTrap uintptr = syscall.SYS_UNLINKAT
const openatTrap uintptr = syscall.SYS_OPENAT
const AT_REMOVEDIR = 0x200
const AT_SYMLINK_NOFOLLOW = 0x100

14
src/internal/syscall/unix/at_sysnum_netbsd.go Normal file
View file

@ -0,0 +1,14 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
import "syscall"
const unlinkatTrap uintptr = syscall.SYS_UNLINKAT
const openatTrap uintptr = syscall.SYS_OPENAT
const fstatatTrap uintptr = syscall.SYS_FSTATAT
const AT_REMOVEDIR = 0x800
const AT_SYMLINK_NOFOLLOW = 0x200

11
src/internal/syscall/unix/at_sysnum_newfstatat_linux.go Normal file
View file

@ -0,0 +1,11 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build amd64 mips64 mips64le ppc64 ppc64le s390x
package unix
import "syscall"
const fstatatTrap uintptr = syscall.SYS_NEWFSTATAT

14
src/internal/syscall/unix/at_sysnum_openbsd.go Normal file
View file

@ -0,0 +1,14 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unix
import "syscall"
const unlinkatTrap uintptr = syscall.SYS_UNLINKAT
const openatTrap uintptr = syscall.SYS_OPENAT
const fstatatTrap uintptr = syscall.SYS_FSTATAT
const AT_REMOVEDIR = 0x08
const AT_SYMLINK_NOFOLLOW = 0x02

2
src/internal/syscall/windows/mksyscall.go
View file

@ -4,4 +4,4 @@
package windows
//go:generate go run $GOROOT/src/syscall/mksyscall_windows.go -output zsyscall_windows.go syscall_windows.go security_windows.go psapi_windows.go
//go:generate go run $GOROOT/src/syscall/mksyscall_windows.go -output zsyscall_windows.go syscall_windows.go security_windows.go psapi_windows.go symlink_windows.go

25
src/internal/syscall/windows/symlink_windows.go
View file

@ -11,4 +11,29 @@ const (
// symlink support for CreateSymbolicLink() starting with Windows 10 (1703, v10.0.14972)
SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE = 0x2
// FileInformationClass values
FileBasicInfo = 0 // FILE_BASIC_INFO
FileStandardInfo = 1 // FILE_STANDARD_INFO
FileNameInfo = 2 // FILE_NAME_INFO
FileStreamInfo = 7 // FILE_STREAM_INFO
FileCompressionInfo = 8 // FILE_COMPRESSION_INFO
FileAttributeTagInfo = 9 // FILE_ATTRIBUTE_TAG_INFO
FileIdBothDirectoryInfo = 0xa // FILE_ID_BOTH_DIR_INFO
FileIdBothDirectoryRestartInfo = 0xb // FILE_ID_BOTH_DIR_INFO
FileRemoteProtocolInfo = 0xd // FILE_REMOTE_PROTOCOL_INFO
FileFullDirectoryInfo = 0xe // FILE_FULL_DIR_INFO
FileFullDirectoryRestartInfo = 0xf // FILE_FULL_DIR_INFO
FileStorageInfo = 0x10 // FILE_STORAGE_INFO
FileAlignmentInfo = 0x11 // FILE_ALIGNMENT_INFO
FileIdInfo = 0x12 // FILE_ID_INFO
FileIdExtdDirectoryInfo = 0x13 // FILE_ID_EXTD_DIR_INFO
FileIdExtdDirectoryRestartInfo = 0x14 // FILE_ID_EXTD_DIR_INFO
)
type FILE_ATTRIBUTE_TAG_INFO struct {
FileAttributes uint32
ReparseTag uint32
}
//sys GetFileInformationByHandleEx(handle syscall.Handle, class uint32, info *byte, bufsize uint32) (err error)

12
src/internal/syscall/windows/syscall_windows.go
View file

@ -12,7 +12,11 @@ import (
const (
ERROR_SHARING_VIOLATION syscall.Errno = 32
ERROR_LOCK_VIOLATION syscall.Errno = 33
ERROR_NOT_SUPPORTED syscall.Errno = 50
ERROR_CALL_NOT_IMPLEMENTED syscall.Errno = 120
ERROR_INVALID_NAME syscall.Errno = 123
ERROR_LOCK_FAILED syscall.Errno = 167
ERROR_NO_UNICODE_TRANSLATION syscall.Errno = 1113
)
@ -255,6 +259,14 @@ func Rename(oldpath, newpath string) error {
return MoveFileEx(from, to, MOVEFILE_REPLACE_EXISTING)
}
//sys LockFileEx(file syscall.Handle, flags uint32, reserved uint32, bytesLow uint32, bytesHigh uint32, overlapped *syscall.Overlapped) (err error) = kernel32.LockFileEx
//sys UnlockFileEx(file syscall.Handle, reserved uint32, bytesLow uint32, bytesHigh uint32, overlapped *syscall.Overlapped) (err error) = kernel32.UnlockFileEx
const (
LOCKFILE_FAIL_IMMEDIATELY = 0x00000001
LOCKFILE_EXCLUSIVE_LOCK = 0x00000002
)
const MB_ERR_INVALID_CHARS = 8
//sys GetACP() (acp uint32) = kernel32.GetACP

83
src/internal/syscall/windows/zsyscall_windows.go
View file

@ -44,28 +44,31 @@ var (
moduserenv = syscall.NewLazyDLL(sysdll.Add("userenv.dll"))
modpsapi = syscall.NewLazyDLL(sysdll.Add("psapi.dll"))
procGetAdaptersAddresses = modiphlpapi.NewProc("GetAdaptersAddresses")
procGetComputerNameExW = modkernel32.NewProc("GetComputerNameExW")
procMoveFileExW = modkernel32.NewProc("MoveFileExW")
procGetModuleFileNameW = modkernel32.NewProc("GetModuleFileNameW")
procWSASocketW = modws2_32.NewProc("WSASocketW")
procGetACP = modkernel32.NewProc("GetACP")
procGetConsoleCP = modkernel32.NewProc("GetConsoleCP")
procMultiByteToWideChar = modkernel32.NewProc("MultiByteToWideChar")
procGetCurrentThread = modkernel32.NewProc("GetCurrentThread")
procNetShareAdd = modnetapi32.NewProc("NetShareAdd")
procNetShareDel = modnetapi32.NewProc("NetShareDel")
procGetFinalPathNameByHandleW = modkernel32.NewProc("GetFinalPathNameByHandleW")
procImpersonateSelf = modadvapi32.NewProc("ImpersonateSelf")
procRevertToSelf = modadvapi32.NewProc("RevertToSelf")
procOpenThreadToken = modadvapi32.NewProc("OpenThreadToken")
procLookupPrivilegeValueW = modadvapi32.NewProc("LookupPrivilegeValueW")
procAdjustTokenPrivileges = modadvapi32.NewProc("AdjustTokenPrivileges")
procDuplicateTokenEx = modadvapi32.NewProc("DuplicateTokenEx")
procSetTokenInformation = modadvapi32.NewProc("SetTokenInformation")
procGetProfilesDirectoryW = moduserenv.NewProc("GetProfilesDirectoryW")
procNetUserGetLocalGroups = modnetapi32.NewProc("NetUserGetLocalGroups")
procGetProcessMemoryInfo = modpsapi.NewProc("GetProcessMemoryInfo")
procGetAdaptersAddresses = modiphlpapi.NewProc("GetAdaptersAddresses")
procGetComputerNameExW = modkernel32.NewProc("GetComputerNameExW")
procMoveFileExW = modkernel32.NewProc("MoveFileExW")
procGetModuleFileNameW = modkernel32.NewProc("GetModuleFileNameW")
procWSASocketW = modws2_32.NewProc("WSASocketW")
procLockFileEx = modkernel32.NewProc("LockFileEx")
procUnlockFileEx = modkernel32.NewProc("UnlockFileEx")
procGetACP = modkernel32.NewProc("GetACP")
procGetConsoleCP = modkernel32.NewProc("GetConsoleCP")
procMultiByteToWideChar = modkernel32.NewProc("MultiByteToWideChar")
procGetCurrentThread = modkernel32.NewProc("GetCurrentThread")
procNetShareAdd = modnetapi32.NewProc("NetShareAdd")
procNetShareDel = modnetapi32.NewProc("NetShareDel")
procGetFinalPathNameByHandleW = modkernel32.NewProc("GetFinalPathNameByHandleW")
procImpersonateSelf = modadvapi32.NewProc("ImpersonateSelf")
procRevertToSelf = modadvapi32.NewProc("RevertToSelf")
procOpenThreadToken = modadvapi32.NewProc("OpenThreadToken")
procLookupPrivilegeValueW = modadvapi32.NewProc("LookupPrivilegeValueW")
procAdjustTokenPrivileges = modadvapi32.NewProc("AdjustTokenPrivileges")
procDuplicateTokenEx = modadvapi32.NewProc("DuplicateTokenEx")
procSetTokenInformation = modadvapi32.NewProc("SetTokenInformation")
procGetProfilesDirectoryW = moduserenv.NewProc("GetProfilesDirectoryW")
procNetUserGetLocalGroups = modnetapi32.NewProc("NetUserGetLocalGroups")
procGetProcessMemoryInfo = modpsapi.NewProc("GetProcessMemoryInfo")
procGetFileInformationByHandleEx = modkernel32.NewProc("GetFileInformationByHandleEx")
)
func GetAdaptersAddresses(family uint32, flags uint32, reserved uintptr, adapterAddresses *IpAdapterAddresses, sizePointer *uint32) (errcode error) {
@ -126,6 +129,30 @@ func WSASocket(af int32, typ int32, protocol int32, protinfo *syscall.WSAProtoco
return
}
func LockFileEx(file syscall.Handle, flags uint32, reserved uint32, bytesLow uint32, bytesHigh uint32, overlapped *syscall.Overlapped) (err error) {
r1, _, e1 := syscall.Syscall6(procLockFileEx.Addr(), 6, uintptr(file), uintptr(flags), uintptr(reserved), uintptr(bytesLow), uintptr(bytesHigh), uintptr(unsafe.Pointer(overlapped)))
if r1 == 0 {
if e1 != 0 {
err = errnoErr(e1)
} else {
err = syscall.EINVAL
}
}
return
}
func UnlockFileEx(file syscall.Handle, reserved uint32, bytesLow uint32, bytesHigh uint32, overlapped *syscall.Overlapped) (err error) {
r1, _, e1 := syscall.Syscall6(procUnlockFileEx.Addr(), 5, uintptr(file), uintptr(reserved), uintptr(bytesLow), uintptr(bytesHigh), uintptr(unsafe.Pointer(overlapped)), 0)
if r1 == 0 {
if e1 != 0 {
err = errnoErr(e1)
} else {
err = syscall.EINVAL
}
}
return
}
func GetACP() (acp uint32) {
r0, _, _ := syscall.Syscall(procGetACP.Addr(), 0, 0, 0, 0)
acp = uint32(r0)
@ -321,3 +348,15 @@ func GetProcessMemoryInfo(handle syscall.Handle, memCounters *PROCESS_MEMORY_COU
}
return
}
func GetFileInformationByHandleEx(handle syscall.Handle, class uint32, info *byte, bufsize uint32) (err error) {
r1, _, e1 := syscall.Syscall6(procGetFileInformationByHandleEx.Addr(), 4, uintptr(handle), uintptr(class), uintptr(unsafe.Pointer(info)), uintptr(bufsize), 0, 0)
if r1 == 0 {
if e1 != 0 {
err = errnoErr(e1)
} else {
err = syscall.EINVAL
}
}
return
}

23
src/net/dial.go
View file

@ -44,16 +44,23 @@ type Dialer struct {
// If nil, a local address is automatically chosen.
LocalAddr Addr
// DualStack enables RFC 6555-compliant "Happy Eyeballs"
// dialing when the network is "tcp" and the host in the
// address parameter resolves to both IPv4 and IPv6 addresses.
// This allows a client to tolerate networks where one address
// family is silently broken.
// DualStack previously enabled RFC 6555 Fast Fallback
// support, also known as "Happy Eyeballs", in which IPv4 is
// tried soon if IPv6 appears to be misconfigured and
// hanging.
//
// Deprecated: Fast Fallback is enabled by default. To
// disable, set FallbackDelay to a negative value.
DualStack bool
// FallbackDelay specifies the length of time to wait before
// spawning a fallback connection, when DualStack is enabled.
// spawning a RFC 6555 Fast Fallback connection. That is, this
// is the amount of time to wait for IPv6 to succeed before
// assuming that IPv6 is misconfigured and falling back to
// IPv4.
//
// If zero, a default delay of 300ms is used.
// A negative value disables Fast Fallback support.
FallbackDelay time.Duration
// KeepAlive specifies the keep-alive period for an active
@ -81,6 +88,8 @@ type Dialer struct {
Control func(network, address string, c syscall.RawConn) error
}
func (d *Dialer) dualStack() bool { return d.FallbackDelay >= 0 }
func minNonzeroTime(a, b time.Time) time.Time {
if a.IsZero() {
return b
@ -393,7 +402,7 @@ func (d *Dialer) DialContext(ctx context.Context, network, address string) (Conn
}
var primaries, fallbacks addrList
if d.DualStack && network == "tcp" {
if d.dualStack() && network == "tcp" {
primaries, fallbacks = addrs.partition(isIPv4)
} else {
primaries = addrs

12
src/net/platform_test.go
View file

@ -7,7 +7,9 @@ package net
import (
"internal/testenv"
"os"
"os/exec"
"runtime"
"strconv"
"strings"
"testing"
)
@ -35,6 +37,16 @@ func testableNetwork(network string) bool {
switch runtime.GOOS {
case "android", "nacl", "plan9", "windows":
return false
case "aix":
// Unix network isn't properly working on AIX 7.2 with Technical Level < 2
out, err := exec.Command("oslevel", "-s").Output()
if err != nil {
return false
}
if tl, err := strconv.Atoi(string(out[5:7])); err != nil || tl < 2 {
return false
}
return true
}
// iOS does not support unix, unixgram.
if runtime.GOOS == "darwin" && (runtime.GOARCH == "arm" || runtime.GOARCH == "arm64") {

31
src/net/tcpsock_test.go
View file

@ -796,3 +796,34 @@ func TestCopyPipeIntoTCP(t *testing.T) {
t.Fatal(err)
}
}
func BenchmarkSetReadDeadline(b *testing.B) {
ln, err := newLocalListener("tcp")
if err != nil {
b.Fatal(err)
}
defer ln.Close()
var serv Conn
done := make(chan error)
go func() {
var err error
serv, err = ln.Accept()
done <- err
}()
c, err := Dial("tcp", ln.Addr().String())
if err != nil {
b.Fatal(err)
}
defer c.Close()
if err := <-done; err != nil {
b.Fatal(err)
}
defer serv.Close()
c.SetWriteDeadline(time.Now().Add(2 * time.Hour))
deadline := time.Now().Add(time.Hour)
b.ResetTimer()
for i := 0; i < b.N; i++ {
c.SetReadDeadline(deadline)
deadline = deadline.Add(1)
}
}

11
src/net/textproto/reader.go
View file

@ -129,12 +129,13 @@ func (r *Reader) readContinuedLineSlice() ([]byte, error) {
}
// Optimistically assume that we have started to buffer the next line
// and it starts with an ASCII letter (the next header key), so we can
// avoid copying that buffered data around in memory and skipping over
// non-existent whitespace.
// and it starts with an ASCII letter (the next header key), or a blank
// line, so we can avoid copying that buffered data around in memory
// and skipping over non-existent whitespace.
if r.R.Buffered() > 1 {
peek, err := r.R.Peek(1)
if err == nil && isASCIILetter(peek[0]) {
peek, _ := r.R.Peek(2)
if len(peek) > 0 && (isASCIILetter(peek[0]) || peek[0] == '\n') ||
len(peek) == 2 && peek[0] == '\r' && peek[1] == '\n' {
return trim(line), nil
}
}

10
src/os/exec/exec_test.go
View file

@ -459,7 +459,7 @@ func basefds() uintptr {
// The poll (epoll/kqueue) descriptor can be numerically
// either between stderr and the testlog-fd, or after
// testlog-fd.
if poll.PollDescriptor() == n {
if poll.IsPollDescriptor(n) {
n++
}
for _, arg := range os.Args {
@ -472,7 +472,7 @@ func basefds() uintptr {
func closeUnexpectedFds(t *testing.T, m string) {
for fd := basefds(); fd <= 101; fd++ {
if fd == poll.PollDescriptor() {
if poll.IsPollDescriptor(fd) {
continue
}
err := os.NewFile(fd, "").Close()
@ -734,6 +734,8 @@ func TestHelperProcess(*testing.T) {
ofcmd = "fstat"
case "plan9":
ofcmd = "/bin/cat"
case "aix":
ofcmd = "procfiles"
}
args := os.Args
@ -837,7 +839,7 @@ func TestHelperProcess(*testing.T) {
// Now verify that there are no other open fds.
var files []*os.File
for wantfd := basefds() + 1; wantfd <= 100; wantfd++ {
if wantfd == poll.PollDescriptor() {
if poll.IsPollDescriptor(wantfd) {
continue
}
f, err := os.Open(os.Args[0])
@ -851,6 +853,8 @@ func TestHelperProcess(*testing.T) {
switch runtime.GOOS {
case "plan9":
args = []string{fmt.Sprintf("/proc/%d/fd", os.Getpid())}
case "aix":
args = []string{fmt.Sprint(os.Getpid())}
default:
args = []string{"-p", fmt.Sprint(os.Getpid())}
}

2
src/os/file_windows.go
View file

@ -362,7 +362,7 @@ func Symlink(oldname, newname string) error {
destpath = dirname(newname) + `\` + oldname
}
fi, err := Lstat(destpath)
fi, err := Stat(destpath)
isdir := err == nil && fi.IsDir()
n, err := syscall.UTF16PtrFromString(fixLongPath(newname))

Some files were not shown because too many files have changed in this diff Show more