Most changes are removing redundant declaration of type when direct
instantiating value of map or slice, e.g. []T{T{}} become []T{{}}.
Small changes are removing the high order of subslice if its value
is the length of slice itself, e.g. T[:len(T)] become T[:].
The following file is excluded due to incompatibility with go1.4,
- src/cmd/compile/internal/gc/ssa.go
Change-Id: Id3abb09401795ce1e6da591a89749cba8502fb26
Reviewed-on: https://go-review.googlesource.com/c/go/+/166437
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The performance improvement is not as big as we hoped.
Until the API is feature complete, we postpone the release
and avoid added complexity.
This change was prepared by reverting all the changes affected
src/cmd/trace and src/internal/traceparser packages after
golang.org/cl/137635, and then bringing back MMU computation
APIs (originally in src/internal/traceparser) to the
src/internal/trace package.
Revert "cmd/trace: use new traceparser to parse the raw trace files"
This reverts https://golang.org/cl/145457
(commit 08816cb8d7).
Revert "internal/traceparser: provide parser that uses less space and parses segments of runtime trace files"
This reverts https://golang.org/cl/137635
(commit daaf361f74).
Change-Id: Ic2a068a7dbaf4053cd9674ca7bde9c58e74385b4
Reviewed-on: https://go-review.googlesource.com/c/150517
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This uses the mutator utilization distribution to compute the p99.9,
p99, and p95 mutator utilization topograph lines and display them
along with the MMU.
Change-Id: I8c7e0ec326aa4bc00619ec7562854253f01cc802
Reviewed-on: https://go-review.googlesource.com/c/60800
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
The current MMU analysis considers all Ps together, so if, for
example, one of four Ps is blocked, mutator utilization is 75%.
However, this is less useful for understanding the impact on
individual goroutines because that one blocked goroutine could be
blocked for a very long time, but we still appear to have good
utilization.
Hence, this introduces a new flag that does a "per-P" analysis where
the utilization of each P is considered independently. The MMU is then
the combination of the MMU for each P's utilization function.
Change-Id: Id67b980d4d82b511d28300cdf92ccbb5ae8f0c78
Reviewed-on: https://go-review.googlesource.com/c/60797
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This adds the ability to click a point on the MMU graph to show a list
of the worst 10 mutator utilization windows of the selected size. This
list in turn links to the trace viewer to drill down on specifically
what happened in each specific window.
Change-Id: Ic1b72d8b37fbf2212211c513cf36b34788b30133
Reviewed-on: https://go-review.googlesource.com/c/60795
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This adds an endpoint to the trace tool that plots the minimum mutator
utilization curve using information on mark assists and GC pauses from
the trace.
This commit implements a fairly straightforward O(nm) algorithm for
computing the MMU (and tests against an even more direct but slower
algorithm). Future commits will extend and optimize this algorithm.
This should be useful for debugging and understanding mutator
utilization issues like #14951, #14812, #18155. #18534, #21107,
particularly once follow-up CLs add trace cross-referencing.
Change-Id: Ic2866869e7da1e6c56ba3e809abbcb2eb9c4923a
Reviewed-on: https://go-review.googlesource.com/c/60790
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns true if")
This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.
(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)
Created with:
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)
Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)
Updates #27864
Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For sweep events, we used to modify the ViewerEvent returned from
ctx.emitSlice later in order to embed more details about the sweep
operation. The trick no longer works after the change
https://golang.org/cl/92375 and caused a regression.
ctx.emit method encodes the ViewerEvent, so any modification to the
ViewerEvent object after ctx.emit returns will not be reflected.
Refactor ctx.emitSlice, so ctx.makeSlice can be used when producing
slices for SWEEP. ctx.emit* methods are meant to truely emit
ViewerEvents.
Fixes#27711
Change-Id: I0b733ebbbfd4facd8714db0535809ec3cab0833d
Reviewed-on: https://go-review.googlesource.com/135775
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.
JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.
Updates #18892
Updates #25506
Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
Reviewed-on: https://go-review.googlesource.com/114197
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
First, the regions sort was buggy, as its last comparison was
ineffective.
Second, the insyscall and insyscallRuntime fields were unsigned, so the
check for them being negative was pointless. Make them signed instead,
to also prevent the possibility of underflows when decreasing numbers
that might realistically be 0.
Third, the color constants were all untyped strings except the first
one. Be consistent with their typing.
Change-Id: I4eb8d08028ed92589493c2a4b9cc5a88d83f769b
Reviewed-on: https://go-review.googlesource.com/113895
Run-TryBot: Daniel Martà <mvdan@mvdan.cc>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I1cb7c8b70a5ae16386f6abb577c23d821f7ff7f0
Reviewed-on: https://go-review.googlesource.com/112197
Reviewed-by: Peter Weinberger <pjw@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Currently isSystemGoroutine has a hard-coded list of known entry
points into system goroutines. This list is annoying to maintain. For
example, it's missing the ensureSigM goroutine.
Replace it with a check that simply looks for any goroutine with
runtime function as its entry point, with a few exceptions. This also
matches the definition recently added to the trace viewer (CL 81315).
Change-Id: Iaed723d4a6e8c2ffb7c0c48fbac1688b00b30f01
Reviewed-on: https://go-review.googlesource.com/81655
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
and assign the same colors for spans belong to the tasks
(sadly, the trace viewer will change the saturation/ligthness
for asynchronous slices so exact color mapping is impossible.
But I hope they are not too far from each other)
Change-Id: Idaaf0828a1e0dac8012d336dcefa1c6572ddca2e
Reviewed-on: https://go-review.googlesource.com/109338
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Also change tasks to be represented as "slices" instead of
asynchronous events which are more efficiently represented in trace
viewer data model. This change allows to utilize the flow events
(arrows) to represent task hierarchies.
Introduced RegionArgs and TaskArgs where the task id infomation and
goroutine id informations are stored for information-purpose.
Change-Id: I11bec7dd716fdfc5f94ea39661b2e51344367a6f
Reviewed-on: https://go-review.googlesource.com/109337
Reviewed-by: Heschi Kreinick <heschi@google.com>
A task may have other user annotation events after the task ends.
So far, task.lastTimestamp returned the task end event if the
event available. This change introduces task.endTimestamp for that
and makes task.lastTimestamp returns the "last" seen event's timestamp
if the task is ended.
If the task is not ended, both returns the last timestamp of the entire
trace assuming the task is still active.
This fixes the task-oriented trace view mode not to drop user
annotation instances when they appear outside a task's lifespan.
Adds a test.
Change-Id: Iba1062914f224edd521b9ee55c6cd5e180e55359
Reviewed-on: https://go-review.googlesource.com/109175
Reviewed-by: Heschi Kreinick <heschi@google.com>
"Span" is a commonly used term in many distributed tracing systems
(Dapper, OpenCensus, OpenTracing, ...). They use it to refer to a
period of time, not necessarily tied into execution of underlying
processor, thread, or goroutine, unlike the "Span" of runtime/trace
package.
Since distributed tracing and go runtime execution tracing are
already similar enough to cause confusion, this CL attempts to avoid
using the same word if possible.
"Region" is being used in a certain tracing system to refer to a code
region which is pretty close to what runtime/trace.Span currently
refers to. So, replace that.
https://software.intel.com/en-us/itc-user-and-reference-guide-defining-and-recording-functions-or-regions
This CL also tweaks APIs a bit based on jbd and heschi's comments:
NewContext -> NewTask
and it now returns a Task object that exports End method.
StartSpan -> StartRegion
and it now returns a Region object that exports End method.
Also, changed WithSpan to WithRegion and it now takes func() with no
context. Another thought is to get rid of WithRegion. It is a nice
concept but in practice, it seems problematic (a lot of code churn,
and polluting stack trace). Already, the tracing concept is very low
level, and we hope this API to be used with great care.
Recommended usage will be
defer trace.StartRegion(ctx, "someRegion").End()
Left old APIs untouched in this CL. Once the usage of them are cleaned
up, they will be removed in a separate CL.
Change-Id: I73880635e437f3aad51314331a035dd1459b9f3a
Reviewed-on: https://go-review.googlesource.com/108296
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
golang.org/cl/102697 attempted to fix the span presentation by utilizing
the position of the span in the span slices of a task. But it is
not complete either.
First, id=0 is omitted in json encoding and the trace viewer silently
drops entries with the missing id field, so we must avoid zero-value id.
Second, it is possible that a goroutine handles multiple tasks. Then,
id collisions will happen.
This takes a simpler approach - have a counter that increments for every
emitSpan call, and use the value as the id value.
Change-Id: Idaa9505634acf6d327c6f00af32d8260955b85e1
Reviewed-on: https://go-review.googlesource.com/106755
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
/spanio, /spanblock, /spansched, /spansyscall provide
the pprof-style summary of span execution's
io, block, scheduling, syscall latency distributions
respectively.
The computation logic for /io, /block, /sched, /syscall
analysis was refactored and extended for reuse in these
new types of analysis. Upon the analysis query, we create
a map of goroutine id to time intervals based on the query
parameter, that represents the interesting time intervals
of matching goroutines. Only the events from the matching
goroutines that fall into the intervals are considered
in the pprof computation.
The new endpoints are not yet hooked into other span
analysis page (e.g. /userspan) yet.
Change-Id: I80c3396e45a2d6631758710de67d132e5832c7ce
Reviewed-on: https://go-review.googlesource.com/105822
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
All tests involving trace collection and parsing
still need handling of failures caused by #16755
(Timestamp issue)
Fixes#24738
Change-Id: I6cd0f9c6f49854a22fad6fce1a00964c168aa614
Reviewed-on: https://go-review.googlesource.com/105821
Reviewed-by: Peter Weinberger <pjw@google.com>
The trace viewer interprets the slice as a non-terminating
time interval which is quite opposit to what trace records indicate
(i.e., almostly immediately terminating time interval).
As observed in the issue #24663 this can result in quite misleading
visualization of the trace.
Work around the trace viewer's issue by setting a small value
(0.0001usec) as the duration if the time interval is not positive.
Change-Id: I1c2aac135c194d0717f5c01a98ca60ffb14ef45c
Reviewed-on: https://go-review.googlesource.com/104716
Reviewed-by: Heschi Kreinick <heschi@google.com>
This mode is similar to the default traceview mode where the execution
trace is presented in P-oriented way. Each row represents a P, and each
slice represents the time interval of a goroutine's execution on the P.
The difference is that, in this mode, only the execution of goroutines
involved in the specified task is highlighted, and other goroutine
execution or events are greyed out. So, users can focus on how a task is
executed while considering other affecting conditions such as other
goroutines, network events, or process scheduling.
Example: https://user-images.githubusercontent.com/4999471/38116793-a6f995f0-337f-11e8-8de9-88eec2f2c497.png
Here, for a while the program remained idle after the first burst of
activity related to the task because all other goroutines were also
being blocked or waiting for events, or no incoming network traffic
(indicated by the lack of any network activity). This is a bit hard to
discover when the usual task-oriented view (/trace?taskid=<taskid>)
mode.
Also, it simplifies the traceview generation mode logic.
/trace ---> 0
/trace?goid ---> modeGoroutineOriented
/trace?taskid ---> modeGoroutineOriented|modeTaskOriented
/trace?focustask ---> modeTaskOriented
Change-Id: Idcc0ae31b708ddfd19766f4e26ee7efdafecd3a5
Reviewed-on: https://go-review.googlesource.com/103555
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Mostly same as golang.org/cl/102156, except the parts that
deal with different data types.
Change-Id: I061b858b73898725e3bf175ed022c2e3e55fc485
Reviewed-on: https://go-review.googlesource.com/103158
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Spans are represented using Async Event types of chrome trace viewer.
According to the doc, the 'id' should be unique within category, scope.
https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview#heading=h.jh64i9l3vwa1
Use the index in the task's span slice as the slice id, so it
can be unique within the task. The scope is the task id which
is unique.
This fixes a visualization bug that caused incorrect or missing
presentation of nested spans.
Change-Id: If1537ee00247f71fa967abfe45569a9e7dbcdce7
Reviewed-on: https://go-review.googlesource.com/102697
Reviewed-by: Heschi Kreinick <heschi@google.com>
Move part of UserSpan event processing from cmd/trace.analyzeAnnotations
to internal/trace.GoroutineStats that returns analyzed per-goroutine
execution information. Now the execution information includes list of
spans and their execution information.
cmd/trace.analyzeAnnotations utilizes the span execution information
from internal/trace.GoroutineStats and connects them with task
information.
Change-Id: Ib7f79a3ba652a4ae55cd81ea17565bcc7e241c5c
Reviewed-on: https://go-review.googlesource.com/101917
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
- Summary: also includes links to pprof data.
- Sortable table: sorting is done on server-side. The intention is
that later, I want to add pagination feature and limit the page
size the browser has to handle.
- Stacked horizontal bar graph to present total time breakdown.
- Human-friendly time representation.
- No dependency on external fancy javascript libraries to allow
it to function without an internet connection.
Change-Id: I91e5c26746e59ad0329dfb61e096e11f768c7b73
Reviewed-on: https://go-review.googlesource.com/102156
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Unused variables in closures are currently not diagnosed by the
compiler (this is Issue #3059), while go/types catches them.
One unused variable in the cmd/trace tests is causing the go/types
test that typechecks the whole standard library to fail:
FAIL: TestStdlib (8.05s)
stdlib_test.go:223: cmd/trace/annotations_test.go:241:6: gcTime
declared but not used
FAIL
Remove it.
Updates #24464
Change-Id: I0f1b9db6ae1f0130616ee649bdbfdc91e38d2184
Reviewed-on: https://go-review.googlesource.com/101815
Reviewed-by: Daniel Martà <mvdan@mvdan.cc>
Add a search box to the top of the user task views that only displays
tasks containing a particular log message.
Change-Id: I92f4aa113f930954e8811416901e37824f0eb884
Reviewed-on: https://go-review.googlesource.com/100843
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Also grey out instants that represent events occurred outside the
task's span. Furthermore, if the unrelated instants represent user
annotation events but not for the task of the interest, skip rendering
completely.
This helps users to focus on the task-related events better.
UI screen shot:
https://gist.github.com/hyangah/1df5d2c8f429fd933c481e9636b89b55#file-golang-org_cl_99035
Change-Id: I2b5aef41584c827f8c1e915d0d8e5c95fe2b4b65
Reviewed-on: https://go-review.googlesource.com/99035
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
This is an updated version of golang.org/cl/96395, with the fix to
TestUserSpan.
This reverts commit 7b6f6267e90a8e4eab37a3f2164ba882e6222adb.
Change-Id: I31eec8ba0997f9178dffef8dac608e731ab70872
Reviewed-on: https://go-review.googlesource.com/98236
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Even though undocumented, the assumption is the Event's link field
points to the following event in the future. The new span/task event
processing breaks the assumption.
Change-Id: I4ce2f30c67c4f525ec0a121a7e43d8bdd2ec3f77
Reviewed-on: https://go-review.googlesource.com/96395
Reviewed-by: Heschi Kreinick <heschi@google.com>
runtime/trace test already skips tests in case of the timestamp
error.
Moreover, relax TestAnalyzeAnnotationGC test condition to
deal with the inaccuracy caused from use of cputicks in tracing.
Fixes#24081
Updates #16755
Change-Id: I708ecc6da202eaec07e431085a75d3dbfbf4cc06
Reviewed-on: https://go-review.googlesource.com/97757
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is for debugging the reported flaky tests.
Update #24081
Change-Id: Ica046928f675d69e38251a47a6f225efedce920c
Reviewed-on: https://go-review.googlesource.com/96855
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Enabled when the tool runs with DEBUG_MEMORY_USAGE=1 env var.
After reporting the usage, it waits until user enters input
(helpful when checking top or other memory monitor)
Also adds net/http/pprof to export debug endpoints.
From the trace included in #21870
$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out
2018/02/21 16:04:49 Parsing trace...
after parsing trace
Alloc: 3385747848 Bytes
Sys: 3661654648 Bytes
HeapReleased: 0 Bytes
HeapSys: 3488907264 Bytes
HeapInUse: 3426377728 Bytes
HeapAlloc: 3385747848 Bytes
Enter to continue...
2018/02/21 16:05:09 Serializing trace...
after generating trace
Alloc: 4908929616 Bytes
Sys: 5319063640 Bytes
HeapReleased: 0 Bytes
HeapSys: 5032411136 Bytes
HeapInUse: 4982865920 Bytes
HeapAlloc: 4908929616 Bytes
Enter to continue...
2018/02/21 16:05:18 Splitting trace...
after spliting trace
Alloc: 4909026200 Bytes
Sys: 5319063640 Bytes
HeapReleased: 0 Bytes
HeapSys: 5032411136 Bytes
HeapInUse: 4983046144 Bytes
HeapAlloc: 4909026200 Bytes
Enter to continue...
2018/02/21 16:05:39 Opening browser. Trace viewer is listening on http://127.0.0.1:33661
after httpJsonTrace
Alloc: 5288336048 Bytes
Sys: 7790245896 Bytes
HeapReleased: 0 Bytes
HeapSys: 7381123072 Bytes
HeapInUse: 5324120064 Bytes
HeapAlloc: 5288336048 Bytes
Enter to continue...
Change-Id: I88bb3cb1af3cb62e4643a8cbafd5823672b2e464
Reviewed-on: https://go-review.googlesource.com/92355
Reviewed-by: Peter Weinberger <pjw@google.com>
