Commit graph

57 commits

Author SHA1 Message Date
Hana (Hyang-Ah) Kim
43b18f4241 cmd/trace: handle invalid goid para in /trace
Change-Id: I1cb7c8b70a5ae16386f6abb577c23d821f7ff7f0
Reviewed-on: https://go-review.googlesource.com/112197
Reviewed-by: Peter Weinberger <pjw@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-05-08 19:25:01 +00:00
Austin Clements
44286b17c5 runtime: replace system goroutine whitelist with symbol test
Currently isSystemGoroutine has a hard-coded list of known entry
points into system goroutines. This list is annoying to maintain. For
example, it's missing the ensureSigM goroutine.

Replace it with a check that simply looks for any goroutine with
runtime function as its entry point, with a few exceptions. This also
matches the definition recently added to the trace viewer (CL 81315).

Change-Id: Iaed723d4a6e8c2ffb7c0c48fbac1688b00b30f01
Reviewed-on: https://go-review.googlesource.com/81655
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-05-07 21:38:40 +00:00
Hana Kim
af5143e384 cmd/trace: use different colors for tasks
and assign the same colors for spans belong to the tasks
(sadly, the trace viewer will change the saturation/ligthness
for asynchronous slices so exact color mapping is impossible.
But I hope they are not too far from each other)

Change-Id: Idaaf0828a1e0dac8012d336dcefa1c6572ddca2e
Reviewed-on: https://go-review.googlesource.com/109338
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-29 18:14:22 +00:00
Hana Kim
e357ece0af cmd/trace: have tasks in a separate section (process group)
Also change tasks to be represented as "slices" instead of
asynchronous events which are more efficiently represented in trace
viewer data model. This change allows to utilize the flow events
(arrows) to represent task hierarchies.

Introduced RegionArgs and TaskArgs where the task id infomation and
goroutine id informations are stored for information-purpose.

Change-Id: I11bec7dd716fdfc5f94ea39661b2e51344367a6f
Reviewed-on: https://go-review.googlesource.com/109337
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-26 20:56:44 +00:00
Hana Kim
011f6c5fa0 cmd/trace: distinguish task endTimestamp and lastTimestamp
A task may have other user annotation events after the task ends.
So far, task.lastTimestamp returned the task end event if the
event available. This change introduces task.endTimestamp for that
and makes task.lastTimestamp returns the "last" seen event's timestamp
if the task is ended.

If the task is not ended, both returns the last timestamp of the entire
trace assuming the task is still active.

This fixes the task-oriented trace view mode not to drop user
annotation instances when they appear outside a task's lifespan.
Adds a test.

Change-Id: Iba1062914f224edd521b9ee55c6cd5e180e55359
Reviewed-on: https://go-review.googlesource.com/109175
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-24 21:49:40 +00:00
Hana Kim
c2d1024368 runtime/trace: rename "Span" with "Region"
"Span" is a commonly used term in many distributed tracing systems
(Dapper, OpenCensus, OpenTracing, ...). They use it to refer to a
period of time, not necessarily tied into execution of underlying
processor, thread, or goroutine, unlike the "Span" of runtime/trace
package.

Since distributed tracing and go runtime execution tracing are
already similar enough to cause confusion, this CL attempts to avoid
using the same word if possible.

"Region" is being used in a certain tracing system to refer to a code
region which is pretty close to what runtime/trace.Span currently
refers to. So, replace that.
https://software.intel.com/en-us/itc-user-and-reference-guide-defining-and-recording-functions-or-regions

This CL also tweaks APIs a bit based on jbd and heschi's comments:

  NewContext -> NewTask
    and it now returns a Task object that exports End method.

  StartSpan -> StartRegion
    and it now returns a Region object that exports End method.

Also, changed WithSpan to WithRegion and it now takes func() with no
context. Another thought is to get rid of WithRegion. It is a nice
concept but in practice, it seems problematic (a lot of code churn,
and polluting stack trace). Already, the tracing concept is very low
level, and we hope this API to be used with great care.

Recommended usage will be
   defer trace.StartRegion(ctx, "someRegion").End()

Left old APIs untouched in this CL. Once the usage of them are cleaned
up, they will be removed in a separate CL.

Change-Id: I73880635e437f3aad51314331a035dd1459b9f3a
Reviewed-on: https://go-review.googlesource.com/108296
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
2018-04-24 16:33:15 +00:00
Hana Kim
1d8fc211f9 cmd/trace: change span id computation for trace view use
golang.org/cl/102697 attempted to fix the span presentation by utilizing
the position of the span in the span slices of a task. But it is
not complete either.

First, id=0 is omitted in json encoding and the trace viewer silently
drops entries with the missing id field, so we must avoid zero-value id.
Second, it is possible that a goroutine handles multiple tasks. Then,
id collisions will happen.

This takes a simpler approach - have a counter that increments for every
emitSpan call, and use the value as the id value.

Change-Id: Idaa9505634acf6d327c6f00af32d8260955b85e1
Reviewed-on: https://go-review.googlesource.com/106755
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-13 16:18:30 +00:00
Hana Kim
e6ab614fda cmd/trace: avoid emitting traceview slice with 0 duration
The trace viewer interprets the slice as a non-terminating
time interval which is quite opposit to what trace records indicate
(i.e., almostly immediately terminating time interval).
As observed in the issue #24663 this can result in quite misleading
visualization of the trace.

Work around the trace viewer's issue by setting a small value
(0.0001usec) as the duration if the time interval is not positive.

Change-Id: I1c2aac135c194d0717f5c01a98ca60ffb14ef45c
Reviewed-on: https://go-review.googlesource.com/104716
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-04 18:15:20 +00:00
Hana Kim
c4874aa2a2 cmd/trace: implement /trace?focustask=<taskid> mode
This mode is similar to the default traceview mode where the execution
trace is presented in P-oriented way. Each row represents a P, and each
slice represents the time interval of a goroutine's execution on the P.

The difference is that, in this mode, only the execution of goroutines
involved in the specified task is highlighted, and other goroutine
execution or events are greyed out. So, users can focus on how a task is
executed while considering other affecting conditions such as other
goroutines, network events, or process scheduling.

Example: https://user-images.githubusercontent.com/4999471/38116793-a6f995f0-337f-11e8-8de9-88eec2f2c497.png

Here, for a while the program remained idle after the first burst of
activity related to the task because all other goroutines were also
being blocked or waiting for events, or no incoming network traffic
(indicated by the lack of any network activity). This is a bit hard to
discover when the usual task-oriented view (/trace?taskid=<taskid>)
mode.

Also, it simplifies the traceview generation mode logic.
  /trace ---> 0
  /trace?goid ---> modeGoroutineOriented
  /trace?taskid ---> modeGoroutineOriented|modeTaskOriented
  /trace?focustask ---> modeTaskOriented

Change-Id: Idcc0ae31b708ddfd19766f4e26ee7efdafecd3a5
Reviewed-on: https://go-review.googlesource.com/103555
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-04 18:14:40 +00:00
Hana Kim
aaeaad6870 cmd/trace: assign a unique span id for slice representation
Spans are represented using Async Event types of chrome trace viewer.
According to the doc, the 'id' should be unique within category, scope.

https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview#heading=h.jh64i9l3vwa1

Use the index in the task's span slice as the slice id, so it
can be unique within the task. The scope is the task id which
is unique.

This fixes a visualization bug that caused incorrect or missing
presentation of nested spans.

Change-Id: If1537ee00247f71fa967abfe45569a9e7dbcdce7
Reviewed-on: https://go-review.googlesource.com/102697
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-27 19:03:54 +00:00
Hana Kim
f0eca373be cmd/trace: add /userspans, /userspan pages
Change-Id: Ifbefb659a8df3b079d69679871af444b179deaeb
Reviewed-on: https://go-review.googlesource.com/102599
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-26 22:30:52 +00:00
Hana Kim
68a1c9c400 internal/trace: compute span stats as computing goroutine stats
Move part of UserSpan event processing from cmd/trace.analyzeAnnotations
to internal/trace.GoroutineStats that returns analyzed per-goroutine
execution information. Now the execution information includes list of
spans and their execution information.

cmd/trace.analyzeAnnotations utilizes the span execution information
from internal/trace.GoroutineStats and connects them with task
information.

Change-Id: Ib7f79a3ba652a4ae55cd81ea17565bcc7e241c5c
Reviewed-on: https://go-review.googlesource.com/101917
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-03-26 16:59:01 +00:00
Hana Kim
6b5a0b5c16 cmd/trace: set cname for span slices
Define a set of color names available in trace viewer

https://user-images.githubusercontent.com/4999471/37063995-5d0bad48-2169-11e8-92be-9cb363e21c38.png

Change-Id: I312fcbc5430d7512b4c39ddc79a769259bad8c22
Reviewed-on: https://go-review.googlesource.com/99055
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-09 01:58:41 +00:00
Hana Kim
476f71c958 cmd/trace: remove unrelated arrows in task-oriented traceview
Also grey out instants that represent events occurred outside the
task's span. Furthermore, if the unrelated instants represent user
annotation events but not for the task of the interest, skip rendering
completely.

This helps users to focus on the task-related events better.

UI screen shot:
https://gist.github.com/hyangah/1df5d2c8f429fd933c481e9636b89b55#file-golang-org_cl_99035

Change-Id: I2b5aef41584c827f8c1e915d0d8e5c95fe2b4b65
Reviewed-on: https://go-review.googlesource.com/99035
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-09 01:34:12 +00:00
Hana Kim
93b0261d0a cmd/trace: force GC occassionally
to return memory to the OS after completing potentially
large operations.

Update #21870

Sys went down to 3.7G

$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out

2018/03/07 09:35:52 Parsing trace...
after parsing trace
 Alloc:	3385754360 Bytes
 Sys:	3662047864 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488907264 Bytes
 HeapInUse:	3426549760 Bytes
 HeapAlloc:	3385754360 Bytes
Enter to continue...
2018/03/07 09:36:09 Splitting trace...
after spliting trace
 Alloc:	3238309424 Bytes
 Sys:	3684410168 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488874496 Bytes
 HeapInUse:	3266461696 Bytes
 HeapAlloc:	3238309424 Bytes
Enter to continue...
2018/03/07 09:36:39 Opening browser. Trace viewer is listening on http://100.101.224.241:12345

after httpJsonTrace
 Alloc:	3000633872 Bytes
 Sys:	3693978424 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488743424 Bytes
 HeapInUse:	3030966272 Bytes
 HeapAlloc:	3000633872 Bytes
Enter to continue...

Change-Id: I56f64cae66c809cbfbad03fba7bd0d35494c1d04
Reviewed-on: https://go-review.googlesource.com/92376
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-03-07 14:39:25 +00:00
Hana Kim
ee465831ec cmd/trace: generate jsontrace data in a streaming fashion
Update #21870

The Sys went down to 4.25G from 6.2G.

$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out
2018/03/07 08:49:01 Parsing trace...
after parsing trace
 Alloc:	3385757184 Bytes
 Sys:	3661195896 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488841728 Bytes
 HeapInUse:	3426516992 Bytes
 HeapAlloc:	3385757184 Bytes
Enter to continue...
2018/03/07 08:49:18 Splitting trace...
after spliting trace
 Alloc:	2352071904 Bytes
 Sys:	4243825464 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	4025712640 Bytes
 HeapInUse:	2377703424 Bytes
 HeapAlloc:	2352071904 Bytes
Enter to continue...
after httpJsonTrace
 Alloc:	3228697832 Bytes
 Sys:	4250379064 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	4025647104 Bytes
 HeapInUse:	3260014592 Bytes
 HeapAlloc:	3228697832 Bytes

Change-Id: I546f26bdbc68b1e58f1af1235a0e299dc0ff115e
Reviewed-on: https://go-review.googlesource.com/92375
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-03-07 14:33:54 +00:00
Hana Kim
a66af7280d cmd/trace: add memory usage reporting
Enabled when the tool runs with DEBUG_MEMORY_USAGE=1 env var.
After reporting the usage, it waits until user enters input
(helpful when checking top or other memory monitor)

Also adds net/http/pprof to export debug endpoints.

From the trace included in #21870

$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out
2018/02/21 16:04:49 Parsing trace...
after parsing trace
 Alloc:	3385747848 Bytes
 Sys:	3661654648 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488907264 Bytes
 HeapInUse:	3426377728 Bytes
 HeapAlloc:	3385747848 Bytes
Enter to continue...
2018/02/21 16:05:09 Serializing trace...
after generating trace
 Alloc:	4908929616 Bytes
 Sys:	5319063640 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	5032411136 Bytes
 HeapInUse:	4982865920 Bytes
 HeapAlloc:	4908929616 Bytes
Enter to continue...
2018/02/21 16:05:18 Splitting trace...
after spliting trace
 Alloc:	4909026200 Bytes
 Sys:	5319063640 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	5032411136 Bytes
 HeapInUse:	4983046144 Bytes
 HeapAlloc:	4909026200 Bytes
Enter to continue...
2018/02/21 16:05:39 Opening browser. Trace viewer is listening on http://127.0.0.1:33661
after httpJsonTrace
 Alloc:	5288336048 Bytes
 Sys:	7790245896 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	7381123072 Bytes
 HeapInUse:	5324120064 Bytes
 HeapAlloc:	5288336048 Bytes
Enter to continue...

Change-Id: I88bb3cb1af3cb62e4643a8cbafd5823672b2e464
Reviewed-on: https://go-review.googlesource.com/92355
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-02-21 21:23:08 +00:00
Hana Kim
3e1ac1b017 cmd/trace: include P info in goroutine slices
The task-oriented trace view presents the execution trace organized
based on goroutines. Often, which P a goroutine was running on is
useful, so this CL includes the P ids in the goroutine execution slices.

R=go1.11

Change-Id: I96539bf8215e5c1cd8cc997a90204f57347c48c8
Reviewed-on: https://go-review.googlesource.com/90221
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-21 20:14:30 +00:00
Hana Kim
f42418b25d cmd/trace: add user log event in the task-oriented trace view
Also append stack traces to task create/end slices.

R=go1.11

Change-Id: I2adb342e92b36d30bee2860393618eb4064450cf
Reviewed-on: https://go-review.googlesource.com/90220
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-21 20:13:47 +00:00
Hana Kim
cacf8127e3 cmd/trace: present the GC time in the usertask view
The GC time for a task is defined by the sum of GC duration
overlapping with the task's duration.

Also, grey out non-overlapping slices in the task-oriented
trace view.

R=go1.11

Change-Id: I42def0eb520f5d9bd07edd265e558706f6fab552
Reviewed-on: https://go-review.googlesource.com/90219
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-21 19:48:35 +00:00
Hana Kim
fdcf4f712b cmd/trace: task-oriented view includes child tasks
R=go1.11

Change-Id: Ibb09e309c745eba811a0b53000c063bc10a055e1
Reviewed-on: https://go-review.googlesource.com/90218
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-02-20 19:44:53 +00:00
Hana Kim
d6856036bf cmd/trace: extend trace view (/trace) for task-oriented view
R=go1.11

Change-Id: I2d2db148fed96d0fcb228bee414b050fe4e46e2c
Reviewed-on: https://go-review.googlesource.com/90217
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-20 19:28:38 +00:00
Hana Kim
d07581f28e cmd/trace: add analyzeAnnotation and /usertasks view.
R=go1.11

Change-Id: I5078ab714c8ac2c652e6ec496e01b063235a014a
Reviewed-on: https://go-review.googlesource.com/90216
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-20 19:25:37 +00:00
Austin Clements
21ced9c748 cmd/trace: encode selection in trace URL
This adds the ability to add a #x:y anchor to the trace view URL that
causes the viewer to initially select from x ms to y ms.

Change-Id: I4a980d8128ecc85dbe41f224e8ae336707a4eaab
Reviewed-on: https://go-review.googlesource.com/60794
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-02-20 18:22:54 +00:00
Hana Kim
a58286c289 cmd/trace: init goroutine info entries with GoCreate event
golang.org/cl/81315 attempted to distinguish system goroutines
by examining the function name in the goroutine stack. It assumes that
the information would be available when GoSysBlock or GoInSyscall
events are processed, but it turned out the stack information is
set too late (when the goroutine gets a chance to run).

This change initializes the goroutine information entry when
processing GoCreate event which should be one of the very first
events for the every goroutine in trace.

Fixes #22574

Change-Id: I1ed37087ce2e78ed27c9b419b7d942eb4140cc69
Reviewed-on: https://go-review.googlesource.com/83595
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-12-20 23:04:21 +00:00
Hana Kim
662938850b cmd/trace: exclude threads in syscall on behalf of runtime
The number of threads in syscall presented by execution tracer's
trace view includes not only the threads calling system calls on behalf
of user created goroutines, but also those running on behalf of system
goroutines.

When the number of such system goroutines was small, the graph was
useful when examining where a program was saturating the CPU.
But as more and more system goroutines are invloved the graph became
less useful for the purpose - for example, after golang.org/cl/34784,
the timer goroutines dominate in the graph with large P
because the runtime creates per-P timer goroutines.

This change excludes the threads in syscall on behalf of runtime (system
goroutines) from the visualization. Alternatively, I could visualize the
count of such threads in a separate counter but in the same graph.
Given that many other debug endpoints (e.g. /debug/pprof/goroutine) hide
the system goroutines, including them in the same graph can confuse users.

Update #22574

Change-Id: If758cd6b9ed0596fde9a471e846b93246580b9d5
Reviewed-on: https://go-review.googlesource.com/81315
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-12-01 21:00:50 +00:00
Russ Cox
918b98ca70 cmd/internal/obj, cmd/trace: restore bounds checks dropped in CL 56950
CL 56950 correctly identified code with checks that were impossible.
But instead of correcting the checks it deleted them.
This CL corrects the code to check what was meant.

Change-Id: Ic89222184ee4fa5cacccae12d750601a9438ac8d
Reviewed-on: https://go-review.googlesource.com/78113
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-11-16 16:29:08 +00:00
Hana Kim
2f40dc79e5 cmd/trace: fix a javascript bug in handling import error
When traceviewer encounters a failure of json trace import
due to data error, onImportFail tried to access an error variable
which was not yet defined.

Change-Id: I431be03f179aafacaf1fd3c62a6337e8b5bd18fb
Reviewed-on: https://go-review.googlesource.com/71970
Reviewed-by: Austin Clements <austin@google.com>
2017-10-20 16:34:50 +00:00
Austin Clements
b0392159f6 runtime,cmd/trace: trace GC STW events
Right now we only kind of sort of trace GC STW events. We emit events
around mark termination, but those start well after stopping the world
and end before starting it again, and we don't emit any events for
sweep termination.

Fix this by generalizing EvGCScanStart/EvGCScanDone. These were
already re-purposed to indicate mark termination (despite the names).
This commit renames them to EvGCSTWStart/EvGCSTWDone, adds an argument
to indicate the STW reason, and shuffles the runtime to generate them
right before stopping the world and right after starting the world,
respectively.

These events will make it possible to generate precise minimum mutator
utilization (MMU) graphs and could be useful in detecting
non-preemptible goroutines (e.g., #20792).

Change-Id: If95783f370781d8ef66addd94886028103a7c26f
Reviewed-on: https://go-review.googlesource.com/55411
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-08-29 21:54:55 +00:00
Hana Kim
05ff6bfe33 misc/trace: update trace-viewer
Generated with
 github.com/catapult/tracing/bin/vulcanize_trace_viewer
catapult @ ab4d571fa

Renamed trace_viewer_lean.html to trace_viewer_full.html
to make it clear we are using the full version of trace viewer
(waiting for https://github.com/catapult-project/catapult/issues/2247
to be fixed).

Update #15302

Change-Id: Ice808bb27ab79a1dec9fc863e0c5a761027ebfbe
Reviewed-on: https://go-review.googlesource.com/58750
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2017-08-25 15:50:29 +00:00
Daniel Martí
943dd0fe33 cmd/*: remove negative uint checks
All of these are uints of different sizes, so checking >= 0 or < 0 are
effectively no-ops.

Found with staticcheck.

Change-Id: I16ac900eb7007bc8f9018b302136d42e483a4180
Reviewed-on: https://go-review.googlesource.com/56950
Reviewed-by: Matt Layher <mdlayher@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Matt Layher <mdlayher@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-18 18:45:05 +00:00
Austin Clements
a1eec645e1 cmd/trace: don't shift trace slices to 0
Currently all trace slices get shifted to start at time 0. This makes
it very difficult to find specific points in time unless they fall in
the first slice.

For example, right now when you click "View trace
(6.005646218s-8.155419698s)" on the trace tool's main page, the trace
view puts the first event in that slice at time 0. If you're looking
for something that happened at time 7s, you have to look at time
0.9943537s in the trace view. And if you want to subtract times taken
from different slices, you have to figure out what those time really
correspond to.

Fix this by telling the trace viewer not to shift the times when it
imports the trace. In the above example, this makes the view of that
second trace slice start at time 6.005646218s, so you don't have to do
any gymnastics to find or calculate times in later slices.

Change-Id: I04e0afda60f5573fdd8ad96238c24013297ef263
Reviewed-on: https://go-review.googlesource.com/54633
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-08-11 17:53:17 +00:00
Austin Clements
55f19b8d1d cmd/trace: update HTML; expand viewer to whole window
This updates the HTML served for the trace viewer to follow the latest
revision of the example from the upstream tracing project.

The main thing this adds is CSS for the trace viewer (which was
actually in the example at the originally referenced revision, so I'm
not sure why it got dropped). In particular, this expands the trace
viewer to use the entire browser client area, which fixes several
problems with the current page:

1. The details pane gets cut off at a strange place and can get a
scroll bar even if there's plenty of room below it on the page. This
fixes the bottom of the details pane to the bottom of the window.

2. If the track view is very tall (lots of procs), there's no way to
view the top tracks and the details pane at the same time. This fixes
this problem by limiting the height of the track view to something
less than the height of the window so it gets a scroll bar of its own
if necessary.

3. Dragging the divider between the track pane and the details pane
actually moves the bottom of the details pane without moving the
divider. Fixing the height of the trace viewer fixes this problem.

Change-Id: Ia811e72a7413417ca21c45e932c9db2724974633
Reviewed-on: https://go-review.googlesource.com/54632
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-08-11 17:52:41 +00:00
Austin Clements
7ca53f2f8b cmd/trace: show swept and reclaimed bytes
This displays the swept and reclaimed bytes for sweep events in the
lower panel of the trace viewer.

Change-Id: If1665a1c02bbc47700e0d9f515e574f013f3f285
Reviewed-on: https://go-review.googlesource.com/40812
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-04-19 18:31:16 +00:00
Heschi Kreinick
4de2efe927 cmd/trace: traces may end with pending mark assists
There's no guarantee that all in-progress mark assists will finish
before the trace does. Don't crash if that happens.

I haven't added a test because there's quite a bit of ceremony involved
and the bug is fairly straightforward.

Change-Id: Ia1369a8e2260fc6a328ad204a1eab1063d2e2c90
Reviewed-on: https://go-review.googlesource.com/37540
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 18:33:54 +00:00
Heschi Kreinick
2a74b9e814 cmd/trace: Record mark assists in execution traces
During the mark phase of garbage collection, goroutines that allocate
may be recruited to assist. This change creates trace events for mark
assists and displays them similarly to sweep assists in the trace
viewer.

Mark assists are different than sweeps in that they can be preempted, so
displaying them in the trace viewer is a little tricky -- we may need to
synthesize multiple slices for one mark assist. This could have been
done in the parser instead, but I thought it might be preferable to keep
the parser as true to the event stream as possible.

Change-Id: I381dcb1027a187a354b1858537851fa68a620ea7
Reviewed-on: https://go-review.googlesource.com/36015
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-02-10 18:03:42 +00:00
Austin Clements
d0b3c169ac cmd/trace: fix goroutine view
Currently, trace processing interleaves state/statistics updates and
emitting trace viewer objects. As a result, if events are being
filtered, either by time or by goroutines, we'll miss those
state/statistics updates. At best, this leads to bad statistics;
however, since we're now strictly checking G state transitions, it
usually leads to a failure to process the trace if there is any
filtering.

Fix this by separating state updates from emitting trace object. State
updates are done before filtering, so we always have correct state
information and statistics. Trace objects are only emitted if we pass
the filter. To determine when we need to emit trace counters, rather
than duplicating the knowledge of which events might modify
statistics, we keep track of the previously emitted counters and emit
a trace counter object whenever these have changed.

Fixes #17719.

Change-Id: Ic66e3ddaef60d1acaaf2ff4c62baa5352799cf99
Reviewed-on: https://go-review.googlesource.com/32810
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-11-18 17:49:18 +00:00
Austin Clements
6da83c6fc0 runtime, cmd/trace: track goroutines blocked on GC assists
Currently when a goroutine blocks on a GC assist, it emits a generic
EvGoBlock event. Since assist blocking events and, in particular, the
length of the blocked assist queue, are important for diagnosing GC
behavior, this commit adds a new EvGoBlockGC event for blocking on a
GC assist. The trace viewer uses this event to report a "waiting on
GC" count in the "Goroutines" row. This makes sense because, unlike
other blocked goroutines, these goroutines do have work to do, so
being blocked on a GC assist is quite similar to being in the
"runnable" state, which we also report in the trace viewer.

Change-Id: Ic21a326992606b121ea3d3d00110d8d1fdc7a5ef
Reviewed-on: https://go-review.googlesource.com/30704
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-28 14:29:47 +00:00
Austin Clements
640e916915 cmd/trace: track each G's state explicitly
Currently the trace tool tracks an overall counts of goroutine states,
but not the states of any individual goroutine. We're about to add
more sophisticated blocked-state tracking, so add this tracking and
base the state counts off the tracked goroutine states.

Change-Id: I943ed61782436cf9540f4ee26c5561715c5b4a1d
Reviewed-on: https://go-review.googlesource.com/30703
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-28 14:29:44 +00:00
Austin Clements
6834839427 runtime, cmd/trace: annotate different mark worker types
Currently mark workers are shown in the trace as regular goroutines
labeled "runtime.gcBgMarkWorker". That's somewhat unhelpful to an end
user because of the opaque label and particularly unhelpful to runtime
developers because it doesn't distinguish the different types of mark
workers.

Fix this by introducing a variant of the GoStart event called
GoStartLabel that lets the runtime indicate a label for a goroutine
execution span and using this to label mark worker executions as "GC
(<mode>)" in the trace viewer.

Since this bumps the trace version to 1.8, we also add test data for
1.7 traces.

Change-Id: Id7b9c0536508430c661ffb9e40e436f3901ca121
Reviewed-on: https://go-review.googlesource.com/30702
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-28 14:29:40 +00:00
Hyang-Ah (Hana) Kim
c24cc40075 cmd/trace: fix a runnable goroutine count bug
When starting tracing, EvGoCreate events are added for existing
goroutines that may have been blocking in syscall. EvGoCreate
increments the runnable goroutine count. This change makes the
following EvGoInSyscall event decrement the runnable goroutine count
because we now know that goroutine is in syscall, and not runnable.

Made generateTrace return an error, at any given time, the number
of runnable/running/insyscall goroutines becomes non-negative.

Added a basic test that checks the number of runnable/running
goroutines don't include the goroutines in syscall - the test failed
before this change.

Change-Id: Ib732c382e7bd17158a437576f9d589ab89097ce6
Reviewed-on: https://go-review.googlesource.com/25552
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-11 12:07:44 +00:00
Austin Clements
94589054d3 cmd/trace: label mark termination spans as such
Currently these are labeled "MARK", which was accurate in the STW
collector, but these really indicate mark termination now, since
marking happens for the full duration of the concurrent GC. Re-label
them as "MARK TERMINATION" to clarify this.

Change-Id: Ie98bd961195acde49598b4fa3f9e7d90d757c0a6
Reviewed-on: https://go-review.googlesource.com/30018
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-07 18:33:23 +00:00
Austin Clements
eed309f5fb cmd/trace: move process-wide GC events to their own row
Currently, the process-wide GC state is attributed to the P that
happened to perform the allocation that exceeded the GC trigger. This
is pretty arbitrary and makes it hard to see when GC is running since
the GC spans are intermingled with a lot of other trace noise.

The current display is particularly confusing because it creates three
sub-rows in the P row that can overlap each other. Usually a P has
just two sub-rows: one showing the current G and another showing that
G's activity. However, because GC is attributed to a proc, it winds up
as a third row that neither subsumes nor is subsumed by any other row.
This in turn screws up the trace's layout and results in overlapping
events.

Fix these problems by creating a new dedicated row like the existing
"Network" and "Timer" rows and displaying process-wide GC events in
this row. Mark termination and sweep events still appear in their
respective P rows because these are meaningfully attributed.

Change-Id: Ie1a1c6cf8c446e4b043f10f3968f91ff1b546d15
Reviewed-on: https://go-review.googlesource.com/30017
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-07 18:33:12 +00:00
Dmitry Vyukov
7ae273923c cmd/trace: split large traces into parts
Trace viewer cannot handle traces larger than 256MB (limit on js string size):
https://github.com/catapult-project/catapult/issues/627
And even that is problematic (chrome hangs and crashes).
Split large traces into 100MB parts. Somewhat clumsy, but I don't see any other
solution (other than rewriting trace viewer). At least it works reliably now.

Fixes #15482

Change-Id: I993b5f43d22072c6f5bd041ab5888ce176f272b2
Reviewed-on: https://go-review.googlesource.com/22731
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2016-05-12 17:53:59 +00:00
Dmitry Vyukov
2c9d773f74 misc/trace: update trace viewer html
The old trace-viewer is broken since Chrome 49:
https://bugs.chromium.org/p/chromium/issues/detail?id=569417
It was fixed in:
506457cbd7

This change updates trace-viewer to the latest version
(now it is called catapult).

This version has a bug in the lean config that we use, though:
https://github.com/catapult-project/catapult/issues/2247
So use full config for now (it works, but leads to larger html).
When the bug is fixed we need to switch back to lean config (issue #15302).

Change-Id: Ifb8d782ced66e3292d81c5604039fe18eaf267c5
Reviewed-on: https://go-review.googlesource.com/22013
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-04-14 14:48:04 +00:00
Brad Fitzpatrick
519474451a all: make copyright headers consistent with one space after period
This is a subset of https://golang.org/cl/20022 with only the copyright
header lines, so the next CL will be smaller and more reviewable.

Go policy has been single space after periods in comments for some time.

The copyright header template at:

    https://golang.org/doc/contribute.html#copyright

also uses a single space.

Make them all consistent.

Change-Id: Icc26c6b8495c3820da6b171ca96a74701b4a01b0
Reviewed-on: https://go-review.googlesource.com/20111
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-01 23:34:33 +00:00
Dmitry Vyukov
e8c4a5b893 cmd/trace: fix static file reference
Use runtime.GOROOT instead of os.Getenv("GOROOT") to reference
trace-viewer html file. GOROOT env var is not necessary set,
runtime.GOROOT has a default value for such case.

Change-Id: I906a720f6822915bd9575756e6cbf6d622857c2b
Reviewed-on: https://go-review.googlesource.com/13593
Reviewed-by: Russ Cox <rsc@golang.org>
2015-08-15 17:38:56 +00:00
Dmitry Vyukov
11f50f41ee cmd/trace: log errors to console in AJAX handler
Fixes #11508

Change-Id: I72e83893b76f75685d6edfe65ca6691d97539226
Reviewed-on: https://go-review.googlesource.com/11864
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-07-02 19:10:00 +00:00
Dmitry Vyukov
1cbbd7f545 cmd/trace: fix time scale
Integrate the latest trace-viewer changes.
It now handles nanoseconds without any issues (thanks to @egonelbre!).
So change timestamps from microseconds to nanoseconds.

Change-Id: I010f27effde7e80c9992e6f276f6912354d27df4
Reviewed-on: https://go-review.googlesource.com/11244
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Egon Elbre <egonelbre@gmail.com>
2015-07-01 10:37:10 +00:00
Dmitry Vyukov
8884fa7476 cmd/trace: sort procs
If you have more than 10 procs, then currently they are sorted alphabetically as
0, 10, 11, ..., 19, 2, 20, ...
Assign explicit order to procs so that they are sorted numerically.

Change-Id: I6d978d2cd439aa2fcbcf147842a643f9073eef75
Reviewed-on: https://go-review.googlesource.com/11750
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2015-06-30 16:50:57 +00:00