go/misc/cgo/test
doujiang24 ccad8a9f9c runtime/cgo: store M for C-created thread in pthread key
This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

Fixes #51676.
Fixes #59294.

Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494
Reviewed-on: https://go-review.googlesource.com/c/go/+/481061
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-03 18:34:11 +00:00
..
gcc68255 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue8756 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue8828 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9026 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9400 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9510a misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9510b misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue20266 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue23555a misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue23555b misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue24161arg misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue24161e0 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue24161e1 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue24161e2 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue24161res misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue26213 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue26430 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue26743 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue27054 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue27340 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue29563 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue30527 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue41761a misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue43639 misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue52611a misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue52611b misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
backdoor.go misc/cgo/test: get rid of the backdoor package 2014-12-24 00:18:21 +00:00
buildid_linux.go misc/cgo/test: avoid endless loop when we can't parse notes 2018-01-11 19:55:49 +00:00
callback.go runtime: unify asmcgocall and systemstack traceback setup 2021-02-19 00:02:06 +00:00
callback_c.c misc/cgo/test: remove timing dependency from TestParallelSleep 2022-04-21 18:43:55 +00:00
callback_c_gc.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
callback_c_gccgo.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
cgo_linux_test.go misc/cgo/test: skip Test9400 on Alpine 2022-11-07 21:15:29 +00:00
cgo_stubs_android_test.go misc/cgo/test: consolidate tests into fewer cgo source files 2019-05-22 12:52:33 +00:00
cgo_test.go runtime/cgo: store M for C-created thread in pthread key 2023-04-03 18:34:11 +00:00
cgo_thread_lock.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
cgo_unix_test.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
cthread_unix.c runtime/cgo: store M for C-created thread in pthread key 2023-04-03 18:34:11 +00:00
cthread_windows.c runtime/cgo: store M for C-created thread in pthread key 2023-04-03 18:34:11 +00:00
gcc68255.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue1435.go all: disable tests that fail on Alpine 2022-08-02 17:23:42 +00:00
issue4029.c cmd/link: support cgo internal/linking on darwin/arm64 2020-10-20 02:25:52 +00:00
issue4029.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue4029w.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue4273.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue4273b.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue4339.c misc/cgo/test: test of issue 4339 2013-09-11 09:56:38 -04:00
issue4339.h misc/cgo/test: test of issue 4339 2013-09-11 09:56:38 -04:00
issue5548_c.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue5740a.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue5740b.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue6833_c.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue6907export_c.c cmd/cgo: permit passing string values directly between Go and C 2017-11-15 03:36:54 +00:00
issue6997_linux.c misc/cgo/test,cmd/dist: enable (most) Cgo tests on Android 2016-06-01 22:06:51 +00:00
issue6997_linux.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue7234_test.go all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue8148.c misc/cgo/test: do not redeclare exported Go functions 2021-04-23 21:42:52 +00:00
issue8148.go misc/cgo/test: do not redeclare exported Go functions 2021-04-23 21:42:52 +00:00
issue8331.h all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue8517.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue8517_windows.c misc/cgo/test: add Test8517 2014-10-09 16:10:51 +11:00
issue8517_windows.go misc/cgo/test: add Test8517 2014-10-09 16:10:51 +11:00
issue8694.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue8756.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue8811.c all: make copyright headers consistent with one space after period 2016-05-02 13:43:18 +00:00
issue8828.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9026.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9400_linux.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue9510.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue18146.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue20266.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue20910.c cmd/cgo: unify cgo output for gc and gccgo 2017-08-16 21:57:56 +00:00
issue21897.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue21897b.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
issue23555.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue24161_darwin_test.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue26430.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue26743.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue27340.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue29563.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue30527.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue31891.c cmd/cgo: build unique C type cache keys from parent names 2019-10-05 00:16:04 +00:00
issue41761.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue42018.go cmd/cgo: add go:notinheap annotation to Windows handle types 2021-09-22 18:10:24 +00:00
issue42018_windows.go cmd/cgo: add go:notinheap annotation to Windows handle types 2021-09-22 18:10:24 +00:00
issue42495.go cmd/cgo: fix initialization of empty argument types 2020-11-13 15:15:15 +00:00
issue43639.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
issue52611.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
setgid2_linux.go misc/cgo/test: use fewer threads in TestSetgidStress in long mode 2022-07-27 20:21:13 +00:00
setgid_linux.go all: simplify the spelling of Linux 2021-04-13 13:56:21 +00:00
sigaltstack.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
sigprocmask.c misc/cgo/test: add retry loop around pthread_create in TestSigprocmask 2018-06-27 04:33:10 +00:00
sigprocmask.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
test.go cmd/cgo: walk {FuncType,TypeSpec}.TypeParams fields 2022-11-30 21:45:10 +00:00
test26213.go misc/cgo/test: simplify for module mode 2022-11-04 20:47:59 +00:00
test_unix.go all: gofmt more (but vendor, testdata, and top-level test directories) 2021-08-13 20:45:17 +00:00
test_windows.go misc/cgo/test: consolidate tests into fewer cgo source files 2019-05-22 12:52:33 +00:00
testx.c misc/cgo/test: do not redeclare exported Go functions 2021-04-23 21:42:52 +00:00
testx.go runtime/cgo: store M for C-created thread in pthread key 2023-04-03 18:34:11 +00:00
typeparam.go cmd/cgo: update to handle ast.IndexListExpr 2021-10-07 18:02:14 +00:00