runtime: split gp.m.locks bits for lock vs acquirem

While the primary purpose of gp.m.locks may be to count the number of
mutex values held by the M, several other parts of the runtime use it as
a general-purpose mechanism to disable preemption, independent of the
mutex implementation.

Adding a sample to the mutex contention profile includes acquiring a
mutex, and the easiest way to know that's safe to do is by waiting until
the M has released its final mutex. But when calls to lock/unlock are
interspersed with acquirem/releasem (or other changes to gp.m.locks),
it's hard to tell when the mutex count will reach zero.

This can cause TestRuntimeLockMetricsAndProfile to drop samples,
attributing more than necessary to _LostContendedRuntimeLock rather than
more specific call stacks. Consider the case where an M experiences
contention on sched.lock in startm and captures the call stack in its
local buffer. The call to unlock is nested within an acquirem/releasem
pair, which makes it look like it's not safe to acquire the mutex that
protects the contention profile buckets. So the M doesn't flush its
local buffer during the unlock call. Then releasem reduces gp.m.locks to
0, but doesn't include any code related to flushing the local contention
sample. Later, the same M experiences contention on another mutex. Since
it already has a sample in its buffer, it needs to choose which one to
keep (adding the other's value to _LostContendedRuntimeLock). This leads
to the profile missing known contention events (such as in #70602).

Change lock/unlock to use a larger delta. Leave gp.m.locks as an int32,
shifting the mutex count by 4 bits (steps of 16), for 2^27-1 before
overflow. Leave acquirem (and other ad-hoc gp.m.locks manipulation) with
a delta of 1, allowing 15 levels of nesting before interfering with
mutex profile sampling.

For #70602
For #79409

Change-Id: I7935d9e9f5b221001285fea1131c4d5ed31f0ce2
Reviewed-on: https://go-review.googlesource.com/c/go/+/779160
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: golang-scoped@luci-project-accounts.iam.gserviceaccount.com <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

src/runtime/lock_js.go

@@ -24,6 +24,8 @@ const (
 	active_spin     = 4
 	active_spin_cnt = 30
 	passive_spin    = 1
+
+	mutexMLocksDelta = 16
 )
 
 type mWaitList struct{}
@@ -48,7 +50,7 @@ func lock2(l *mutex) {
 	if gp.m.locks < 0 {
 		throw("lock count")
 	}
-	gp.m.locks++
+	gp.m.locks += mutexMLocksDelta
 	l.key = mutex_locked
 }
 
@@ -61,7 +63,7 @@ func unlock2(l *mutex) {
 		throw("unlock of unlocked lock")
 	}
 	gp := getg()
-	gp.m.locks--
+	gp.m.locks -= mutexMLocksDelta
 	if gp.m.locks < 0 {
 		throw("lock count")
 	}

src/runtime/lock_spinbit.go

@@ -68,6 +68,15 @@ const (
 	mutexPassiveSpinCount = 1
 
 	mutexTailWakePeriod = 16
+
+	// mutexMLocksDelta is the change in gp.m.locks for each lock/unlock of a
+	// mutex. The gp.m.locks field is shared with acquirem/releasem, and with
+	// other code that needs to disable preemption, which changes its value by
+	// 1. Using a different delta for mutex in particular lets us notice when
+	// the M is releasing its last mutex (so it can safely acquire another
+	// mutex, for profiling), even if the M has preemption disabled for other
+	// reasons.
+	mutexMLocksDelta = 16
 )
 
 //go:nosplit
@@ -157,7 +166,7 @@ func lock2(l *mutex) {
 	if gp.m.locks < 0 {
 		throw("runtime·lock: lock count")
 	}
-	gp.m.locks++
+	gp.m.locks += mutexMLocksDelta
 
 	k8 := key8(&l.key)
 
@@ -315,7 +324,7 @@ func unlock2(l *mutex) {
 	}
 
 	gp.m.mLockProfile.store()
-	gp.m.locks--
+	gp.m.locks -= mutexMLocksDelta
 	if gp.m.locks < 0 {
 		throw("runtime·unlock: lock count")
 	}

src/runtime/lock_wasip1.go

@@ -17,6 +17,8 @@ const (
 
 	active_spin     = 4
 	active_spin_cnt = 30
+
+	mutexMLocksDelta = 16
 )
 
 type mWaitList struct{}
@@ -41,7 +43,7 @@ func lock2(l *mutex) {
 	if gp.m.locks < 0 {
 		throw("lock count")
 	}
-	gp.m.locks++
+	gp.m.locks += mutexMLocksDelta
 	l.key = mutex_locked
 }
 
@@ -54,7 +56,7 @@ func unlock2(l *mutex) {
 		throw("unlock of unlocked lock")
 	}
 	gp := getg()
-	gp.m.locks--
+	gp.m.locks -= mutexMLocksDelta
 	if gp.m.locks < 0 {
 		throw("lock count")
 	}

src/runtime/mprof.go

@@ -745,7 +745,7 @@ func (prof *mLockProfile) captureStack() {
 //
 //go:nowritebarrierrec
 func (prof *mLockProfile) store() {
-	if gp := getg(); gp.m.locks == 1 && gp.m.mLockProfile.haveStack {
+	if gp := getg(); gp.m.locks/mutexMLocksDelta == 1 && gp.m.mLockProfile.haveStack {
 		prof.storeSlow()
 	}
 }