Stowage/go
2
0
Fork 0
mirror of https://github.com/golang/go.git synced 2025-12-08 06:10:04 +00:00
Code Issues Projects Releases Packages Wiki Activity

runtime: clean up allocation zeroing

Browse source 
Currently, the runtime zeroes allocations in several ways. First, small
object spans are always zeroed if they come from mheap, and their slots
are zeroed later in mallocgc if needed. Second, large object spans
(objects that have their own spans) plumb the need for zeroing down into
mheap. Thirdly, large objects that have no pointers have their zeroing
delayed until after preemption is reenabled, but before returning in
mallocgc.

All of this has two consequences:
1. Spans for small objects that come from mheap are sometimes
   unnecessarily zeroed, even if the mallocgc call that created them
   doesn't need the object slot to be zeroed.
2. This is all messy and difficult to reason about.

This CL simplifies this code, resolving both (1) and (2). First, it
recognizes that zeroing in mheap is unnecessary for small object spans;
mallocgc and its callees in mcache and mcentral, by design, are *always*
able to deal with non-zeroed spans. They must, for they deal with
recycled spans all the time. Once this fact is made clear, the only
remaining use of zeroing in mheap is for large objects.

As a result, this CL lifts mheap zeroing for large objects into
mallocgc, to parallel all the other codepaths in mallocgc. This is makes
the large object allocation code less surprising.

Next, this CL sets the flag for the delayed zeroing explicitly in the one
case where it matters, and inverts and renames the flag from isZeroed to
delayZeroing.

Finally, it adds a check to make sure that only pointer-free allocations
take the delayed zeroing codepath, as an extra safety measure.

Benchmark results: https://perf.golang.org/search?q=upload:20211028.8

Inspired by tapir.liu@gmail.com's CL 343470.

Change-Id: I7e1296adc19ce8a02c8d93a0a5082aefb2673e8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359477
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
This commit is contained in:
Michael Anthony Knyszek 2021-10-28 17:52:22 +00:00 • committed by Michael Knyszek
parent a88575d662
commit d8fc7f785e
4 changed files with 29 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

9
src/runtime/mcache.go
View file

@ -206,10 +206,7 @@ func (c *mcache) refill(spc spanClass) {
}
// allocLarge allocates a span for a large object.
// The boolean result indicates whether the span is known-zeroed.
// If it did not need to be zeroed, it may not have been zeroed;
// but if it came directly from the OS, it is already zeroed.
func (c *mcache) allocLarge(size uintptr, needzero bool, noscan bool) (*mspan, bool) {
func (c *mcache) allocLarge(size uintptr, noscan bool) *mspan {
if size+_PageSize < size {
throw("out of memory")
}
@ -224,7 +221,7 @@ func (c *mcache) allocLarge(size uintptr, needzero bool, noscan bool) (*mspan, b
deductSweepCredit(npages*_PageSize, npages)
spc := makeSpanClass(0, noscan)
s, isZeroed := mheap_.alloc(npages, spc, needzero)
s := mheap_.alloc(npages, spc)
if s == nil {
throw("out of memory")
}
@ -248,7 +245,7 @@ func (c *mcache) allocLarge(size uintptr, needzero bool, noscan bool) (*mspan, b
mheap_.central[spc].mcentral.fullSwept(mheap_.sweepgen).push(s)
s.limit = s.base() + size
heapBitsForAddr(s.base()).initSpan(s)
return s, isZeroed
return s
}
func (c *mcache) releaseAll() {