Stowage/go
2
0
Fork 0
mirror of https://github.com/golang/go.git synced 2025-12-08 06:10:04 +00:00
Code Issues Projects Releases Packages Wiki Activity
go/src/runtime/mpagealloc_test.go

747 lines
21 KiB
Go
Raw Normal View History

runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package runtime_test
import (
"fmt"
. "runtime"
"testing"
)
func checkPageAlloc(t *testing.T, want, got *PageAlloc) {
// Ensure start and end are correct.
wantStart, wantEnd := want.Bounds()
gotStart, gotEnd := got.Bounds()
if gotStart != wantStart {
t.Fatalf("start values not equal: got %d, want %d", gotStart, wantStart)
}
if gotEnd != wantEnd {
t.Fatalf("end values not equal: got %d, want %d", gotEnd, wantEnd)
}
for i := gotStart; i < gotEnd; i++ {
runtime: convert page allocator bitmap to sparse array Currently the page allocator bitmap is implemented as a single giant memory mapping which is reserved at init time and committed as needed. This causes problems on systems that don't handle large uncommitted mappings well, or institute low virtual address space defaults as a memory limiting mechanism. This change modifies the implementation of the page allocator bitmap away from a directly-mapped set of bytes to a sparse array in same vein as mheap.arenas. This will hurt performance a little but the biggest gains are from the lockless allocation possible with the page allocator, so the impact of this extra layer of indirection should be minimal. In fact, this is exactly what we see: https://perf.golang.org/search?q=upload:20191125.5 This reduces the amount of mapped (PROT_NONE) memory needed on systems with 48-bit address spaces to ~600 MiB down from almost 9 GiB. The bulk of this remaining memory is used by the summaries. Go processes with 32-bit address spaces now always commit to 128 KiB of memory for the bitmap. Previously it would only commit the pages in the bitmap which represented the range of addresses (lowest address to highest address, even if there are unused regions in that range) used by the heap. Updates #35568. Updates #35451. Change-Id: I0ff10380156568642b80c366001eefd0a4e6c762 Reviewed-on: https://go-review.googlesource.com/c/go/+/207497 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-14 23:58:50 +00:00
// Check the bitmaps. Note that we may have nil data.
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
gb, wb := got.PallocData(i), want.PallocData(i)
runtime: convert page allocator bitmap to sparse array Currently the page allocator bitmap is implemented as a single giant memory mapping which is reserved at init time and committed as needed. This causes problems on systems that don't handle large uncommitted mappings well, or institute low virtual address space defaults as a memory limiting mechanism. This change modifies the implementation of the page allocator bitmap away from a directly-mapped set of bytes to a sparse array in same vein as mheap.arenas. This will hurt performance a little but the biggest gains are from the lockless allocation possible with the page allocator, so the impact of this extra layer of indirection should be minimal. In fact, this is exactly what we see: https://perf.golang.org/search?q=upload:20191125.5 This reduces the amount of mapped (PROT_NONE) memory needed on systems with 48-bit address spaces to ~600 MiB down from almost 9 GiB. The bulk of this remaining memory is used by the summaries. Go processes with 32-bit address spaces now always commit to 128 KiB of memory for the bitmap. Previously it would only commit the pages in the bitmap which represented the range of addresses (lowest address to highest address, even if there are unused regions in that range) used by the heap. Updates #35568. Updates #35451. Change-Id: I0ff10380156568642b80c366001eefd0a4e6c762 Reviewed-on: https://go-review.googlesource.com/c/go/+/207497 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-14 23:58:50 +00:00
if gb == nil && wb == nil {
continue
}
if (gb == nil && wb != nil) || (gb != nil && wb == nil) {
t.Errorf("chunk %d nilness mismatch", i)
}
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
if !checkPallocBits(t, gb.PallocBits(), wb.PallocBits()) {
t.Logf("in chunk %d (mallocBits)", i)
}
if !checkPallocBits(t, gb.Scavenged(), wb.Scavenged()) {
t.Logf("in chunk %d (scavenged)", i)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
}
}
// TODO(mknyszek): Verify summaries too?
}
func TestPageAllocAlloc(t *testing.T) {
type hit struct {
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
npages, base, scav uintptr
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
}
tests := map[string]struct {
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav map[ChunkIdx][]BitRange
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
before map[ChunkIdx][]BitRange
after map[ChunkIdx][]BitRange
hits []hit
}{
"AllFree1": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 1}, {2, 2}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{1, PageBase(BaseChunkIdx, 0), PageSize},
{1, PageBase(BaseChunkIdx, 1), 0},
{1, PageBase(BaseChunkIdx, 2), PageSize},
{1, PageBase(BaseChunkIdx, 3), PageSize},
{1, PageBase(BaseChunkIdx, 4), 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 5}},
},
},
"ManyArena1": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages - 1}},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{1, PageBase(BaseChunkIdx+2, PallocChunkPages-1), PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
},
},
"NotContiguous1": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 0xff: {{0, 0}},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 0xff: {{0, PallocChunkPages}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{1, PageBase(BaseChunkIdx+0xff, 0), PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 0xff: {{0, 1}},
},
},
"AllFree2": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 3}, {7, 1}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{2, PageBase(BaseChunkIdx, 0), 2 * PageSize},
{2, PageBase(BaseChunkIdx, 2), PageSize},
{2, PageBase(BaseChunkIdx, 4), 0},
{2, PageBase(BaseChunkIdx, 6), PageSize},
{2, PageBase(BaseChunkIdx, 8), 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 10}},
},
},
"Straddle2": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages - 1}},
BaseChunkIdx + 1: {{1, PallocChunkPages - 1}},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{PallocChunkPages - 1, 1}},
BaseChunkIdx + 1: {},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{2, PageBase(BaseChunkIdx, PallocChunkPages-1), PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
},
},
"AllFree5": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 8}, {9, 1}, {17, 5}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{5, PageBase(BaseChunkIdx, 0), 5 * PageSize},
{5, PageBase(BaseChunkIdx, 5), 4 * PageSize},
{5, PageBase(BaseChunkIdx, 10), 0},
{5, PageBase(BaseChunkIdx, 15), 3 * PageSize},
{5, PageBase(BaseChunkIdx, 20), 2 * PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 25}},
},
},
"AllFree64": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{21, 1}, {63, 65}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{64, PageBase(BaseChunkIdx, 0), 2 * PageSize},
{64, PageBase(BaseChunkIdx, 64), 64 * PageSize},
{64, PageBase(BaseChunkIdx, 128), 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 192}},
},
},
"AllFree65": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{129, 1}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{65, PageBase(BaseChunkIdx, 0), 0},
{65, PageBase(BaseChunkIdx, 65), PageSize},
{65, PageBase(BaseChunkIdx, 130), 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 195}},
},
},
// TODO(mknyszek): Add tests close to the chunk size.
"ExhaustPallocChunkPages-3": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{10, 1}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages - 3, PageBase(BaseChunkIdx, 0), PageSize},
{PallocChunkPages - 3, 0, 0},
{1, PageBase(BaseChunkIdx, PallocChunkPages-3), 0},
{2, PageBase(BaseChunkIdx, PallocChunkPages-2), 0},
{1, 0, 0},
{PallocChunkPages - 3, 0, 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
},
"AllFreePallocChunkPages": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 1}, {PallocChunkPages - 1, 1}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages, PageBase(BaseChunkIdx, 0), 2 * PageSize},
{PallocChunkPages, 0, 0},
{1, 0, 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
},
"StraddlePallocChunkPages": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages / 2}},
BaseChunkIdx + 1: {{PallocChunkPages / 2, PallocChunkPages / 2}},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {{3, 100}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages, PageBase(BaseChunkIdx, PallocChunkPages/2), 100 * PageSize},
{PallocChunkPages, 0, 0},
{1, 0, 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
},
},
"StraddlePallocChunkPages+1": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages / 2}},
BaseChunkIdx + 1: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages + 1, PageBase(BaseChunkIdx, PallocChunkPages/2), (PallocChunkPages + 1) * PageSize},
{PallocChunkPages, 0, 0},
{1, PageBase(BaseChunkIdx+1, PallocChunkPages/2+1), PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages/2 + 2}},
},
},
"AllFreePallocChunkPages*2": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages * 2, PageBase(BaseChunkIdx, 0), 0},
{PallocChunkPages * 2, 0, 0},
{1, 0, 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
},
},
"NotContiguousPallocChunkPages*2": {
before: map[ChunkIdx][]BitRange{
runtime: make the test addresses for pageAlloc smaller on 32-bit This change makes the test addresses start at 1 GiB instead of 2 GiB to support mips and mipsle, which only have 31-bit address spaces. It also changes some tests to use smaller offsets for the chunk index to avoid jumping too far ahead in the address space to support 31-bit address spaces. The tests don't require such large jumps for what they're testing anyway. Updates #35112. Fixes #35440. Change-Id: Ic68ff2b0a1f10ef37ac00d4bb5b910ddcdc76f2e Reviewed-on: https://go-review.googlesource.com/c/go/+/205938 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-07 22:42:38 +00:00
BaseChunkIdx: {},
BaseChunkIdx + 0x40: {},
BaseChunkIdx + 0x41: {},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
runtime: make the test addresses for pageAlloc smaller on 32-bit This change makes the test addresses start at 1 GiB instead of 2 GiB to support mips and mipsle, which only have 31-bit address spaces. It also changes some tests to use smaller offsets for the chunk index to avoid jumping too far ahead in the address space to support 31-bit address spaces. The tests don't require such large jumps for what they're testing anyway. Updates #35112. Fixes #35440. Change-Id: Ic68ff2b0a1f10ef37ac00d4bb5b910ddcdc76f2e Reviewed-on: https://go-review.googlesource.com/c/go/+/205938 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-07 22:42:38 +00:00
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 0x40: {},
BaseChunkIdx + 0x41: {},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: make the test addresses for pageAlloc smaller on 32-bit This change makes the test addresses start at 1 GiB instead of 2 GiB to support mips and mipsle, which only have 31-bit address spaces. It also changes some tests to use smaller offsets for the chunk index to avoid jumping too far ahead in the address space to support 31-bit address spaces. The tests don't require such large jumps for what they're testing anyway. Updates #35112. Fixes #35440. Change-Id: Ic68ff2b0a1f10ef37ac00d4bb5b910ddcdc76f2e Reviewed-on: https://go-review.googlesource.com/c/go/+/205938 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-07 22:42:38 +00:00
{PallocChunkPages * 2, PageBase(BaseChunkIdx+0x40, 0), 0},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{21, PageBase(BaseChunkIdx, 0), 21 * PageSize},
{1, PageBase(BaseChunkIdx, 21), PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
runtime: make the test addresses for pageAlloc smaller on 32-bit This change makes the test addresses start at 1 GiB instead of 2 GiB to support mips and mipsle, which only have 31-bit address spaces. It also changes some tests to use smaller offsets for the chunk index to avoid jumping too far ahead in the address space to support 31-bit address spaces. The tests don't require such large jumps for what they're testing anyway. Updates #35112. Fixes #35440. Change-Id: Ic68ff2b0a1f10ef37ac00d4bb5b910ddcdc76f2e Reviewed-on: https://go-review.googlesource.com/c/go/+/205938 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-07 22:42:38 +00:00
BaseChunkIdx: {{0, 22}},
BaseChunkIdx + 0x40: {{0, PallocChunkPages}},
BaseChunkIdx + 0x41: {{0, PallocChunkPages}},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
},
"StraddlePallocChunkPages*2": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages / 2}},
BaseChunkIdx + 1: {},
BaseChunkIdx + 2: {{PallocChunkPages / 2, PallocChunkPages / 2}},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, 7}},
BaseChunkIdx + 1: {{3, 5}, {121, 10}},
BaseChunkIdx + 2: {{PallocChunkPages/2 + 12, 2}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages * 2, PageBase(BaseChunkIdx, PallocChunkPages/2), 15 * PageSize},
{PallocChunkPages * 2, 0, 0},
{1, 0, 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
},
},
"StraddlePallocChunkPages*5/4": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages * 3 / 4}},
BaseChunkIdx + 2: {{0, PallocChunkPages * 3 / 4}},
BaseChunkIdx + 3: {{0, 0}},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{PallocChunkPages / 2, PallocChunkPages/4 + 1}},
BaseChunkIdx + 2: {{PallocChunkPages / 3, 1}},
BaseChunkIdx + 3: {{PallocChunkPages * 2 / 3, 1}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages * 5 / 4, PageBase(BaseChunkIdx+2, PallocChunkPages*3/4), PageSize},
{PallocChunkPages * 5 / 4, 0, 0},
{1, PageBase(BaseChunkIdx+1, PallocChunkPages*3/4), PageSize},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages*3/4 + 1}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
BaseChunkIdx + 3: {{0, PallocChunkPages}},
},
},
"AllFreePallocChunkPages*7+5": {
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
BaseChunkIdx + 2: {},
BaseChunkIdx + 3: {},
BaseChunkIdx + 4: {},
BaseChunkIdx + 5: {},
BaseChunkIdx + 6: {},
BaseChunkIdx + 7: {},
},
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
scav: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{50, 1}},
BaseChunkIdx + 1: {{31, 1}},
BaseChunkIdx + 2: {{7, 1}},
BaseChunkIdx + 3: {{200, 1}},
BaseChunkIdx + 4: {{3, 1}},
BaseChunkIdx + 5: {{51, 1}},
BaseChunkIdx + 6: {{20, 1}},
BaseChunkIdx + 7: {{1, 1}},
},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
hits: []hit{
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
{PallocChunkPages*7 + 5, PageBase(BaseChunkIdx, 0), 8 * PageSize},
{PallocChunkPages*7 + 5, 0, 0},
{1, PageBase(BaseChunkIdx+7, 5), 0},
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
BaseChunkIdx + 3: {{0, PallocChunkPages}},
BaseChunkIdx + 4: {{0, PallocChunkPages}},
BaseChunkIdx + 5: {{0, PallocChunkPages}},
BaseChunkIdx + 6: {{0, PallocChunkPages}},
BaseChunkIdx + 7: {{0, 6}},
},
},
}
for name, v := range tests {
v := v
t.Run(name, func(t *testing.T) {
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
b := NewPageAlloc(v.before, v.scav)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(b)
for iter, i := range v.hits {
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
a, s := b.Alloc(i.npages)
if a != i.base {
t.Fatalf("bad alloc #%d: want base 0x%x, got 0x%x", iter+1, i.base, a)
}
if s != i.scav {
t.Fatalf("bad alloc #%d: want scav %d, got %d", iter+1, i.scav, s)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
}
}
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
want := NewPageAlloc(v.after, v.scav)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(want)
checkPageAlloc(t, want, b)
})
}
}
func TestPageAllocExhaust(t *testing.T) {
for _, npages := range []uintptr{1, 2, 3, 4, 5, 8, 16, 64, 1024, 1025, 2048, 2049} {
npages := npages
t.Run(fmt.Sprintf("%d", npages), func(t *testing.T) {
// Construct b.
bDesc := make(map[ChunkIdx][]BitRange)
for i := ChunkIdx(0); i < 4; i++ {
bDesc[BaseChunkIdx+i] = []BitRange{}
}
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
b := NewPageAlloc(bDesc, nil)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(b)
// Allocate into b with npages until we've exhausted the heap.
nAlloc := (PallocChunkPages * 4) / int(npages)
for i := 0; i < nAlloc; i++ {
addr := PageBase(BaseChunkIdx, uint(i)*uint(npages))
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
if a, _ := b.Alloc(npages); a != addr {
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
t.Fatalf("bad alloc #%d: want 0x%x, got 0x%x", i+1, addr, a)
}
}
// Check to make sure the next allocation fails.
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
if a, _ := b.Alloc(npages); a != 0 {
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
t.Fatalf("bad alloc #%d: want 0, got 0x%x", nAlloc, a)
}
// Construct what we want the heap to look like now.
allocPages := nAlloc * int(npages)
wantDesc := make(map[ChunkIdx][]BitRange)
for i := ChunkIdx(0); i < 4; i++ {
if allocPages >= PallocChunkPages {
wantDesc[BaseChunkIdx+i] = []BitRange{{0, PallocChunkPages}}
allocPages -= PallocChunkPages
} else if allocPages > 0 {
wantDesc[BaseChunkIdx+i] = []BitRange{{0, uint(allocPages)}}
allocPages = 0
} else {
wantDesc[BaseChunkIdx+i] = []BitRange{}
}
}
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
want := NewPageAlloc(wantDesc, nil)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(want)
// Check to make sure the heap b matches what we want.
checkPageAlloc(t, want, b)
})
}
}
func TestPageAllocFree(t *testing.T) {
tests := map[string]struct {
before map[ChunkIdx][]BitRange
after map[ChunkIdx][]BitRange
npages uintptr
frees []uintptr
}{
"Free1": {
npages: 1,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
PageBase(BaseChunkIdx, 1),
PageBase(BaseChunkIdx, 2),
PageBase(BaseChunkIdx, 3),
PageBase(BaseChunkIdx, 4),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{5, PallocChunkPages - 5}},
},
},
"ManyArena1": {
npages: 1,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, PallocChunkPages/2),
PageBase(BaseChunkIdx+1, 0),
PageBase(BaseChunkIdx+2, PallocChunkPages-1),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages / 2}, {PallocChunkPages/2 + 1, PallocChunkPages/2 - 1}},
BaseChunkIdx + 1: {{1, PallocChunkPages - 1}},
BaseChunkIdx + 2: {{0, PallocChunkPages - 1}},
},
},
"Free2": {
npages: 2,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
PageBase(BaseChunkIdx, 2),
PageBase(BaseChunkIdx, 4),
PageBase(BaseChunkIdx, 6),
PageBase(BaseChunkIdx, 8),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{10, PallocChunkPages - 10}},
},
},
"Straddle2": {
npages: 2,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{PallocChunkPages - 1, 1}},
BaseChunkIdx + 1: {{0, 1}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, PallocChunkPages-1),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
},
},
"Free5": {
npages: 5,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
PageBase(BaseChunkIdx, 5),
PageBase(BaseChunkIdx, 10),
PageBase(BaseChunkIdx, 15),
PageBase(BaseChunkIdx, 20),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{25, PallocChunkPages - 25}},
},
},
"Free64": {
npages: 64,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
PageBase(BaseChunkIdx, 64),
PageBase(BaseChunkIdx, 128),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{192, PallocChunkPages - 192}},
},
},
"Free65": {
npages: 65,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
PageBase(BaseChunkIdx, 65),
PageBase(BaseChunkIdx, 130),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{195, PallocChunkPages - 195}},
},
},
"FreePallocChunkPages": {
npages: PallocChunkPages,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
},
},
"StraddlePallocChunkPages": {
npages: PallocChunkPages,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{PallocChunkPages / 2, PallocChunkPages / 2}},
BaseChunkIdx + 1: {{0, PallocChunkPages / 2}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, PallocChunkPages/2),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
},
},
"StraddlePallocChunkPages+1": {
npages: PallocChunkPages + 1,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, PallocChunkPages/2),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages / 2}},
BaseChunkIdx + 1: {{PallocChunkPages/2 + 1, PallocChunkPages/2 - 1}},
},
},
"FreePallocChunkPages*2": {
npages: PallocChunkPages * 2,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
},
},
"StraddlePallocChunkPages*2": {
npages: PallocChunkPages * 2,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, PallocChunkPages/2),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages / 2}},
BaseChunkIdx + 1: {},
BaseChunkIdx + 2: {{PallocChunkPages / 2, PallocChunkPages / 2}},
},
},
"AllFreePallocChunkPages*7+5": {
npages: PallocChunkPages*7 + 5,
before: map[ChunkIdx][]BitRange{
BaseChunkIdx: {{0, PallocChunkPages}},
BaseChunkIdx + 1: {{0, PallocChunkPages}},
BaseChunkIdx + 2: {{0, PallocChunkPages}},
BaseChunkIdx + 3: {{0, PallocChunkPages}},
BaseChunkIdx + 4: {{0, PallocChunkPages}},
BaseChunkIdx + 5: {{0, PallocChunkPages}},
BaseChunkIdx + 6: {{0, PallocChunkPages}},
BaseChunkIdx + 7: {{0, PallocChunkPages}},
},
frees: []uintptr{
PageBase(BaseChunkIdx, 0),
},
after: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
BaseChunkIdx + 2: {},
BaseChunkIdx + 3: {},
BaseChunkIdx + 4: {},
BaseChunkIdx + 5: {},
BaseChunkIdx + 6: {},
BaseChunkIdx + 7: {{5, PallocChunkPages - 5}},
},
},
}
for name, v := range tests {
v := v
t.Run(name, func(t *testing.T) {
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
b := NewPageAlloc(v.before, nil)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(b)
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
for _, addr := range v.frees {
b.Free(addr, v.npages)
}
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
want := NewPageAlloc(v.after, nil)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(want)
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
checkPageAlloc(t, want, b)
})
}
}
func TestPageAllocAllocAndFree(t *testing.T) {
type hit struct {
alloc bool
npages uintptr
base uintptr
}
tests := map[string]struct {
init map[ChunkIdx][]BitRange
hits []hit
}{
// TODO(mknyszek): Write more tests here.
"Chunks8": {
init: map[ChunkIdx][]BitRange{
BaseChunkIdx: {},
BaseChunkIdx + 1: {},
BaseChunkIdx + 2: {},
BaseChunkIdx + 3: {},
BaseChunkIdx + 4: {},
BaseChunkIdx + 5: {},
BaseChunkIdx + 6: {},
BaseChunkIdx + 7: {},
},
hits: []hit{
{true, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
{false, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
{true, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
{false, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
{true, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
{false, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
{true, 1, PageBase(BaseChunkIdx, 0)},
{false, 1, PageBase(BaseChunkIdx, 0)},
{true, PallocChunkPages * 8, PageBase(BaseChunkIdx, 0)},
},
},
}
for name, v := range tests {
v := v
t.Run(name, func(t *testing.T) {
runtime: add scavenging code for new page allocator This change adds a scavenger for the new page allocator along with tests. The scavenger walks over the heap backwards once per GC, looking for memory to scavenge. It walks across the heap without any lock held, searching optimistically. If it finds what appears to be a scavenging candidate it acquires the heap lock and attempts to verify it. Upon verification it then scavenges. Notably, unlike the old scavenger, it doesn't show any preference for huge pages and instead follows a more strict last-page-first policy. Updates #35112. Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/195697 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-21 00:24:25 +00:00
b := NewPageAlloc(v.init, nil)
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
defer FreePageAlloc(b)
for iter, i := range v.hits {
if i.alloc {
runtime: count scavenged bits for new allocation for new page allocator This change makes it so that the new page allocator returns the number of pages that are scavenged in a new allocation so that mheap can update memstats appropriately. The accounting could be embedded into pageAlloc, but that would make the new allocator more difficult to test. Updates #35112. Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12 Reviewed-on: https://go-review.googlesource.com/c/go/+/195698 Reviewed-by: Austin Clements <austin@google.com>
2019-09-10 18:53:51 +00:00
if a, _ := b.Alloc(i.npages); a != i.base {
runtime: add new page allocator core This change adds a new bitmap-based allocator to the runtime with tests. It does not yet integrate the page allocator into the runtime and thus this change is almost purely additive. Updates #35112. Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf Reviewed-on: https://go-review.googlesource.com/c/go/+/190622 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-08-14 16:32:12 +00:00
t.Fatalf("bad alloc #%d: want 0x%x, got 0x%x", iter+1, i.base, a)
}
} else {
b.Free(i.base, i.npages)
}
}
})
}
}
Reference in a new issue Copy permalink