Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-10-19 15:43:20 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Userland/Libraries/LibJS/Heap/Heap.cpp

384 lines
12 KiB
C++
Raw Normal View History

LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
/*
LibJS: Add a 96-byte CellAllocator Two of our most frequently allocated objects are Shape (88 bytes) and DeclarativeEnvironment (80 bytes). Putting these into 128-byte cells was quite wasteful, so let's add a more suitable allocator for them.
2022-01-31 16:16:59 +01:00
* Copyright (c) 2020-2022, Andreas Kling <kling@serenityos.org>
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 01:24:48 -07:00
* SPDX-License-Identifier: BSD-2-Clause
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
*/
LibJS: Make sure we mark everything reachable from the scope stack This ensures that local variables survive GC.
2020-03-09 21:29:22 +01:00
#include <AK/Badge.h>
Everywhere: Hook up remaining debug macros to Debug.h.
2021-01-24 15:28:26 +01:00
#include <AK/Debug.h>
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
#include <AK/HashTable.h>
LibJS+AK: Move cross-platform stack bounds code from JS::Heap to AK::StackInfo This will be useful for other things than the Heap, maybe even outside of LibJS.
2020-11-08 12:48:16 +00:00
#include <AK/StackInfo.h>
Everywhere: Add missing <AK/TemporaryChange.h> includes Don't rely on HashTable.h pulling this in.
2020-10-15 20:46:52 +02:00
#include <AK/TemporaryChange.h>
LibJS: Add API for doing GC with a little debug log report at end You can now pass print_report=true to Heap::collect_garbage() and it will print out a little summary of the time spent, and counts of live vs freed cells and blocks.
2020-08-16 20:33:56 +02:00
#include <LibCore/ElapsedTimer.h>
LibJS: Rename Allocator => CellAllocator Now that we have a BlockAllocator as well, it seems appropriate to name the allocator-that-allocates-cells something more specific to match.
2021-05-27 19:03:41 +02:00
#include <LibJS/Heap/CellAllocator.h>
LibJS: Add Handle<T>, a strong C++ handle for keeping GC objects alive This is pretty heavy and unoptimized, but it will do the trick for now. Basically, Heap now has a HashTable<HandleImpl*> and you can call JS::make_handle(T*) to construct a Handle<T> that guarantees that the pointee will always survive GC until the Handle<T> is destroyed.
2020-03-18 20:03:17 +01:00
#include <LibJS/Heap/Handle.h>
LibJS: Add "Heap" and "Runtime" subdirectories Let's try to keep LibJS tidy as it expands. :^)
2020-03-16 14:20:30 +01:00
#include <LibJS/Heap/Heap.h>
#include <LibJS/Heap/HeapBlock.h>
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
#include <LibJS/Interpreter.h>
LibJS: Add "Heap" and "Runtime" subdirectories Let's try to keep LibJS tidy as it expands. :^)
2020-03-16 14:20:30 +01:00
#include <LibJS/Runtime/Object.h>
LibJS: Generify the garbage collector's weak container notifications This will allow us to use the same interface for other JS weak containers like the WeakMap & WeakRef.
2021-06-12 05:23:33 +03:00
#include <LibJS/Runtime/WeakContainer.h>
LibJS: Add JS::SafeFunction, like Function but protects captures from GC SafeFunction automatically registers its closure memory area in a place where the JS garbage collector can find it. This means that you can capture JS::Value and arbitrary pointers into the GC heap in closures, as long as you're using a SafeFunction, and the GC will not zap those values! There's probably some performance impact from this, and there's a lot of things that could be nicer/smarter about it, but let's build something that ensures safety first, and we can worry about performance later. :^)
2022-09-24 11:56:43 +02:00
#include <LibJS/SafeFunction.h>
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
#include <setjmp.h>
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
Everywhere: Replace uses of __serenity__ with AK_OS_SERENITY Now that we have OS macros for essentially every supported OS, let's try to use them everywhere.
2022-10-09 15:23:23 -06:00
#ifdef AK_OS_SERENITY
LibJS: Emit a profile signpost when starting a garbage collection
2021-08-11 20:31:11 +02:00
# include <serenity.h>
#endif
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
namespace JS {
Everywhere: Replace uses of __serenity__ with AK_OS_SERENITY Now that we have OS macros for essentially every supported OS, let's try to use them everywhere.
2022-10-09 15:23:23 -06:00
#ifdef AK_OS_SERENITY
Kernel: Make sys$perf_register_string() generate the string ID's Making userspace provide a global string ID was silly, and made the API extremely difficult to use correctly in a global profiling context. Instead, simply make the kernel do the string ID allocation for us. This also allows us to convert the string storage to a Vector in the kernel (and an array in the JSON profile data.)
2021-08-11 20:41:29 +02:00
static int gc_perf_string_id;
LibJS: Emit a profile signpost when starting a garbage collection
2021-08-11 20:31:11 +02:00
#endif
LibJS: Add JS::SafeFunction, like Function but protects captures from GC SafeFunction automatically registers its closure memory area in a place where the JS garbage collector can find it. This means that you can capture JS::Value and arbitrary pointers into the GC heap in closures, as long as you're using a SafeFunction, and the GC will not zap those values! There's probably some performance impact from this, and there's a lot of things that could be nicer/smarter about it, but let's build something that ensures safety first, and we can worry about performance later. :^)
2022-09-24 11:56:43 +02:00
// NOTE: We keep a per-thread list of custom ranges. This hinges on the assumption that there is one JS VM per thread.
static __thread HashMap<FlatPtr*, size_t>* s_custom_ranges_for_conservative_scan = nullptr;
LibJS+Clients: Add JS::VM object, separate Heap from Interpreter Taking a big step towards a world of multiple global object, this patch adds a new JS::VM object that houses the JS::Heap. This means that the Heap moves out of Interpreter, and the same Heap can now be used by multiple Interpreters, and can also outlive them. The VM keeps a stack of Interpreter pointers. We push/pop on this stack when entering/exiting execution with a given Interpreter. This allows us to make this change without disturbing too much of the existing code. There is still a 1-to-1 relationship between Interpreter and the global object. This will change in the future. Ultimately, the goal here is to make Interpreter a transient object that only needs to exist while you execute some code. Getting there will take a lot more work though. :^) Note that in LibWeb, the global JS::VM is called main_thread_vm(), to distinguish it from future worker VM's.
2020-09-20 19:24:44 +02:00
Heap::Heap(VM& vm)
: m_vm(vm)
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
{
Everywhere: Replace uses of __serenity__ with AK_OS_SERENITY Now that we have OS macros for essentially every supported OS, let's try to use them everywhere.
2022-10-09 15:23:23 -06:00
#ifdef AK_OS_SERENITY
LibJS: Emit a profile signpost when starting a garbage collection
2021-08-11 20:31:11 +02:00
auto gc_signpost_string = "Garbage collection"sv;
Kernel: Make sys$perf_register_string() generate the string ID's Making userspace provide a global string ID was silly, and made the API extremely difficult to use correctly in a global profiling context. Instead, simply make the kernel do the string ID allocation for us. This also allows us to convert the string storage to a Vector in the kernel (and an array in the JSON profile data.)
2021-08-11 20:41:29 +02:00
gc_perf_string_id = perf_register_string(gc_signpost_string.characters_without_null_termination(), gc_signpost_string.length());
LibJS: Emit a profile signpost when starting a garbage collection
2021-08-11 20:31:11 +02:00
#endif
LibJS: Expose minimum possible cell size of JS::Heap Use this to avoid creating a 16 byte cell allocator on x86_64, where the size of FreelistEntry is 24 bytes. Every JS::Cell must be at least the size of the FreelistEntry or things start crashing, so the 16 byte allocator was wasted on that platform.
2021-05-29 06:24:30 -06:00
if constexpr (HeapBlock::min_possible_cell_size <= 16) {
m_allocators.append(make<CellAllocator>(16));
}
static_assert(HeapBlock::min_possible_cell_size <= 24, "Heap Cell tracking uses too much data!");
LibJS: Rename Allocator => CellAllocator Now that we have a BlockAllocator as well, it seems appropriate to name the allocator-that-allocates-cells something more specific to match.
2021-05-27 19:03:41 +02:00
m_allocators.append(make<CellAllocator>(32));
m_allocators.append(make<CellAllocator>(64));
LibJS: Add a 96-byte CellAllocator Two of our most frequently allocated objects are Shape (88 bytes) and DeclarativeEnvironment (80 bytes). Putting these into 128-byte cells was quite wasteful, so let's add a more suitable allocator for them.
2022-01-31 16:16:59 +01:00
m_allocators.append(make<CellAllocator>(96));
LibJS: Rename Allocator => CellAllocator Now that we have a BlockAllocator as well, it seems appropriate to name the allocator-that-allocates-cells something more specific to match.
2021-05-27 19:03:41 +02:00
m_allocators.append(make<CellAllocator>(128));
m_allocators.append(make<CellAllocator>(256));
m_allocators.append(make<CellAllocator>(512));
m_allocators.append(make<CellAllocator>(1024));
m_allocators.append(make<CellAllocator>(3072));
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
Heap::~Heap()
{
LibJS: Keep track of PrimitiveStrings and share them VM now has a string cache which tracks all live PrimitiveStrings and reuses an existing one if possible. This drastically reduces the number of GC-allocated strings in many real-word situations.
2021-10-02 01:36:57 +02:00
vm().string_cache().clear();
LibJS: Always collect all garbage when destroying Heap When the Heap is going down, it's our last chance to run destructors, so add a separate collector mode where we simply skip over the marking phase and go directly to sweeping. This causes everything to get swept and all live cells get destroyed. This way, valgrind reports 0 leaks on exit. :^)
2020-03-23 14:11:19 +01:00
collect_garbage(CollectionType::CollectEverything);
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
LibJS: Rename Allocator => CellAllocator Now that we have a BlockAllocator as well, it seems appropriate to name the allocator-that-allocates-cells something more specific to match.
2021-05-27 19:03:41 +02:00
ALWAYS_INLINE CellAllocator& Heap::allocator_for_size(size_t cell_size)
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
{
for (auto& allocator : m_allocators) {
if (allocator->cell_size() >= cell_size)
return *allocator;
}
LibJS: Add dbgln() to Heap::allocator_for_size() before crashing If we can't get a CellAllocator for the requested cell size, at least print a debug message before dying.
2021-06-06 23:08:15 +01:00
dbgln("Cannot get CellAllocator for cell size {}, largest available is {}!", cell_size, m_allocators.last()->cell_size());
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 20:42:32 +01:00
VERIFY_NOT_REACHED();
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
}
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
Cell* Heap::allocate_cell(size_t size)
{
LibJS: Do a garbage collection every N allocations (N=10'000) To prevent the heap from growing infinitely large, we now do a full GC every 10'000 allocations. :^)
2020-04-06 12:36:49 +02:00
if (should_collect_on_every_allocation()) {
LibJS+js: Add a debug option (js -g) to GC after every allocation This is very useful for discovering collector bugs.
2020-03-16 19:18:46 +01:00
collect_garbage();
LibJS: Do a garbage collection every N allocations (N=10'000) To prevent the heap from growing infinitely large, we now do a full GC every 10'000 allocations. :^)
2020-04-06 12:36:49 +02:00
} else if (m_allocations_since_last_gc > m_max_allocations_between_gc) {
m_allocations_since_last_gc = 0;
collect_garbage();
} else {
++m_allocations_since_last_gc;
}
LibJS+js: Add a debug option (js -g) to GC after every allocation This is very useful for discovering collector bugs.
2020-03-16 19:18:46 +01:00
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
auto& allocator = allocator_for_size(size);
return allocator.allocate_cell(*this);
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
LibJS: Add API for doing GC with a little debug log report at end You can now pass print_report=true to Heap::collect_garbage() and it will print out a little summary of the time spent, and counts of live vs freed cells and blocks.
2020-08-16 20:33:56 +02:00
void Heap::collect_garbage(CollectionType collection_type, bool print_report)
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
{
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 20:42:32 +01:00
VERIFY(!m_collecting_garbage);
LibJS: Assert if garbage collection is restarted while ongoing We can't GC while we're already in GC. Assert if this happens.
2020-09-21 14:35:19 +02:00
TemporaryChange change(m_collecting_garbage, true);
Everywhere: Replace uses of __serenity__ with AK_OS_SERENITY Now that we have OS macros for essentially every supported OS, let's try to use them everywhere.
2022-10-09 15:23:23 -06:00
#ifdef AK_OS_SERENITY
LibJS: Emit a profile signpost when starting a garbage collection
2021-08-11 20:31:11 +02:00
static size_t global_gc_counter = 0;
perf_event(PERF_EVENT_SIGNPOST, gc_perf_string_id, global_gc_counter++);
#endif
LibJS: Use ElapsedTimer::start_new();
2021-09-12 08:51:44 -07:00
auto collection_measurement_timer = Core::ElapsedTimer::start_new();
LibJS: Always collect all garbage when destroying Heap When the Heap is going down, it's our last chance to run destructors, so add a separate collector mode where we simply skip over the marking phase and go directly to sweeping. This causes everything to get swept and all live cells get destroyed. This way, valgrind reports 0 leaks on exit. :^)
2020-03-23 14:11:19 +01:00
if (collection_type == CollectionType::CollectGarbage) {
LibJS: Add DeferGC, a RAII way to prevent GC temporarily
2020-04-19 11:30:47 +02:00
if (m_gc_deferrals) {
m_should_gc_when_deferral_ends = true;
return;
}
LibJS: Always collect all garbage when destroying Heap When the Heap is going down, it's our last chance to run destructors, so add a separate collector mode where we simply skip over the marking phase and go directly to sweeping. This causes everything to get swept and all live cells get destroyed. This way, valgrind reports 0 leaks on exit. :^)
2020-03-23 14:11:19 +01:00
HashTable<Cell*> roots;
gather_roots(roots);
mark_live_cells(roots);
}
LibJS: Add API for doing GC with a little debug log report at end You can now pass print_report=true to Heap::collect_garbage() and it will print out a little summary of the time spent, and counts of live vs freed cells and blocks.
2020-08-16 20:33:56 +02:00
sweep_dead_cells(print_report, collection_measurement_timer);
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
LibJS: Rename collect_roots() => gather_roots() Since this is about finding the things we should *not* garabge collect, it seemed wrong to call it "collect_something" :^)
2020-03-15 15:12:34 +01:00
void Heap::gather_roots(HashTable<Cell*>& roots)
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
{
LibJS: GC should gather roots from all active interpreters If we are in a nested execution context, we shouldn't only mark things used by the active interpreter.
2020-09-21 13:47:33 +02:00
vm().gather_roots(roots);
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
gather_conservative_roots(roots);
LibJS: Use IntrusiveList for keeping track of HandleImpls This allows us to remove a HashTable from heap and cuts down on some of the malloc traffic when creating handles.
2021-07-21 19:45:21 +02:00
for (auto& handle : m_handles)
roots.set(handle.cell());
LibJS: Add Handle<T>, a strong C++ handle for keeping GC objects alive This is pretty heavy and unoptimized, but it will do the trick for now. Basically, Heap now has a HashTable<HandleImpl*> and you can call JS::make_handle(T*) to construct a Handle<T> that guarantees that the pointee will always survive GC until the Handle<T> is destroyed.
2020-03-18 20:03:17 +01:00
LibJS: Let MarkedVector<T> inherit from Vector and handle Cell* + Value Note: MarkedVector is still relatively new and has zero users right now, so these changes don't affect any code other than the class itself. Reasons for this are the rather limited API: - Despite the name and unlike MarkedValueList, MarkedVector isn't actually a Vector, it *wraps* a Vector. This means that plenty of convenient APIs are unavailable and have to be exported on the class separately and forwarded to the internal Vector, or need to go through the exposed Span - both not great options. - Exposing append(Cell*) and prepend(Cell*) on the base class means that it was possible to append any Cell type, not just T! All the strong typing guarantees are basically gone, and MarkedVector doesn't do much more than casting Cells to the appropriate type through the exposed Span. All of this combined means that MarkedVector - in its current form - doesn't provide much value over MarkedValueList, and that we have to maintain two separate, yet almost identical classes. Let's fix this! The updated MarkedVector steals various concepts from the existing MarkedValueList, especially the ability to copy. On the other hand, it remains generic enough to handle both Cell* and Value for T, making MarkedValueList effectively redundant :^) Additionally, by inheriting from Vector we get all the current and future APIs without having to select and expose them separately. MarkedVectorBase remains and takes care of communicating creation and destruction of the class to the heap. Visiting the contained values is handled via a pure virtual method gather_roots(), which is being called by the Heap's function of the same name; much like the VM has one. From there, values are added to the roots HashTable if they are cells for T = Value, and unconditionally for any other T. As a small additional improvement the template now also takes an inline_capacity parameter, defaulting to 32, and forwards it to the Vector template; allowing for possible future optimizations of current uses of MarkedValueList, which hard-codes it to 32.
2022-02-09 09:40:49 +00:00
for (auto& vector : m_marked_vectors)
vector.gather_roots(roots);
LibJS: Add MarkedVector<T> This abstracts a vector of Cell* with a strongly typed span() accessor that gives you Span<T*> instead of Span<Cell*>. It is intended to replace MarkedValueList in situations where you only need to store pointers to Cell (or an even more specific type of Cell). The API can definitely be improved, it's just the bare basics for now.
2021-12-16 18:54:06 +01:00
LibJS: Use 'if constexpr' instead of '#if HEAP_DEBUG'
2021-04-18 18:12:33 +02:00
if constexpr (HEAP_DEBUG) {
dbgln("gather_roots:");
for (auto* root : roots)
dbgln(" + {}", root);
}
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
LibJS: Disable ASAN during the conservative GC stack scan This allows the JS fuzzer to survive garbage collection (so we can find more interesting bugs!) Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=29266
2021-01-05 12:26:23 +01:00
__attribute__((no_sanitize("address"))) void Heap::gather_conservative_roots(HashTable<Cell*>& roots)
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
{
FlatPtr dummy;
LibJS: Use dbgln_if in Heap.cpp
2021-04-07 15:12:32 +02:00
dbgln_if(HEAP_DEBUG, "gather_conservative_roots:");
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
jmp_buf buf;
setjmp(buf);
HashTable<FlatPtr> possible_pointers;
LibJS: Fix clang-tidy warnings about redundant types in Heap.cpp
2021-05-25 19:03:30 +02:00
auto* raw_jmp_buf = reinterpret_cast<FlatPtr const*>(buf);
LibJS: Port garbage collector to Linux Well that was easy. LibJS can now run on Linux :^)
2020-03-23 13:14:57 +01:00
LibJS: Use NaN boxing to decrease the memory size of Values Using the fact that there are 2^52-2 NaN representations we can "NaN-box" all the Values possible. This means that Value no longer has an explicit "Type" but that information is now stored in the bits of a double. This is done by "tagging" the top two bytes of the double. For a full explanation see the large comment with asserts at the top of Value. We can also use the exact representation of the tags to make checking properties like nullish, or is_cell quicker. But the largest gains are in the fact that the size of a Value is now halved. The SunSpider and other benchmarks have been ran to confirm that there are no regressions in performance compared to the previous implementation. The tests never performed worse and in some cases performed better. But the biggest differences can be seen in memory usage when large arrays are allocated. A simple test which allocates a 1000 arrays of size 100000 has roughly half the memory usage. There is also space in the representations for future expansions such as tuples and records. To ensure that Values on the stack and registers are not lost during garbage collection we also have to add a check to the Heap to check for any of the cell tags and extracting the canonical form of the pointer if it matches.
2022-02-25 01:26:52 +01:00
auto add_possible_value = [&](FlatPtr data) {
if constexpr (sizeof(FlatPtr*) == sizeof(Value)) {
// Because Value stores pointers in non-canonical form we have to check if the top bytes
// match any pointer-backed tag, in that case we have to extract the pointer to its
// canonical form and add that as a possible pointer.
if ((data & SHIFTED_IS_CELL_PATTERN) == SHIFTED_IS_CELL_PATTERN)
AK+LibJS: Handle NaN-boxing pointers on AArch64 JS::Value stores 48 bit pointers to separately allocated objects in its payload. On x86-64, canonical addresses have their top 16 bits set to the same value as bit 47, effectively meaning that the value has to be sign-extended to get the pointer. AArch64, however, expects the topmost bits to be all zeros. This commit gates sign extension behind `#if ARCH(X86_64)`, and adds an `#error` for unsupported architectures, so that we do not forget to think about pointer handling when porting to a new architecture. Fixes #15290 Fixes SerenityOS/ladybird#56
2022-09-20 18:09:33 +02:00
possible_pointers.set(Value::extract_pointer_bits(data));
LibJS: Use NaN boxing to decrease the memory size of Values Using the fact that there are 2^52-2 NaN representations we can "NaN-box" all the Values possible. This means that Value no longer has an explicit "Type" but that information is now stored in the bits of a double. This is done by "tagging" the top two bytes of the double. For a full explanation see the large comment with asserts at the top of Value. We can also use the exact representation of the tags to make checking properties like nullish, or is_cell quicker. But the largest gains are in the fact that the size of a Value is now halved. The SunSpider and other benchmarks have been ran to confirm that there are no regressions in performance compared to the previous implementation. The tests never performed worse and in some cases performed better. But the biggest differences can be seen in memory usage when large arrays are allocated. A simple test which allocates a 1000 arrays of size 100000 has roughly half the memory usage. There is also space in the representations for future expansions such as tuples and records. To ensure that Values on the stack and registers are not lost during garbage collection we also have to add a check to the Heap to check for any of the cell tags and extracting the canonical form of the pointer if it matches.
2022-02-25 01:26:52 +01:00
else
possible_pointers.set(data);
} else {
static_assert((sizeof(Value) % sizeof(FlatPtr*)) == 0);
// In the 32-bit case we will look at the top and bottom part of Value separately we just
// add both the upper and lower bytes as possible pointers.
possible_pointers.set(data);
}
};
LibJS: Be a bit more explicit about sizeof(buf) / sizeof(FlatPtr) This (seemingly) no-op cast communicates our intention to clang
2020-05-08 22:31:43 +04:30
for (size_t i = 0; i < ((size_t)sizeof(buf)) / sizeof(FlatPtr); i += sizeof(FlatPtr))
LibJS: Use NaN boxing to decrease the memory size of Values Using the fact that there are 2^52-2 NaN representations we can "NaN-box" all the Values possible. This means that Value no longer has an explicit "Type" but that information is now stored in the bits of a double. This is done by "tagging" the top two bytes of the double. For a full explanation see the large comment with asserts at the top of Value. We can also use the exact representation of the tags to make checking properties like nullish, or is_cell quicker. But the largest gains are in the fact that the size of a Value is now halved. The SunSpider and other benchmarks have been ran to confirm that there are no regressions in performance compared to the previous implementation. The tests never performed worse and in some cases performed better. But the biggest differences can be seen in memory usage when large arrays are allocated. A simple test which allocates a 1000 arrays of size 100000 has roughly half the memory usage. There is also space in the representations for future expansions such as tuples and records. To ensure that Values on the stack and registers are not lost during garbage collection we also have to add a check to the Heap to check for any of the cell tags and extracting the canonical form of the pointer if it matches.
2022-02-25 01:26:52 +01:00
add_possible_value(raw_jmp_buf[i]);
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
LibJS: Fix clang-tidy warnings about redundant types in Heap.cpp
2021-05-25 19:03:30 +02:00
auto stack_reference = bit_cast<FlatPtr>(&dummy);
LibJS+AK: Move cross-platform stack bounds code from JS::Heap to AK::StackInfo This will be useful for other things than the Heap, maybe even outside of LibJS.
2020-11-08 12:48:16 +00:00
auto& stack_info = m_vm.stack_info();
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
LibJS+AK: Move cross-platform stack bounds code from JS::Heap to AK::StackInfo This will be useful for other things than the Heap, maybe even outside of LibJS.
2020-11-08 12:48:16 +00:00
for (FlatPtr stack_address = stack_reference; stack_address < stack_info.top(); stack_address += sizeof(FlatPtr)) {
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
auto data = *reinterpret_cast<FlatPtr*>(stack_address);
LibJS: Use NaN boxing to decrease the memory size of Values Using the fact that there are 2^52-2 NaN representations we can "NaN-box" all the Values possible. This means that Value no longer has an explicit "Type" but that information is now stored in the bits of a double. This is done by "tagging" the top two bytes of the double. For a full explanation see the large comment with asserts at the top of Value. We can also use the exact representation of the tags to make checking properties like nullish, or is_cell quicker. But the largest gains are in the fact that the size of a Value is now halved. The SunSpider and other benchmarks have been ran to confirm that there are no regressions in performance compared to the previous implementation. The tests never performed worse and in some cases performed better. But the biggest differences can be seen in memory usage when large arrays are allocated. A simple test which allocates a 1000 arrays of size 100000 has roughly half the memory usage. There is also space in the representations for future expansions such as tuples and records. To ensure that Values on the stack and registers are not lost during garbage collection we also have to add a check to the Heap to check for any of the cell tags and extracting the canonical form of the pointer if it matches.
2022-02-25 01:26:52 +01:00
add_possible_value(data);
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
}
LibJS: Add JS::SafeFunction, like Function but protects captures from GC SafeFunction automatically registers its closure memory area in a place where the JS garbage collector can find it. This means that you can capture JS::Value and arbitrary pointers into the GC heap in closures, as long as you're using a SafeFunction, and the GC will not zap those values! There's probably some performance impact from this, and there's a lot of things that could be nicer/smarter about it, but let's build something that ensures safety first, and we can worry about performance later. :^)
2022-09-24 11:56:43 +02:00
// NOTE: If we have any custom ranges registered, scan those as well.
// This is where JS::SafeFunction closures get marked.
if (s_custom_ranges_for_conservative_scan) {
for (auto& custom_range : *s_custom_ranges_for_conservative_scan) {
for (size_t i = 0; i < (custom_range.value / sizeof(FlatPtr)); ++i) {
add_possible_value(custom_range.key[i]);
}
}
}
LibJS: Use a HashTable to identify potential cell pointers in GC scan Previously we would iterate over all the live HeapBlocks in order to learn if an arbitrary pointer-sized value was a pointer into a live HeapBlock. This was quite time-consuming. Instead of that, just put all the live HeapBlock*'s in a HashTable and identify pointers by doing a bit-masked lookup into the table.
2020-11-10 20:24:08 +01:00
HashTable<HeapBlock*> all_live_heap_blocks;
for_each_block([&](auto& block) {
all_live_heap_blocks.set(&block);
return IterationDecision::Continue;
});
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
for (auto possible_pointer : possible_pointers) {
if (!possible_pointer)
continue;
Everywhere: Run clang-format
2022-04-01 20:58:27 +03:00
dbgln_if(HEAP_DEBUG, " ? {}", (void const*)possible_pointer);
auto* possible_heap_block = HeapBlock::from_cell(reinterpret_cast<Cell const*>(possible_pointer));
LibJS: Use a HashTable to identify potential cell pointers in GC scan Previously we would iterate over all the live HeapBlocks in order to learn if an arbitrary pointer-sized value was a pointer into a live HeapBlock. This was quite time-consuming. Instead of that, just put all the live HeapBlock*'s in a HashTable and identify pointers by doing a bit-masked lookup into the table.
2020-11-10 20:24:08 +01:00
if (all_live_heap_blocks.contains(possible_heap_block)) {
if (auto* cell = possible_heap_block->cell_from_possible_pointer(possible_pointer)) {
LibJS: Replace Cell live bit with a cell state So far we only have two states: Live and Dead. In the future, we can add additional states to support incremental sweeping and/or multi- stage cell destruction.
2021-05-25 18:35:27 +02:00
if (cell->state() == Cell::State::Live) {
Everywhere: Run clang-format
2022-04-01 20:58:27 +03:00
dbgln_if(HEAP_DEBUG, " ?-> {}", (void const*)cell);
LibJS: Use a HashTable to identify potential cell pointers in GC scan Previously we would iterate over all the live HeapBlocks in order to learn if an arbitrary pointer-sized value was a pointer into a live HeapBlock. This was quite time-consuming. Instead of that, just put all the live HeapBlock*'s in a HashTable and identify pointers by doing a bit-masked lookup into the table.
2020-11-10 20:24:08 +01:00
roots.set(cell);
} else {
Everywhere: Run clang-format
2022-04-01 20:58:27 +03:00
dbgln_if(HEAP_DEBUG, " #-> {}", (void const*)cell);
LibJS: Use a HashTable to identify potential cell pointers in GC scan Previously we would iterate over all the live HeapBlocks in order to learn if an arbitrary pointer-sized value was a pointer into a live HeapBlock. This was quite time-consuming. Instead of that, just put all the live HeapBlock*'s in a HashTable and identify pointers by doing a bit-masked lookup into the table.
2020-11-10 20:24:08 +01:00
}
LibJS: Implement basic conservative garbage collection We now scan the stack and CPU registers for potential pointers into the GC heap, and include any valid Cell pointers in the set of roots. This works pretty well but we'll also need to solve marking of things passed to native functions, since those are currently in Vector<Value> and the Vector storage is on the heap (not scanned.)
2020-03-16 19:08:59 +01:00
}
}
}
}
LibJS: Make the GC marking phase cycle-proof Don't visit cells that are already marked. This prevents the marking phase from looping forever when two cells refer to each other. Also do the marking directly from the CellVisitor, removing another unnecessary phase of the collector. :^)
2020-03-09 22:11:22 +01:00
class MarkingVisitor final : public Cell::Visitor {
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
public:
Libraries: Use default constructors/destructors in LibJS https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#cother-other-default-operation-rules "The compiler is more likely to get the default semantics right and you cannot implement these functions better than the compiler."
2022-03-14 10:25:06 -06:00
MarkingVisitor() = default;
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
Everywhere: Fix -Winconsistent-missing-override warnings from Clang This option is already enabled when building Lagom, so let's enable it for the main build too. We will no longer be surprised by Lagom Clang CI builds failing while everything compiles locally. Furthermore, the stronger `-Wsuggest-override` warning is enabled in this commit, which enforces the use of the `override` keyword in all classes, not just those which already have some methods marked as `override`. This works with both GCC and Clang.
2021-12-04 10:09:09 +01:00
virtual void visit_impl(Cell& cell) override
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
{
LibJS: Make Cell::Visitor::visit_impl() take a Cell& Passing a null cell pointer is not supported.
2021-05-25 18:39:01 +02:00
if (cell.is_marked())
LibJS: Make the GC marking phase cycle-proof Don't visit cells that are already marked. This prevents the marking phase from looping forever when two cells refer to each other. Also do the marking directly from the CellVisitor, removing another unnecessary phase of the collector. :^)
2020-03-09 22:11:22 +01:00
return;
LibJS: Fix broken dbgln_if(HEAP_DEBUG)
2021-05-25 19:44:32 +02:00
dbgln_if(HEAP_DEBUG, " ! {}", &cell);
LibJS+js+test-js: Add GC debug mode that keeps cells "alive" as zombies This patch adds a `-z` option to js and test-js. When run in this mode, garbage cells are never actually destroyed. We instead keep them around in a special zombie state. This allows us to validate that zombies don't get marked in future GC scans (since there were not supposed to be any more references!) :^) Cells get notified when they become a zombie (via did_become_zombie()) and this is used by WeakContainer cells to deregister themselves from the heap.
2021-09-11 16:44:40 +02:00
LibJS: Make Cell::Visitor::visit_impl() take a Cell& Passing a null cell pointer is not supported.
2021-05-25 18:39:01 +02:00
cell.set_marked(true);
cell.visit_edges(*this);
LibJS: Make the GC marking phase cycle-proof Don't visit cells that are already marked. This prevents the marking phase from looping forever when two cells refer to each other. Also do the marking directly from the CellVisitor, removing another unnecessary phase of the collector. :^)
2020-03-09 22:11:22 +01:00
}
};
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
Everywhere: Run clang-format
2022-04-01 20:58:27 +03:00
void Heap::mark_live_cells(HashTable<Cell*> const& roots)
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
{
LibJS: Use dbgln_if in Heap.cpp
2021-04-07 15:12:32 +02:00
dbgln_if(HEAP_DEBUG, "mark_live_cells:");
test-js: Add a mark_as_garbage method to force GC to collect that object This should fix the flaky tests of test-js. It also fixes the tests when running with the -g flag since the values will not be garbage collected too soon.
2021-09-07 17:14:05 +02:00
LibJS: Make the GC marking phase cycle-proof Don't visit cells that are already marked. This prevents the marking phase from looping forever when two cells refer to each other. Also do the marking directly from the CellVisitor, removing another unnecessary phase of the collector. :^)
2020-03-09 22:11:22 +01:00
MarkingVisitor visitor;
LibJS: Allow cells to mark null pointers This simplifies the cell visiting functions by letting them not worry about the pointers they pass to the visitor being null.
2020-04-16 16:07:50 +02:00
for (auto* root : roots)
LibJS: Make the GC marking phase cycle-proof Don't visit cells that are already marked. This prevents the marking phase from looping forever when two cells refer to each other. Also do the marking directly from the CellVisitor, removing another unnecessary phase of the collector. :^)
2020-03-09 22:11:22 +01:00
visitor.visit(root);
test-js: Add a mark_as_garbage method to force GC to collect that object This should fix the flaky tests of test-js. It also fixes the tests when running with the -g flag since the values will not be garbage collected too soon.
2021-09-07 17:14:05 +02:00
for (auto& inverse_root : m_uprooted_cells)
inverse_root->set_marked(false);
m_uprooted_cells.clear();
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
Everywhere: Run clang-format
2022-04-01 20:58:27 +03:00
void Heap::sweep_dead_cells(bool print_report, Core::ElapsedTimer const& measurement_timer)
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
{
LibJS: Use dbgln_if in Heap.cpp
2021-04-07 15:12:32 +02:00
dbgln_if(HEAP_DEBUG, "sweep_dead_cells:");
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
Vector<HeapBlock*, 32> empty_blocks;
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
Vector<HeapBlock*, 32> full_blocks_that_became_usable;
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
LibJS: Add API for doing GC with a little debug log report at end You can now pass print_report=true to Heap::collect_garbage() and it will print out a little summary of the time spent, and counts of live vs freed cells and blocks.
2020-08-16 20:33:56 +02:00
size_t collected_cells = 0;
size_t live_cells = 0;
size_t collected_cell_bytes = 0;
size_t live_cell_bytes = 0;
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
for_each_block([&](auto& block) {
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
bool block_has_live_cells = false;
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
bool block_was_full = block.is_full();
LibJS: Replace Cell live bit with a cell state So far we only have two states: Live and Dead. In the future, we can add additional states to support incremental sweeping and/or multi- stage cell destruction.
2021-05-25 18:35:27 +02:00
block.template for_each_cell_in_state<Cell::State::Live>([&](Cell* cell) {
if (!cell->is_marked()) {
dbgln_if(HEAP_DEBUG, " ~ {}", cell);
LibJS: Remove the JS_TRACK_ZOMBIE_CELLS option This feature had bitrotted somewhat and would trigger errors because PrimitiveStrings were "destroyed" but because of this mode they were not removed from the string cache. Even fixing that case running test-js with the options still failed in more places.
2022-02-04 16:12:39 +01:00
block.deallocate(cell);
LibJS: Replace Cell live bit with a cell state So far we only have two states: Live and Dead. In the future, we can add additional states to support incremental sweeping and/or multi- stage cell destruction.
2021-05-25 18:35:27 +02:00
++collected_cells;
collected_cell_bytes += block.cell_size();
} else {
cell->set_marked(false);
block_has_live_cells = true;
++live_cells;
live_cell_bytes += block.cell_size();
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
});
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
if (!block_has_live_cells)
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
empty_blocks.append(&block);
else if (block_was_full != block.is_full())
full_blocks_that_became_usable.append(&block);
return IterationDecision::Continue;
});
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
LibJS: Prune WeakContainers before freeing HeapBlocks WeakContainers need to look at the Cell::State bits to know if their weak pointees got swept by garbage collection. So we must do this before potentially freeing one or more HeapBlocks by notifying the allocator that a block became empty.
2021-10-08 19:47:25 +02:00
for (auto& weak_container : m_weak_containers)
weak_container.remove_dead_cells({});
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
for (auto* block : empty_blocks) {
LibJS: Use dbgln_if in Heap.cpp
2021-04-07 15:12:32 +02:00
dbgln_if(HEAP_DEBUG, " - HeapBlock empty @ {}: cell_size={}", block, block->cell_size());
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
allocator_for_size(block->cell_size()).block_did_become_empty({}, *block);
LibJS: Delete fully-empty HeapBlocks after garbage collection We now deallocate GC blocks when they are found to have no live cells inside them.
2020-03-21 11:45:50 +01:00
}
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
for (auto* block : full_blocks_that_became_usable) {
LibJS: Use dbgln_if in Heap.cpp
2021-04-07 15:12:32 +02:00
dbgln_if(HEAP_DEBUG, " - HeapBlock usable again @ {}: cell_size={}", block, block->cell_size());
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
allocator_for_size(block->cell_size()).block_did_become_usable({}, *block);
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
LibJS: Use 'if constexpr' instead of '#if HEAP_DEBUG'
2021-04-18 18:12:33 +02:00
if constexpr (HEAP_DEBUG) {
for_each_block([&](auto& block) {
dbgln(" > Live HeapBlock @ {}: cell_size={}", &block, block.cell_size());
return IterationDecision::Continue;
});
}
LibJS: Add API for doing GC with a little debug log report at end You can now pass print_report=true to Heap::collect_garbage() and it will print out a little summary of the time spent, and counts of live vs freed cells and blocks.
2020-08-16 20:33:56 +02:00
int time_spent = measurement_timer.elapsed();
if (print_report) {
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
size_t live_block_count = 0;
for_each_block([&](auto&) {
++live_block_count;
return IterationDecision::Continue;
});
LibJS: Replace a few dbg() with dbgln()
2020-10-04 15:44:40 +01:00
dbgln("Garbage collection report");
dbgln("=============================================");
dbgln(" Time spent: {} ms", time_spent);
dbgln(" Live cells: {} ({} bytes)", live_cells, live_cell_bytes);
dbgln("Collected cells: {} ({} bytes)", collected_cells, collected_cell_bytes);
LibJS: Split Heap into per-cell-size allocators Instead of keeping all the HeapBlocks in one big list, we now split it into two levels: - Heap has a set of Allocators, each with a specific cell size. - Allocators have two lists of blocks, "full" and "usable". Allocating a new cell no longer has to scan the entire set of blocks, but instead just needs to find the right allocator and then pop a cell from its freelist. If all the blocks in the allocator are full, a new block will be created. Blocks are moved from the "full" to "usable" list after sweeping has determined that they are not completely empty and not completely full. There are certainly many ways we can improve on this. This patch is mostly about getting the new allocator architecture in place. :^)
2020-10-06 18:50:47 +02:00
dbgln(" Live blocks: {} ({} bytes)", live_block_count, live_block_count * HeapBlock::block_size);
LibJS: Replace a few dbg() with dbgln()
2020-10-04 15:44:40 +01:00
dbgln(" Freed blocks: {} ({} bytes)", empty_blocks.size(), empty_blocks.size() * HeapBlock::block_size);
dbgln("=============================================");
LibJS: Add API for doing GC with a little debug log report at end You can now pass print_report=true to Heap::collect_garbage() and it will print out a little summary of the time spent, and counts of live vs freed cells and blocks.
2020-08-16 20:33:56 +02:00
}
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
LibJS: Add Handle<T>, a strong C++ handle for keeping GC objects alive This is pretty heavy and unoptimized, but it will do the trick for now. Basically, Heap now has a HashTable<HandleImpl*> and you can call JS::make_handle(T*) to construct a Handle<T> that guarantees that the pointee will always survive GC until the Handle<T> is destroyed.
2020-03-18 20:03:17 +01:00
void Heap::did_create_handle(Badge<HandleImpl>, HandleImpl& impl)
{
LibJS: Use IntrusiveList for keeping track of HandleImpls This allows us to remove a HashTable from heap and cuts down on some of the malloc traffic when creating handles.
2021-07-21 19:45:21 +02:00
VERIFY(!m_handles.contains(impl));
m_handles.append(impl);
LibJS: Add Handle<T>, a strong C++ handle for keeping GC objects alive This is pretty heavy and unoptimized, but it will do the trick for now. Basically, Heap now has a HashTable<HandleImpl*> and you can call JS::make_handle(T*) to construct a Handle<T> that guarantees that the pointee will always survive GC until the Handle<T> is destroyed.
2020-03-18 20:03:17 +01:00
}
void Heap::did_destroy_handle(Badge<HandleImpl>, HandleImpl& impl)
{
LibJS: Use IntrusiveList for keeping track of HandleImpls This allows us to remove a HashTable from heap and cuts down on some of the malloc traffic when creating handles.
2021-07-21 19:45:21 +02:00
VERIFY(m_handles.contains(impl));
m_handles.remove(impl);
LibJS: Add Handle<T>, a strong C++ handle for keeping GC objects alive This is pretty heavy and unoptimized, but it will do the trick for now. Basically, Heap now has a HashTable<HandleImpl*> and you can call JS::make_handle(T*) to construct a Handle<T> that guarantees that the pointee will always survive GC until the Handle<T> is destroyed.
2020-03-18 20:03:17 +01:00
}
LibJS: Add MarkedVector<T> This abstracts a vector of Cell* with a strongly typed span() accessor that gives you Span<T*> instead of Span<Cell*>. It is intended to replace MarkedValueList in situations where you only need to store pointers to Cell (or an even more specific type of Cell). The API can definitely be improved, it's just the bare basics for now.
2021-12-16 18:54:06 +01:00
void Heap::did_create_marked_vector(Badge<MarkedVectorBase>, MarkedVectorBase& vector)
{
VERIFY(!m_marked_vectors.contains(vector));
m_marked_vectors.append(vector);
}
void Heap::did_destroy_marked_vector(Badge<MarkedVectorBase>, MarkedVectorBase& vector)
{
VERIFY(m_marked_vectors.contains(vector));
m_marked_vectors.remove(vector);
}
LibJS: Generify the garbage collector's weak container notifications This will allow us to use the same interface for other JS weak containers like the WeakMap & WeakRef.
2021-06-12 05:23:33 +03:00
void Heap::did_create_weak_container(Badge<WeakContainer>, WeakContainer& set)
LibJS: Notify WeakSets when heap cells are sweeped This is an implementation of the following optional optimization: https://tc39.es/ecma262/#sec-weakref-execution
2021-06-09 20:10:47 +03:00
{
LibJS: Use IntrusiveList for keeping track of WeakContainers
2021-07-21 19:57:41 +02:00
VERIFY(!m_weak_containers.contains(set));
m_weak_containers.append(set);
LibJS: Notify WeakSets when heap cells are sweeped This is an implementation of the following optional optimization: https://tc39.es/ecma262/#sec-weakref-execution
2021-06-09 20:10:47 +03:00
}
LibJS: Generify the garbage collector's weak container notifications This will allow us to use the same interface for other JS weak containers like the WeakMap & WeakRef.
2021-06-12 05:23:33 +03:00
void Heap::did_destroy_weak_container(Badge<WeakContainer>, WeakContainer& set)
LibJS: Notify WeakSets when heap cells are sweeped This is an implementation of the following optional optimization: https://tc39.es/ecma262/#sec-weakref-execution
2021-06-09 20:10:47 +03:00
{
LibJS: Use IntrusiveList for keeping track of WeakContainers
2021-07-21 19:57:41 +02:00
VERIFY(m_weak_containers.contains(set));
m_weak_containers.remove(set);
LibJS: Notify WeakSets when heap cells are sweeped This is an implementation of the following optional optimization: https://tc39.es/ecma262/#sec-weakref-execution
2021-06-09 20:10:47 +03:00
}
LibJS: Add DeferGC, a RAII way to prevent GC temporarily
2020-04-19 11:30:47 +02:00
void Heap::defer_gc(Badge<DeferGC>)
{
++m_gc_deferrals;
}
void Heap::undefer_gc(Badge<DeferGC>)
{
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 20:42:32 +01:00
VERIFY(m_gc_deferrals > 0);
LibJS: Add DeferGC, a RAII way to prevent GC temporarily
2020-04-19 11:30:47 +02:00
--m_gc_deferrals;
if (!m_gc_deferrals) {
if (m_should_gc_when_deferral_ends)
collect_garbage();
m_should_gc_when_deferral_ends = false;
}
}
test-js: Add a mark_as_garbage method to force GC to collect that object This should fix the flaky tests of test-js. It also fixes the tests when running with the -g flag since the values will not be garbage collected too soon.
2021-09-07 17:14:05 +02:00
void Heap::uproot_cell(Cell* cell)
{
m_uprooted_cells.append(cell);
}
LibJS: Add JS::SafeFunction, like Function but protects captures from GC SafeFunction automatically registers its closure memory area in a place where the JS garbage collector can find it. This means that you can capture JS::Value and arbitrary pointers into the GC heap in closures, as long as you're using a SafeFunction, and the GC will not zap those values! There's probably some performance impact from this, and there's a lot of things that could be nicer/smarter about it, but let's build something that ensures safety first, and we can worry about performance later. :^)
2022-09-24 11:56:43 +02:00
void register_safe_function_closure(void* base, size_t size)
{
if (!s_custom_ranges_for_conservative_scan) {
// FIXME: This per-thread HashMap is currently leaked on thread exit.
s_custom_ranges_for_conservative_scan = new HashMap<FlatPtr*, size_t>;
}
auto result = s_custom_ranges_for_conservative_scan->set(reinterpret_cast<FlatPtr*>(base), size);
VERIFY(result == AK::HashSetResult::InsertedNewEntry);
}
void unregister_safe_function_closure(void* base, size_t)
{
VERIFY(s_custom_ranges_for_conservative_scan);
bool did_remove = s_custom_ranges_for_conservative_scan->remove(reinterpret_cast<FlatPtr*>(base));
VERIFY(did_remove);
}
LibJS: Add a basic mark&sweep garbage collector :^) Objects can now be allocated via the interpreter's heap. Objects that are allocated in this way will need to be provably reachable from at least one of the known object graph roots. The roots are currently determined by Heap::collect_roots(). Anything that wants be collectable garbage should inherit from Cell, the fundamental atom of the GC heap. This is pretty neat! :^)
2020-03-08 19:23:58 +01:00
}
Reference in a new issue Copy permalink