Previously, iframes were rasterized synchronously as nested display
lists inside their parent's display list: the parent's paint walk called
record_display_list() on each hosted iframe document and emitted a
PaintNestedDisplayList command that the player would recurse into. Only
the top-level traversable's RenderingThread was ever active, even though
every Navigable already owned one.
The motivation for splitting this apart:
- Work in the outer document no longer has to be re-recorded when only
an iframe changes. The parent's cached display list now references the
iframe's rasterized output live via an ExternalContentSource, so an
iframe invalidation just needs the parent's display list replayed, not
re-recorded.
- Each iframe now has a self-contained rasterization pipeline, which is
prep work for moving iframes into separate sandboxed processes.
WebContent process keeps session history entries for pages we have
navigated away from. Before this change, those entries could prevent GC
objects (e.g. PolicyContainer and its CSP PolicyList) from being
collected, since the GC-allocated SHE/DocumentState held live GC::Ref
pointers into the heap.
By making both classes RefCounted and storing SerializedPolicyContainer
instead of a live PolicyContainer, history entries no longer keep alive
any GC objects. This eliminates the leak and is also a step toward
moving the session history entry tree to the UI process.
This cannot happen inside the Make Active algorithm, since that gets
called during document creation, which commonly happens before the
document's navigable is created.
Aligns us with a recent spec change and rids us of some AD_HOC
behavior.
Now that Navigable directly owns its active document (m_active_document)
we can have Navigable maintain a back-pointer on Document instead of
using the old cache-with-validation pattern that fell back to a linear
scan of all navigables via navigable_with_active_document().
Previously, the active document's lifecycle was bound to
SessionHistoryEntry via DocumentState. The ownership chain was:
Navigable → SessionHistoryEntry → DocumentState → Document
This made it impossible to move SessionHistoryEntry to the UI process
(which cannot own DOM::Document). This commit decouples the two by
giving Navigable a direct m_active_document field that serves as the
authoritative source for active_document().
- Navigable owns m_active_document directly; active_document() reads
from it instead of going through the active session history entry.
- DocumentState no longer holds a Document pointer. Instead, it stores
a document_id for "same document?" checks. Same-document navigations
share a DocumentState and thus the same document_id, while
cross-document navigations create a new DocumentState with a new ID.
- A pending_document parameter is threaded through
finalize_a_cross_document_navigation → apply_the_push_or_replace →
apply_the_history_step so the newly created document reaches
activation without being stored on DocumentState.
- For traversal, the population output delivers the document.
A resolved_document is computed per continuation from either the
pending document, the population output, or the current active
document (for same-document traversals).
Replace the blocking spin_processing_tasks_with_source_until calls
in apply_the_history_step_after_unload_check() with an event-driven
ApplyHistoryStepState GC cell that tracks 5 phases, following the
same pattern used by CheckUnloadingCanceledState.
Key changes:
- Introduce ApplyHistoryStepState with phases:
WaitingForDocumentPopulation, ProcessingContinuations,
WaitingForChangeJobCompletion, WaitingForNonChangingJobs and Completed
- Add on_complete callbacks to apply_the_push_or_replace_history_step,
finalize_a_same_document_navigation,
finalize_a_cross_document_navigation, and
update_for_navigable_creation_or_destruction
- Remove spin_until from Document::open()
- Use null-document tasks for non-changing navigable updates and
document unload/destroy to avoid stuck tasks when documents become
non-fully-active
- Defer completely_finish_loading when document has no navigable yet,
and re-trigger post-load steps in activate_history_entry for documents
that completed loading before activation
Co-Authored-By: Shannon Booth <shannon@serenityos.org>
HTMLParser::the_end() had three spin_until calls that blocked the event
loop: step 5 (deferred scripts), step 7 (ASAP scripts), and step 8
(load event delay). This replaces them with an HTMLParserEndState state
machine that progresses asynchronously via callbacks.
The state machine has three phases matching the three spin_until calls:
- WaitingForDeferredScripts: loops executing ready deferred scripts
- WaitingForASAPScripts: waits for ASAP script lists to empty
- WaitingForLoadEventDelay: waits for nothing to delay the load event
Notification triggers re-evaluate the state machine when conditions
change: HTMLScriptElement::mark_as_ready, stylesheet unblocking in
StyleElementBase/HTMLLinkElement, did_stop_being_active_document, and
DocumentLoadEventDelayer decrements. NavigableContainer state changes
(session history readiness, content navigable cleared, lazy load flag)
also trigger re-evaluation of the load event delay check.
Key design decisions and why:
1. Microtask checkpoint in schedule_progress_check(): The old spin_until
called perform_a_microtask_checkpoint() before checking conditions.
This is critical because HTMLImageElement::update_the_image_data step
8 queues a microtask that creates the DocumentLoadEventDelayer.
Without the checkpoint, check_progress() would see zero delayers and
complete before images start delaying the load event.
2. deferred_invoke in schedule_progress_check():
I tried Core::Timer (0ms), queue_global_task, and synchronous calls.
Timers caused non-deterministic ordering with the HTML event loop's
task processing timer, leading to image layout tests failing (wrong
subtest pass/fail patterns). Synchronous calls fired too early during
image load processing before dimensions were set, causing 0-height
images in layout tests. queue_global_task had task ordering issues
with the session history traversal queue. deferred_invoke runs after
the current callback returns but within the same event loop pump,
giving the right balance.
3. Navigation load event guard (m_navigation_load_event_guard): During
cross-document navigation, finalize_a_cross_document_navigation step
2 calls set_delaying_load_events(false) before the session history
traversal activates the new document. This creates a transient state
where the parent's load event delay check sees the about:blank (which
has ready_for_post_load_tasks=true) as the active document and
completes prematurely.
Replace the two spin_processing_tasks_with_source_until() calls in
TraversableNavigable::check_if_unloading_is_canceled() with a
callback-based GC cell (CheckUnloadingCanceledState) that tracks
completion across both phases (traverse navigate event + per-document
beforeunload handlers) and invokes a callback when done.
This required making check_if_unloading_is_canceled() async
(callback-based), splitting apply_the_history_step() into pre-check
and continuation parts, and updating all callers to move session
history traversal queue promise resolution into callbacks.
The trusted-event test is rebaselined because beforeunload now fires
as a queued NavigationAndTraversal task rather than being processed
inline by spin_until. This allows the unhandledrejection microtask
to run before the beforeunload task, swapping their order.
With apply_to() now self-contained (carrying its own replacement
DocumentState rather than reading from the live entry), the clone at
the traversal call site is no longer needed.
The clone previously served two purposes:
1. Input snapshot: freeze entry fields before deferred population.
Now solved by changing populate_session_history_entry_document() to
take explicit input parameters, snapshotted before the
deferred_invoke.
2. Output isolation: absorb apply_to() and post-population adjustments
without mutating the live entry during unload. Now solved by storing
the PopulateSessionHistoryEntryDocumentOutput on the continuation
state and deferring all mutations (including the origin-based
classic_history_api_state reset and navigable_target_name clear)
to after_potential_unload.
The post-population adjustments run unconditionally in
after_potential_unload, covering both the population path and the
non-population path (e.g. traversal to an already-populated error
entry).
Previously, populate_session_history_entry_document() took a
SessionHistoryEntry as both input and output — reading URL and
document_state fields while also mutating the entry across a chain of
async functions. This made it very hard to reason about data flow.
Refactor the internal helpers
(create_navigation_params_from_a_srcdoc_resource,
create_navigation_params_by_fetching, NavigationParamsFetchStateHolder,
perform_navigation_params_fetch) to take individual field values instead
of reading from the entry, and accumulate redirect mutations on the
state holder rather than writing them to the entry immediately.
Introduce PopulateSessionHistoryEntryDocumentOutput, a GC cell that
collects all mutations (document, redirect URL, classic history API
state, replacement document state, resource cleared flag, and
finalization data). The completion_steps callback now receives this
output object (or nullptr on cancellation), and callers apply it to the
entry via apply_to().
The replacement DocumentState for the redirect path is built eagerly at
redirect time from values captured on the state holder, making
apply_to() fully self-contained — it never reads from the target entry's
live document_state. This is important for the traversal path where the
entry may be mutated during unload (e.g. window.name writes
navigable_target_name through the active session history entry).
Other browsers appear to only do this for form submission, not for
all javascript URL navigations. Let's remove the handling in the
general javascript URL navigation handling so that our behaviour
diference to other browsers is limited specifically to form
elements, instead of the general case.
Unfortunately this does (expectedly) cause the test added in
3e0ea4f62 to start timing out, so that test is marked as skipped.
The scroll state collection loop in
record_display_list_and_scroll_state() called paintable() on hosted
documents, which asserts layout is up to date. This crashes when a
nested document has stale layout but a cached display list, e.g. a
render-blocked iframe whose DOM was modified by document.open().
Since scroll offsets are independent of layout freshness, use
unsafe_paintable() to skip the assertion.
There was already a null check before this task was queued, but it's
possible for the active window to become null between the time the task
is queued and executed, so we need to check again.
Previously, destroyed-document tasks were forced to be runnable to
prevent them from leaking in the task queue. Instead, discard them
during task selection so their callbacks never run with stale state.
This used to cause issues with a couple of `spin_until()`s in the past,
but since we've removed some of them that had to do with the document
lifecycle, let's see if we can stick closer to the spec now.
Replace per-node heap-allocated AtomicRefCounted
AccumulatedVisualContext objects with a single contiguous Vector inside
AccumulatedVisualContextTree. All nodes for a frame are now stored in
one allocation, using type-safe VisualContextIndex instead of RefPtr
pointers.
This reduces allocation churn, improves cache locality, and opens the
door for future snapshotting of visual context state — similar to how
scroll offsets are snapshotted today.
Extract the repeated pattern of transforming a rectangle from absolute
coordinates to viewport coordinates via the accumulated visual context
into a helper method.
Rename Document::set_needs_display() to set_needs_repaint() and make it
private. External callers must now go through Node/Paintable which
route the request to the document internally.
Fix one existing misuse in AnimationEffect that was calling
document-level set_needs_display() instead of routing through the
target element's paintable.
This is preparation for per-paintable display list command caching:
repaint requests must go through specific paintables so their cached
command lists can be invalidated.
This change means the right click context menu is displayed in the right
place when clicking inside an iframe on a scrolled page, including when
the iframe has CSS transforms applied to it.
Add unsafe_layout_node(), unsafe_paintable(), and unsafe_paintable_box()
accessors that skip layout-staleness verification. These are for use in
contexts where accessing layout/paintable data is legitimate despite
layout not being up to date: tree construction, style recalculation,
painting, animation interpolation, DOM mutation, and invalidation
propagation.
Also add wrapper APIs on Node to centralize common patterns:
- set_needs_display() wraps if (unsafe_paintable()) ...set_needs_display
- set_needs_paint_only_properties_update() wraps similar
- set_needs_layout_update() wraps if (unsafe_layout_node()) ...
And add Document::layout_is_up_to_date() which checks whether layout
tree update flags are all clear.
Replace the Window::scroll_by(0, 0) call at the end of
Document::update_layout() with a dedicated
Navigable::clamp_viewport_scroll_offset() that directly clamps the
viewport scroll offset to valid bounds.
The old approach re-entered layout from within layout, since scroll_by()
would trigger another layout update. The new approach is called from the
event loop's rendering steps, after layout is complete.
perform_a_scroll_of_the_viewport() accesses paintable_box() without
ensuring layout is up to date. This can lead to a null dereference
if the paintable tree was torn down (e.g. by adding a dialog to the top
layer via showModal()) between the last layout update and the scroll.
One concrete path: Window::scroll() has an optimization that skips
update_layout when scrolling to (0, 0), but still calls
perform_a_scroll_of_the_viewport if the viewport is at a non-zero
position.
Fix by adding an update_layout call at the top of
perform_a_scroll_of_the_viewport.
In `TraversableNavigable::apply_the_history_step()`, we have a
`spin_until(..)` that pumps the event loop. If it's invoked as a result
of `Navigable::begin_navigation()` and we've received an IPC call at the
same time, we might reenter `begin_navigation()` which then drops the
new navigation as there is an ongoing navigation.
The IPC call most likely causing tests to time out, is
`page_did_finish_loading`.
This is a workaround until we get rid of the `spin_until()`. It fixes
almost all of the test timeouts I had locally on macOS.
The set_viewport_size and set_device_pixel_ratio IPC messages were sent
separately, potentially causing a race condition when the DPR changes
(e.g. moving a window between screens): the DPR message would arrive
and use a stale viewport size, computing a temporarily wrong CSS
viewport. Combine both into a single set_viewport IPC that updates the
device viewport size and DPR together.
The guard for setting top-level navigation initiator origin called
top_level_traversable()->parent() == nullptr, which is tautologically
true: top_level_traversable() already walks to the topmost traversable,
whose parent is always null. This caused the field to be set on every
navigation, including child navigable navigations inside iframes.
The value was also read from document_state()->origin() instead of
document_state()->initiator_origin(), giving the document's own origin
rather than the origin of whoever initiated the navigation.
Use is_top_level_traversable() and initiator_origin() to match the
spec step.
Step 8 of "create navigation params by fetching" says "if
request's body is null, then set entry's document state's
resource to null." The condition is inverted: it checks
!body().has<Empty>(), entering the block when the body is
non-null instead of null. For GET navigations this is mostly
a no-op since resource is typically already Empty, but for
POST navigations the inverted check incorrectly clears the
POSTResource from the document state, breaking session history
entries that rely on it for form re-submission.
Step 7.3 of finalize_session_history_entry checks whether
navigationParams "is a navigation params (i.e., neither null
nor a non-fetch scheme navigation params)." The condition
checks has<NonFetchSchemeNavigationParams>() instead of
has<NavigationParams>(), matching the wrong variant type.
The block body already accesses the variant as NavigationParams,
so if the condition ever matched it would VERIFY-crash. The
identical spec clause five lines above (step 7.2.3.1) is
implemented correctly as has<NavigationParams>().
The spec assertion "navigationParams's request is not null"
does not hold for srcdoc navigations, which create
NavigationParams with a null request. The condition guards
on request being non-null instead, with an NB comment
explaining the divergence.
Remove 11 heavy includes from Document.h that were only needed for
pointer/reference types (already forward-declared in Forward.h), and
extract the nested ViewportClient interface to a standalone header.
This reduces Document.h's recompilation cascade from ~1228 files to
~717 files (42% reduction). Headers like BrowsingContext.h that were
previously transitively included see even larger improvements (from
~1228 down to ~73 dependents).
When an iframe is removed before its post-connection session history
steps finish processing, the queued steps still run and call
navigate() on the now-destroyed navigable. This calls
set_delaying_load_events(true), creating a DocumentLoadEventDelayer
on the parent document. Since the navigable is destroyed, no
finalize step ever runs to clear the delayer, permanently blocking
the parent document's load event.
Fix this by checking has_been_destroyed() at the start of
begin_navigation() and bailing out early.
When the visual viewport scrolls (e.g. panning while zoomed in), the
transform matrix baked into the AccumulatedVisualContext tree becomes
stale. Previously only set_needs_display(InvalidateDisplayList::No) was
called, which re-rendered with the old transform — making hit testing
and painting incorrect after visual viewport panning.
I noticed that when performing a smooth scroll up or down, we would
repeatedly enter `Navigable::perform_a_scroll_of_the_viewport()` when
the top or bottom (respectively) of the page was reached. Optimize this
case by returning early when we've reached the intended target scroll
position and we're not performing any vertical or horizontal scrolling.
In preparation for handling input events on the rendering thread, move
backpressure management to RenderingThread. The rendering thread needs
to manage this independently without querying the main thread.
Previously, the main thread would block when the UI process hadn't yet
released the backing surface. Now, the main thread can continue
producing display lists while the UI process is busy, allowing more work
to happen in parallel. When rasterization is slow and display lists are
produced faster than they can be consumed, presentation requests are
naturally coalesced using a flag-based approach -
multiple present_frame() calls simply update the pending state,
resulting in a single rasterization with the latest display list.
See: 976d272
This fixes a timeout in the included test. In order for the
test to run properly in our test harness it is modified slightly
(as we are loading the HTML from a file:// URL).
This is preparation for future work where the rendering thread will
initiate rasterization independently and notify the UI process without
requiring coordination with the main thread.
This change prepares for a future where the rendering thread handles
input events directly, allowing it to trigger repainting without
waiting for the main thread. To support this, the compositor needs to
own the display list, scroll state, and backing stores rather than
receiving them per-frame from the main thread.
Previously, when loading a document, we would try to sniff the MIME
type by reading from the response body's source. However, for streaming
HTTP responses, the body source is Empty (the data comes through the
stream instead), so we had no bytes to sniff.
This caused pages like hypr.land (which sends no Content-Type header)
to be misidentified as plain text instead of HTML, since the MIME
sniffing algorithm would receive zero bytes and fall back to the
default type.
The fix captures the first bytes of the response body during fetch,
storing them on the Body object. These bytes are the "resource header"
defined by the MIME Sniffing spec - up to 1445 bytes, which is enough
to identify any MIME type the spec can detect.
Since bytes may arrive asynchronously during streaming, we use a
callback mechanism: if bytes aren't ready yet when load_document()
needs them, it registers a callback that fires once enough bytes have
been captured (or the stream ends).
The flow is:
1. FetchedDataReceiver receives network bytes, buffers them
2. When Body is created, buffered bytes are flushed to Body's sniff
buffer, and subsequent bytes are appended as they arrive
3. Before calling load_document(), Navigable waits for sniff bytes
4. load_document() passes the bytes to MimeSniff::Resource::sniff()
When scrolling with a visual viewport offset (from pinch-to-zoom),
scroll_viewport_by_delta() was passing m_viewport_scroll_offset + delta
to perform_a_scroll_of_the_viewport(). However, that function calculates
the scroll delta as `position - page_top()`, where page_top() includes
the visual viewport offset. This caused the effective scroll delta to be
reduced by the visual offset amount.
Fix by using the current page position (which includes the visual
offset) as the base for the delta calculation.
Regression from 0a57e1e8ac.
Cache the scroll state snapshot in ViewportPaintable when
refresh_scroll_state() is called.
The upcoming AccumulatedVisualContext integration requires access to
the scroll state snapshot during hit testing to transform screen
coordinates through scroll frames. Without caching, each hit test would
allocate a new snapshot (a Vector<Entry>), causing many temporary
allocations during mouse movement. Caching the snapshot eliminates this
overhead.
A version of this was added in a610639119
and reverted in 70671b4c11. The bugs
there (confusing scroll-to-position and scroll-by-delta, and not having
an execution context in some cases) have been fixed in this version.
