LibJS+DevTools: Implement console.trace() with source locations
- Add Console::TraceFrame struct with source location data
- Implement Console::trace() to gather stack information
- Add WebView::StackFrame and ConsoleTrace for IPC
- Implement DevToolsConsoleClient::printer() for traces
- Update FrameActor to format traces for DevTools
- Update WorkerDebugConsoleClient trace handling
- Update ReplConsoleClient to format trace output
This patch introduces a cookie cache in the WebContent process to reduce
blocking IPC calls when JS accesses document.cookie. The UI process now
maintains a cookie version counter per-domain in shared memory. When JS
reads document.cookie, we check whether we have a valid cached cookie by
comparing the current shared version to the last used version. If they
match, the cached cookie is returned without IPC.
This optimization is based on Chromium's shared versioning, in which it
was observed that 87% of document.cookie accesses were redundant. See:
https://blog.chromium.org/2024/06/introducing-shared-memory-versioning-to.html
Note that this cache only supports document.cookie, not HTTP Cookie
headers. HTTP cookies are attached to requests with varying URLs and
paths. The cookies that match the document URL might not match the
request URL, which we wouldn't know from WebContent. So attaching the
cached document cookie would be incorrect.
On https://twinings.co.uk, we see approximately 600 document.cookie
requests while the page loads. This patch reduces the time spent in
the document.cookie getter from ~45ms to 2-3ms.
This introduces a simple FileDownloader to download files in the UI
process from RequestServer. We use this to download the context menu
image - this download is likely to hit the disk cache.
We currently always save screenshots to the Downloads folder. We will
now always ask for a save location.
This will just let an upcoming feature to save images from web pages
behave the same way. We will want the user to be able to choose a file
name, since the file name from the URL might be nonsense or already
exist.
Previously we would resolve font features
(https://drafts.csswg.org/css-fonts-4/#feature-variation-precedence)
per element, while this works for the current subset of the font feature
resolution algorithm that we support, some as yet unimplemented parts
require us to know whether we are resolving against a CSS @font-face
rule, and if so which one (e.g. applying descriptors from the @font-face
rule, deciding which @font-feature-values rules to apply, etc).
To achieve this we store the data required to resolve font features in a
struct and pass that to `FontComputer` which resolves the font features
and stores them with the computed `Font`.
We no longer need to invalidate the font shaping cache when features
change since the features are defined per font (and therefore won't ever
change).
While our default font supporting variations is unlikely, this is
nevertheless required for our fallback font to be considered equal to
it's non-default/fallback equivalent (i.e. `font-family: serif`) which
in turn is required for LineBuilder to merge chunks into a single
fragment.
When dumping a GC graph, we now write the output as a .js file
containing `var GC_GRAPH_DUMP = <json>;` instead of raw JSON.
This allows gc-heap-explorer.html to load the dump via a
dynamically created <script> element, avoiding CORS restrictions
that prevent file:// pages from fetching other file:// URLs.
After dumping, both the browser and test-web print a clickable
file:// URL that opens the heap explorer with the dump pre-loaded.
The heap explorer's drag-and-drop file picker also accepts both
the new .js format and plain .json files.
These can get very large, exceeding the new IPC message size limits.
Instead of serializing them into messages (which was silly anyway)
we now send them as Core::AnonymousBuffer which uses shared memory.
When cookies change or expire, we currently send a list of all changed
cookies to all WebContent processes. We then filter that list in the
WebContent process for cookies that match the page's URL before sending
out cookie change events to JS.
We now perform this filtering in the UI process, so each WebContent
process only receives the cookies it would be interested in, if any.
This serves two purposes:
1. Less IPC chatter.
2. This will let each ViewImplementation know that its cookie value has
actually changed.
(2) is for an upcoming change that will introduce a cookie cache, and
will allow each view to know it should bust that cache.
Note that for this filtering to work, we must iterate ViewImplementation
instances rather than WebContentClient in order to have the view's URL.
We must then associate the IPC with the view's page ID.
No changes to the /cookiestore WPT subtests.
If a did_paint message is in-flight and we create a new process when
navigating to another website, we would still have the view registered
with the client when eventually the stale did_paint message is handled.
In server_did_paint, we retrieve the client which points to the new
process, not the original process the view was registered for.
At that point, there might not be any queued rasterization tasks in the
RenderingThread for the new process, causing a crash because of:
VERIFY(m_queued_rasterization_tasks >= 1 && ..);
Fix this by unregistering the view before proceeding with a new
WebContent process.
This is expected by WPT. For this to work, we must be able to determine
the network partition key for shared worker environments. So we now set
a top-level origin for these environments, with a FIXME to implement it
in accordance with the Client-Side Storage Partitioning spec.
We had skipped some steps in the spec and were:
* Always broadcasting an old value of null, instead of what it
actually was previously.
* Still broadcasting a storage event even if the value had
not changed in storage compared to the last value.
Fix both issues by returning what the old value is in the setter and
implementing the missing logic.
When rendering text, if none of the fonts in the cascade list contain a
glyph for a given code point, we now query Skia's font manager to find
a system font that can render it.
We currently disable the disk cache because the WPT runner will run more
than one RequestServer process at a time. The SQLite database does not
handle this concurrent read/write access well.
We will now enable the disk cache with a per-process database. This is
needed to ensure that WPT Fetch cache tests are sufficiently handled by
RequestServer.
This adds support for intercepting network requests and serving local
file content instead. When a URL matches an entry in the substitution
map, the local file is served while preserving the original URL's
origin for cross-origin checks.
Usage:
Ladybird --resource-map=/path/to/map.json
The JSON file format is:
{
"substitutions": [
{
"url": "https://example.com/script.js",
"file": "/path/to/local/script.js",
"content_type": "application/javascript",
"status_code": 200
}
]
}
Fields:
- url (required): Exact URL to intercept (query string and fragment
are stripped before matching)
- file (required): Absolute path to local file to serve
- content_type (optional): Override Content-Type header (defaults to
guessing from filename)
- status_code (optional): HTTP status code (defaults to 200)
This is incredibly useful for debugging production websites: you can
intercept any script, stylesheet, or other resource and replace it with
a local copy containing your own debug instrumentation, console.log
statements, or experimental fixes - all without modifying the actual
site or setting up a local dev server.
Add support for WPT test variants, which allow a single test file to be
run multiple times with different URL query parameters. Tests declare
variants using `<meta name="variant" content="?param=value">` tags.
When test-web encounters a test with variants, it expands that test into
multiple runs, each with its own expectation file using the naming
convention `testname@variant.txt` (e.g., `test@run_type=uri.txt`).
Implementation details:
- WebContent observes variant meta tags and communicates them to the
test runner via a new `did_receive_test_variant_metadata` IPC call
- test-web dynamically expands tests with variants during execution,
waking idle views after each test completion to pick up new work
- Use index-based test tracking to avoid dangling references when the
test vector grows during variant expansion
- Introduce TestRunContext to group test run state, and store a static
pointer to it for signal handler access
This enables proper testing of WPT tests that use variants, such as the
html5lib parsing tests (which test uri, write, and write_single modes)
and the editing/bold tests (which split across multiple ranges).
To avoid unnecessary IPC traffic, we now only send network response
bodies when a DevTools client is connected.
This requires tracking DevTools connection state in ViewImplementation
so we can propagate it to new WebContent processes created during
cross-site navigation.
Propagate the request initiator type (e.g., "xmlhttprequest", "fetch",
"script", "stylesheet") from LibWeb through the IPC layer to DevTools.
This enables Firefox DevTools to correctly identify XHR/fetch requests
and display appropriate cause types in the Network panel's "Initiator"
column.
This adds support for viewing request payloads (POST data) and response
bodies in the Firefox DevTools network panel.
Request bodies are captured when network requests start and passed
through IPC to the NetworkEventActor, which returns them via the
getRequestPostData protocol method.
Response bodies are streamed via a new IPC message as data is received,
accumulated in NetworkEventActor (with a 10MB size limit to prevent
memory issues), and returned via getResponseContent. Text content is
returned as UTF-8, while binary content (images, etc.) is base64.
Previously, console messages were sent using an index-based system where
DevTools would be notified of new message indices and then request them
in batches. This created synchronization issues during page navigation
when the WebContent process resets while DevTools still has stale index
state.
This changes to a push-based model where console messages are sent
immediately as resources when they are logged, matching how Firefox
DevTools handles console messages. Each message is pushed through IPC
and forwarded to DevTools as a "console-message" or "error-message"
resource.
This eliminates the need for index tracking in FrameActor and simplifies
the entire console message pipeline from WebContent through to DevTools.
When a page navigates, send document-event resources with
"will-navigate" and tabNavigated messages so Firefox DevTools
can follow along and clear the Network panel appropriately.
Hook ResourceLoader to emit network request lifecycle events through
IPC to the UI process, where FrameActor creates NetworkEventActor
instances that serialize requests using Firefox's Remote Debug Protocol.
The Network panel now shows requests with method, URL, status, MIME
type, size, and timing information. Several features remain stubbed
(POST data, response content, cause detection) marked with FIXMEs.
When multiple views share a WebContent process (e.g. parent and child
views created via window.open()), we need to notify ALL of them when
the process crashes, not just one.
Previously, each view would overwrite the single crash callback on
WebContentClient, so only the last view to initialize would be notified.
This adds WebContentClient::notify_all_views_of_crash() which iterates
over all registered views and notifies each one. Child views also now
propagate crashes to their parent, and can be disconnected between
tests to prevent stale crashes from affecting subsequent tests.
This commit makes several improvements to crash handling for headless
mode (used by test-web and other automated tools):
1. Always respawn WebContent after crashes, ignoring the crash count
limit. The limit is meant for interactive use to prevent infinite
crash loops; in headless mode, each test needs a working WebContent.
2. Skip the error page when respawning, as there's no UI to display it.
3. Suppress crash log messages that would corrupt live terminal output.
These changes allow test-web to properly recover from WebContent crashes
and continue running subsequent tests.
When a child process exits, the SIGCHLD handler was destroying the
Process object synchronously within the signal dispatch context. This
triggered Process::~Process() which called m_connection->shutdown(),
attempting to join the IPC IO thread.
Joining threads from within signal handler context can cause deadlocks,
as observed when WebContent crashes during test-web runs.
Fix this by deferring the on_process_exited callback using
Core::deferred_invoke(), ensuring the Process destruction happens in
normal event loop context.
- Add --results-dir CLI flag to specify output directory
- Default to {tempdir}/test-web-results if not specified
- Capture stdout/stderr from all helper processes (WebContent,
RequestServer, ImageDecoder) to prevent output spam
- Save captured output to per-test files in results directory
- Save test diffs (expected vs actual) to results directory
- Generate HTML index of failed tests with links to diffs
- Display live-updating concurrent test status with progress bar
- Defer warning messages until after test run completes
Once upon a time, we needed the UI-specific event loops outside of the
UI process. This is no longer the case. Let's move the event loops back
to the UI folder to remove the awkward interface library we were left
with.
The DeferredInvocationContext only existed to satisfy the requirement
in ThreadEventQueue that each event has an EventReceiver. However,
deferred_invoke() was not even using the EventReceiver to call its
callback. Therefore, we don't need to allocate one for every deferred
invocation.
This also prevents WeakPtr::strong_ref() from racing and leaking the
context object when invoking a function across threads.
This enables the disk cache and flips the related command line flag to
allow disabling it. We've reached a point where it works well, so this
will let us get more mileage on it.
SocketAddressWindows.h contains (re)definitions of a bunch of system-
level types. This leads to a bunch of conflicts when trying to include
proper Windows headers in SystemWindows.cpp. This patch removes this
inclusion and just forward-declares the couple of types we need.
The end goal here is for LibHTTP to be the home of our RFC 9111 (HTTP
caching) implementation. We currently have one implementation in LibWeb
for our in-memory cache and another in RequestServer for our disk cache.
The implementations both largely revolve around interacting with HTTP
headers. But in LibWeb, we are using Fetch's header infra, and in RS we
are using are home-grown header infra from LibHTTP.
So to give these a common denominator, this patch replaces the LibHTTP
implementation with Fetch's infra. Our existing LibHTTP implementation
was not particularly compliant with any spec, so this at least gives us
a standards-based common implementation.
This migration also required moving a handful of other Fetch AOs over
to LibHTTP. (It turns out these AOs were all from the Fetch/Infra/HTTP
folder, so perhaps it makes sense for LibHTTP to be the implementation
of that entire set of facilities.)
In 11b8bbe one thing that was claimed was that we now properly set the
Notifier's actual fd on the NotifierActivationEvent. It turns out that
claim was false because a crucial step was forgotten: actually set the
m_notifier_fd when registering. Despite that mistake, it ultimately was
irrelevant as the methods on NotifierActivationEvent are currently
unused code. We were posting the event to the correct Notifier receiver
so the on_activation was still getting invoked.
Given they are unused, NotifierActivationEvent can be defined the same
way as TimerEvent is, where we just pass the event type enum to the
Event base class. Additionally, NotificationType can be moved to
the Notifier header as this enum is now always used in the context of
creating or using a Notifier instance.
This mode allows us to test the HTTP disk cache with two mechanisms:
1. If RequestServer is launched with --http-disk-cache-mode=testing, it
will cache requests with a X-Ladybird-Enable-Disk-Cache header.
2. In test mode, RS will include a X-Ladybird-Disk-Cache-Status response
header indicating how the response was handled by the cache. There is
no standard way for a web request to know what happened with respect
to the disk cache, so this fills that hole for testing.
This mode is not exposed to users.
