The preload list is static, immutable data compiled into LibHTTP.
ResourceLoader consults it in-process at the fetch layer before
falling back to the dynamic, per-profile store in the browser
process, so a preloaded host is upgraded without a synchronous IPC.
HSTSStore stays the dynamic store only; the preload list cannot be
unset by a max-age=0 response because it is consulted first, before
the store is ever queried.
Add internals.setHSTSPolicy, internals.ingestHSTSHeader, and
internals.isKnownHSTSHost so tests can drive HSTS state without an
HTTPS server. Extract ResourceLoader::try_store_hsts_policy_for_url
so the ingest helper exercises the same code path the network layer
uses for Strict-Transport-Security response headers.
When an HTTPS response carries a Strict-Transport-Security header, the
received policy is now respected. Subsequent HTTP requests to a known
HSTS host are upgraded to HTTPS before the fetch algorithm makes
further decisions such as CORS and mixed content.
Fixes tpexpress.co.uk, where an XHR redirects HTTPS -> HTTP -> HTTPS,
relying on a HSTS policy received on the document response to avoid the
CORS failure.
Carry the initiating URL, fetch destination, initiator type, and request
mode from Fetch into LoadRequest and pass them to ContentBlocker. The
existing substring matcher still matches only the request URL, but the
blocker API now receives the context needed by source-aware backends.
Preserve that metadata for CORS preflights and connection hints, and add
coverage for resource-type mapping and blob source URL normalization.
Rename the local content blocking implementation and its tests from
ContentFilter to ContentBlocker while keeping the existing substring
matcher backend and behavior.
Update the WebContent IPC method, WebView option names, debug toggle,
and default config file name to use content blocker terminology.
Let CacheEntryWriter expose an explicit body-file handoff after a
successful flush. RequestServer sends that file to WebContent before the
request finishes, so Fetch can replace its retained memory-cache buffer
with a file-backed ImmutableBytes body.
Keep the plain flush path fd-free for existing callers, and add coverage
for mapping the body file returned by the writer. The focused disk and
memory cache web tests continue to pass.
Carry response chunks through LibRequests and ResourceLoader as a
ResponseData wrapper. A disk cache hit can retain its mapped storage,
while ordinary streamed chunks still use borrowed bytes.
Store completed memory cache entries as Core::ImmutableBytes. This lets
mapped disk-cache responses stay file-backed while retained by the HTTP
memory cache, instead of forcing another ByteBuffer copy.
Attach cached JavaScript bytecode sidecars to HTTP response headers so
WebContent can materialize classic and module scripts directly from a
decoded cache blob on cache hits.
Carry the disk cache vary key with the sidecar and reuse it when storing
fresh bytecode, avoiding mismatches against the augmented network
request headers used to create the cache entry.
Keep CORS-filtered module responses intact for status, MIME, and script
creation checks. Read bytecode sidecar data only from the internal
response, and treat decode or materialization failure as a cache miss
that falls back to normal source compilation.
Passing the browser command line and executable path to every WebContent
process just in case we load about:version always felt a bit weird. We
now use the WebUI framework to load this information on demand.
Instead of passing RequestServer and ImageDecoder socket FDs as
command-line arguments to WebContent, send them over the main IPC
channel after launch. This unifies initial connection and reconnection
into a single code path.
We currently attach HTTP cookie headers from LibWeb within Fetch. This
has the downside that the cookie IPC, and the infrastructure around it,
are all synchronous. This blocks the WebContent process entirely while
the cookie is being retrieved, for every request on a page.
We now attach cookie headers from RequestServer. The state machine in
RequestServer::Request allows us to easily do this work asynchronously.
We can also skip this work entirely when the response is served from
disk cache.
Note that we will continue to parse cookies in the WebContent process.
If something goes awry during parsing. we limit the damage to that
process, instead of the UI or RequestServer.
Also note that WebSocket requests still have cookie headers attached
attached from LibWeb. This will be handled in a future patch.
In the future, we may want to introduce a memory cache for cookies in
RequestServer to avoid IPC altogether as able.
The cookie RFC strongly suggests that cookies only contain ASCII, and
that non-ASCII values be encoded with e.g. base64. Web reality differs,
however, and browsers are expected to support UTF-8 encoded cookies.
This aligns with document.cookie.
This fixes the following WPT tests:
/cookies/encoding/charset.html
/cookiestore/encoding.https.any.html
No test added here because we don't have a mechanism yet to set an
HTTP cookie and inspect it via our file:// URL test infra.
It's possible for the cookie value from a Set-Cookie header to contain
invalid UTF-8. We must isomorphic decode this header.
This fixes the /cookies/domain/domain-attribute-idn-host.sub.https.html
WPT test. The test added here is a crash test rather than a text test
because we cannot access the received Set-Cookie header from JS on the
file:// test URL.
If the cache mode is no-store, we must not interact with the cache at
all.
If the cache mode is reload, we must not use any cached response.
If the cache-mode is only-if-cached or force-cache, we are permitted
to respond with stale cache responses.
Note that we currently cannot test only-if-cached in test-web. Setting
this mode also requires setting the cors mode to same-origin, but our
http-test-server infra requires setting the cors mode to cors.
Propagate the request initiator type (e.g., "xmlhttprequest", "fetch",
"script", "stylesheet") from LibWeb through the IPC layer to DevTools.
This enables Firefox DevTools to correctly identify XHR/fetch requests
and display appropriate cause types in the Network panel's "Initiator"
column.
This adds support for viewing request payloads (POST data) and response
bodies in the Firefox DevTools network panel.
Request bodies are captured when network requests start and passed
through IPC to the NetworkEventActor, which returns them via the
getRequestPostData protocol method.
Response bodies are streamed via a new IPC message as data is received,
accumulated in NetworkEventActor (with a 10MB size limit to prevent
memory issues), and returned via getResponseContent. Text content is
returned as UTF-8, while binary content (images, etc.) is base64.
Hook ResourceLoader to emit network request lifecycle events through
IPC to the UI process, where FrameActor creates NetworkEventActor
instances that serialize requests using Firefox's Remote Debug Protocol.
The Network panel now shows requests with method, URL, status, MIME
type, size, and timing information. Several features remain stubbed
(POST data, response content, cause detection) marked with FIXMEs.
The end goal here is for LibHTTP to be the home of our RFC 9111 (HTTP
caching) implementation. We currently have one implementation in LibWeb
for our in-memory cache and another in RequestServer for our disk cache.
The implementations both largely revolve around interacting with HTTP
headers. But in LibWeb, we are using Fetch's header infra, and in RS we
are using are home-grown header infra from LibHTTP.
So to give these a common denominator, this patch replaces the LibHTTP
implementation with Fetch's infra. Our existing LibHTTP implementation
was not particularly compliant with any spec, so this at least gives us
a standards-based common implementation.
This migration also required moving a handful of other Fetch AOs over
to LibHTTP. (It turns out these AOs were all from the Fetch/Infra/HTTP
folder, so perhaps it makes sense for LibHTTP to be the implementation
of that entire set of facilities.)
The only thing in HTTPResponse being used is reason_phrase_for_code,
which is just a static helper method. Move it to its own file and remove
HTTPResponse.
This is just one less thing to have to port to an upcoming HTTP header
refactor.
Disallow calling `StringBase::bytes()` on temporaries to avoid returning
`ReadonlyBytes` that outlive the underlying string.
With this change, we catch a real UAF:
`load_result.data = maybe_response.release_value().bytes();`
All other updated call sites were already safe, they just needed to use
an intermediate named variable to satisfy the new lvalue-only
requirement.
Previously, unbuffered requests were only available as a special mode
for EventSource. With this change, they are enabled by default, which
means chunks can be read from the stream as soon as they arrive.
This unlocks some interesting possibilities, such as starting to parse
HTML documents before the entire response has been received (that, in
turn, allows us to initiate subresource fetches earlier or begin
executing scripts sooner), or start rendering videos before they are
fully downloaded.
Co-authored-by: Timothy Flynn <trflynn89@pm.me>
JavaScript module requests (in a non-worker context) always have CORS
enabled. However, CORS requests are only allowed for same-origin or
HTTP/S requests. This patch extends this to allow resource:// requests
from opaque origins (e.g. about: URLs).
We must also set the Access-Control-Allow-Origin header to "null" to
ensure that the response is accepted by the CORS checks. This does not
affect requesting resource:// URLs from resource:// URLs as those are
same-origin and skip CORS checks.
This ultimately enables requesting resource:// JS modules from the
about:settings page.
This commit:
- Prevents path traversal via the about: scheme
- Prevents loading about:inspector
- Requires about: URIs to be opaque paths
- Prevents crashes with invalid percent encoded paths
We only need a Page for file:// urls. At some point we probably
needed it for other kinds of requests, but the current functionality
doesn't need to store the Page pointer on the ResourceLoader.
Resulting in a massive rename across almost everywhere! Alongside the
namespace change, we now have the following names:
* JS::NonnullGCPtr -> GC::Ref
* JS::GCPtr -> GC::Ptr
* JS::HeapFunction -> GC::Function
* JS::CellImpl -> GC::Cell
* JS::Handle -> GC::Root
