Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2026-04-19 02:10:26 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Libraries/LibWeb/Fetch/Infrastructure/HTTP/Bodies.cpp

226 lines
8.8 KiB
C++
Raw Permalink Normal View History

LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
/*
LibWeb/Fetch: Port infrastructure to new String
2023-03-02 23:26:35 +00:00
* Copyright (c) 2022-2023, Linus Groh <linusg@serenityos.org>
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
* Copyright (c) 2024, Jelle Raaijmakers <jelle@ladybird.org>
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
*
* SPDX-License-Identifier: BSD-2-Clause
*/
LibJS: Move PromiseCapability into its own cpp/h file This is not strictly connected to PromiseReaction in any way. Preparation before doing some actual work on it :^)
2022-10-02 10:59:22 +01:00
#include <LibJS/Runtime/PromiseCapability.h>
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
#include <LibWeb/Bindings/ExceptionOrUtils.h>
LibWeb: Stub out Fetch::Infrastructure::Body::clone()
2022-09-25 19:25:53 +01:00
#include <LibWeb/Bindings/MainThreadVM.h>
LibWeb: Implement 'Byte sequence as body' AO
2022-10-30 14:39:32 +00:00
#include <LibWeb/Fetch/BodyInit.h>
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
#include <LibWeb/Fetch/Infrastructure/HTTP/Bodies.h>
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
#include <LibWeb/Fetch/Infrastructure/IncrementalReadLoopReadRequest.h>
LibWeb/Fetch: Implement Body's "fully read" function from the spec Required by XHR's reliance on Fetch.
2023-02-28 18:12:44 +00:00
#include <LibWeb/Fetch/Infrastructure/Task.h>
LibWeb: Implement a slightly better ad-hoc Body::clone method Just creating a stream on the JS heap isn't enough, as we will later crash when trying to read from that stream as it hasn't been properly initialized. Instead, until we have teeing implemented (which is a rather huge part of the Streams spec), create streams using proper AOs that do initialize the stream.
2024-01-27 09:16:09 -05:00
#include <LibWeb/HTML/Scripting/TemporaryExecutionContext.h>
LibWeb: Move ReadableStream AOs into their own file The main streams AO file has gotten very large, and is a bit difficult to navigate. In an effort to improve DX, this migrates ReadableStream AOs to their own file. And the helper classes used for the tee and pipe- to operations are also in their own files.
2025-04-17 15:47:53 -04:00
#include <LibWeb/Streams/ReadableStream.h>
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
LibWeb: Move Fetch infra into the Web::Fetch::Infrastructure namespace The Fetch spec unfortunately will cause a name clash between the Request concept and the Request JS object - both cannot live in the Web::Fetch namespace, and WrapperGenerator generally assumes `Web::<Name>` for things living in the `<Name>/` subdirectory, so let's instead move infra code into its own namespace - it already sits in a (sub-)subdirectory anyway.
2022-07-17 23:52:02 +01:00
namespace Web::Fetch::Infrastructure {
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC_DEFINE_ALLOCATOR(Body);
LibWeb: Put most LibWeb GC objects in type-specific heap blocks With this change, we now have ~1200 CellAllocators across both LibJS and LibWeb in a normal WebContent instance. This gives us a minimum heap size of 4.7 MiB in the scenario where we only have one cell allocated per type. Of course, in practice there will be many more of each type, so the effective overhead is quite a bit smaller than that in practice. I left a few types unconverted to this mechanism because I got tired of doing this. :^)
2023-11-19 19:47:52 +01:00
LibWeb: Support MIME type sniffing for streaming HTTP responses Previously, when loading a document, we would try to sniff the MIME type by reading from the response body's source. However, for streaming HTTP responses, the body source is Empty (the data comes through the stream instead), so we had no bytes to sniff. This caused pages like hypr.land (which sends no Content-Type header) to be misidentified as plain text instead of HTML, since the MIME sniffing algorithm would receive zero bytes and fall back to the default type. The fix captures the first bytes of the response body during fetch, storing them on the Body object. These bytes are the "resource header" defined by the MIME Sniffing spec - up to 1445 bytes, which is enough to identify any MIME type the spec can detect. Since bytes may arrive asynchronously during streaming, we use a callback mechanism: if bytes aren't ready yet when load_document() needs them, it registers a callback that fires once enough bytes have been captured (or the stream ends). The flow is: 1. FetchedDataReceiver receives network bytes, buffers them 2. When Body is created, buffered bytes are flushed to Body's sniff buffer, and subsequent bytes are appended as they arrive 3. Before calling load_document(), Navigable waits for sniff bytes 4. load_document() passes the bytes to MimeSniff::Resource::sniff()
2026-01-24 12:25:01 +01:00
// https://mimesniff.spec.whatwg.org/#reading-the-resource-header
// To read the resource header, a user agent MUST read bytes of the resource until one of the following conditions is met:
// - the end of the resource is reached
// - 1445 or more bytes have been read
static constexpr size_t MAX_SNIFF_BYTES = 1445;
LibWeb/Fetch: Store Body Blob as Ref not Root To avoid some churn, Body::source() returns the SourceTypeInternal object, as it's not exposed to JS. The constructor is also adjusted so that Body::clone() doesn't have to convert to a SourceType and then immediately back again.
2026-02-12 16:11:55 +00:00
static Body::SourceTypeInternal to_source_type_internal(Body::SourceType&& source_type)
{
return source_type.visit(
[](Empty) -> Body::SourceTypeInternal { return Empty {}; },
[](ByteBuffer& buffer) -> Body::SourceTypeInternal { return move(buffer); },
[](GC::Root<FileAPI::Blob> const& blob) -> Body::SourceTypeInternal { return GC::Ref { *blob }; });
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ref<Body> Body::create(JS::VM& vm, GC::Ref<Streams::ReadableStream> stream)
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
{
LibJS+LibWeb: Rename Heap::allocate_without_realm to Heap::allocate Now that the heap has no knowledge about a JavaScript realm and is purely for managing the memory of the heap, it does not make sense to name this function to say that it is a non-realm variant.
2024-11-14 06:13:46 +13:00
return vm.heap().allocate<Body>(stream);
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ref<Body> Body::create(JS::VM& vm, GC::Ref<Streams::ReadableStream> stream, SourceType source, Optional<u64> length)
LibWeb/Fetch: Store Body Blob as Ref not Root To avoid some churn, Body::source() returns the SourceTypeInternal object, as it's not exposed to JS. The constructor is also adjusted so that Body::clone() doesn't have to convert to a SourceType and then immediately back again.
2026-02-12 16:11:55 +00:00
{
return create(vm, stream, to_source_type_internal(move(source)), length);
}
GC::Ref<Body> Body::create(JS::VM& vm, GC::Ref<Streams::ReadableStream> stream, SourceTypeInternal source, Optional<u64> length)
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
{
LibJS+LibWeb: Rename Heap::allocate_without_realm to Heap::allocate Now that the heap has no knowledge about a JavaScript realm and is purely for managing the memory of the heap, it does not make sense to name this function to say that it is a non-realm variant.
2024-11-14 06:13:46 +13:00
return vm.heap().allocate<Body>(stream, source, length);
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
Body::Body(GC::Ref<Streams::ReadableStream> stream)
LibWeb: Move ReadableStream AOs into their own file The main streams AO file has gotten very large, and is a bit difficult to navigate. In an effort to improve DX, this migrates ReadableStream AOs to their own file. And the helper classes used for the tee and pipe- to operations are also in their own files.
2025-04-17 15:47:53 -04:00
: m_stream(stream)
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
{
}
LibWeb/Fetch: Store Body Blob as Ref not Root To avoid some churn, Body::source() returns the SourceTypeInternal object, as it's not exposed to JS. The constructor is also adjusted so that Body::clone() doesn't have to convert to a SourceType and then immediately back again.
2026-02-12 16:11:55 +00:00
Body::Body(GC::Ref<Streams::ReadableStream> stream, SourceTypeInternal source, Optional<u64> length)
LibWeb: Move ReadableStream AOs into their own file The main streams AO file has gotten very large, and is a bit difficult to navigate. In an effort to improve DX, this migrates ReadableStream AOs to their own file. And the helper classes used for the tee and pipe- to operations are also in their own files.
2025-04-17 15:47:53 -04:00
: m_stream(stream)
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
, m_source(move(source))
, m_length(move(length))
{
}
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
void Body::visit_edges(Cell::Visitor& visitor)
{
Base::visit_edges(visitor);
visitor.visit(m_stream);
LibWeb: Support MIME type sniffing for streaming HTTP responses Previously, when loading a document, we would try to sniff the MIME type by reading from the response body's source. However, for streaming HTTP responses, the body source is Empty (the data comes through the stream instead), so we had no bytes to sniff. This caused pages like hypr.land (which sends no Content-Type header) to be misidentified as plain text instead of HTML, since the MIME sniffing algorithm would receive zero bytes and fall back to the default type. The fix captures the first bytes of the response body during fetch, storing them on the Body object. These bytes are the "resource header" defined by the MIME Sniffing spec - up to 1445 bytes, which is enough to identify any MIME type the spec can detect. Since bytes may arrive asynchronously during streaming, we use a callback mechanism: if bytes aren't ready yet when load_document() needs them, it registers a callback that fires once enough bytes have been captured (or the stream ends). The flow is: 1. FetchedDataReceiver receives network bytes, buffers them 2. When Body is created, buffered bytes are flushed to Body's sniff buffer, and subsequent bytes are appended as they arrive 3. Before calling load_document(), Navigable waits for sniff bytes 4. load_document() passes the bytes to MimeSniff::Resource::sniff()
2026-01-24 12:25:01 +01:00
visitor.visit(m_sniff_bytes_callback);
LibWeb/Fetch: Store Body Blob as Ref not Root To avoid some churn, Body::source() returns the SourceTypeInternal object, as it's not exposed to JS. The constructor is also adjusted so that Body::clone() doesn't have to convert to a SourceType and then immediately back again.
2026-02-12 16:11:55 +00:00
m_source.visit(
[&](GC::Ref<FileAPI::Blob> const& blob) { visitor.visit(blob); },
[](auto const&) {});
LibWeb: Support MIME type sniffing for streaming HTTP responses Previously, when loading a document, we would try to sniff the MIME type by reading from the response body's source. However, for streaming HTTP responses, the body source is Empty (the data comes through the stream instead), so we had no bytes to sniff. This caused pages like hypr.land (which sends no Content-Type header) to be misidentified as plain text instead of HTML, since the MIME sniffing algorithm would receive zero bytes and fall back to the default type. The fix captures the first bytes of the response body during fetch, storing them on the Body object. These bytes are the "resource header" defined by the MIME Sniffing spec - up to 1445 bytes, which is enough to identify any MIME type the spec can detect. Since bytes may arrive asynchronously during streaming, we use a callback mechanism: if bytes aren't ready yet when load_document() needs them, it registers a callback that fires once enough bytes have been captured (or the stream ends). The flow is: 1. FetchedDataReceiver receives network bytes, buffers them 2. When Body is created, buffered bytes are flushed to Body's sniff buffer, and subsequent bytes are appended as they arrive 3. Before calling load_document(), Navigable waits for sniff bytes 4. load_document() passes the bytes to MimeSniff::Resource::sniff()
2026-01-24 12:25:01 +01:00
}
void Body::append_sniff_bytes(ReadonlyBytes bytes)
{
if (m_sniff_bytes_complete)
return;
size_t space_remaining = MAX_SNIFF_BYTES - m_sniff_bytes.size();
if (space_remaining == 0) {
set_sniff_bytes_complete();
return;
}
size_t to_append = min(bytes.size(), space_remaining);
m_sniff_bytes.append(bytes.slice(0, to_append));
if (m_sniff_bytes.size() >= MAX_SNIFF_BYTES)
set_sniff_bytes_complete();
}
void Body::set_sniff_bytes_complete()
{
if (m_sniff_bytes_complete)
return;
m_sniff_bytes_complete = true;
if (m_sniff_bytes_callback) {
auto callback = exchange(m_sniff_bytes_callback, nullptr);
callback->function()(m_sniff_bytes);
}
}
Optional<ReadonlyBytes> Body::sniff_bytes_if_available() const
{
// Non-streaming body: source has bytes
if (m_source.has<ByteBuffer>()) {
auto const& buffer = m_source.get<ByteBuffer>();
return buffer.bytes().slice(0, min(buffer.size(), MAX_SNIFF_BYTES));
}
LibWeb/Fetch: Store Body Blob as Ref not Root To avoid some churn, Body::source() returns the SourceTypeInternal object, as it's not exposed to JS. The constructor is also adjusted so that Body::clone() doesn't have to convert to a SourceType and then immediately back again.
2026-02-12 16:11:55 +00:00
if (m_source.has<GC::Ref<FileAPI::Blob>>()) {
auto raw = m_source.get<GC::Ref<FileAPI::Blob>>()->raw_bytes();
LibWeb: Support MIME type sniffing for streaming HTTP responses Previously, when loading a document, we would try to sniff the MIME type by reading from the response body's source. However, for streaming HTTP responses, the body source is Empty (the data comes through the stream instead), so we had no bytes to sniff. This caused pages like hypr.land (which sends no Content-Type header) to be misidentified as plain text instead of HTML, since the MIME sniffing algorithm would receive zero bytes and fall back to the default type. The fix captures the first bytes of the response body during fetch, storing them on the Body object. These bytes are the "resource header" defined by the MIME Sniffing spec - up to 1445 bytes, which is enough to identify any MIME type the spec can detect. Since bytes may arrive asynchronously during streaming, we use a callback mechanism: if bytes aren't ready yet when load_document() needs them, it registers a callback that fires once enough bytes have been captured (or the stream ends). The flow is: 1. FetchedDataReceiver receives network bytes, buffers them 2. When Body is created, buffered bytes are flushed to Body's sniff buffer, and subsequent bytes are appended as they arrive 3. Before calling load_document(), Navigable waits for sniff bytes 4. load_document() passes the bytes to MimeSniff::Resource::sniff()
2026-01-24 12:25:01 +01:00
return raw.slice(0, min(raw.size(), MAX_SNIFF_BYTES));
}
// Streaming body: bytes captured during fetch
if (m_sniff_bytes_complete)
return m_sniff_bytes;
// Still waiting for bytes
return {};
}
void Body::wait_for_sniff_bytes(SniffBytesCallback on_ready)
{
if (auto bytes = sniff_bytes_if_available(); bytes.has_value()) {
on_ready->function()(bytes.value());
return;
}
// Wait for bytes to arrive
m_sniff_bytes_callback = on_ready;
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
}
LibWeb: Stub out Fetch::Infrastructure::Body::clone()
2022-09-25 19:25:53 +01:00
// https://fetch.spec.whatwg.org/#concept-body-clone
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ref<Body> Body::clone(JS::Realm& realm)
LibWeb: Stub out Fetch::Infrastructure::Body::clone()
2022-09-25 19:25:53 +01:00
{
LibWeb: Make TemporaryExecutionContext take a Realm&
2024-10-24 20:39:18 +13:00
HTML::TemporaryExecutionContext execution_context { realm, HTML::TemporaryExecutionContext::CallbacksEnabled::Yes };
LibWeb: Implement a slightly better ad-hoc Body::clone method Just creating a stream on the JS heap isn't enough, as we will later crash when trying to read from that stream as it hasn't been properly initialized. Instead, until we have teeing implemented (which is a rather huge part of the Streams spec), create streams using proper AOs that do initialize the stream.
2024-01-27 09:16:09 -05:00
LibWeb: Stub out Fetch::Infrastructure::Body::clone()
2022-09-25 19:25:53 +01:00
// To clone a body body, run these steps:
LibWeb: Use the stream teeing methods for cloning Fetch response bodies
2024-01-28 14:21:32 -05:00
// 1. Let « out1, out2 » be the result of teeing body’s stream.
LibWeb: Use the correct target realm to tee a stream We currently store Web::Fetch::Infrastructure::Response objects in the HTTP cache. They are associated with their original realm, but when we use a cached response, we clone it into the target realm. For example, two <iframe> objects loading the same HTML will be in different realms. When we clone the response, we must use the target realm throughout the entire cloning process. We neglected to do this for the cloned response body stream, which is cloned via teeing. The result was the the stream for the "cloned" response was created in the original realm, causing issues down the line when reading from that stream tried to handle read promises on behalf of the original realm. There are protections in place to prevent this from happening, and the cached response read would never complete.
2025-04-30 07:31:15 -04:00
auto [out1, out2] = m_stream->tee(&realm).release_value_but_fixme_should_propagate_errors();
LibWeb: Implement a slightly better ad-hoc Body::clone method Just creating a stream on the JS heap isn't enough, as we will later crash when trying to read from that stream as it hasn't been properly initialized. Instead, until we have teeing implemented (which is a rather huge part of the Streams spec), create streams using proper AOs that do initialize the stream.
2024-01-27 09:16:09 -05:00
LibWeb: Use the stream teeing methods for cloning Fetch response bodies
2024-01-28 14:21:32 -05:00
// 2. Set body’s stream to out1.
m_stream = out1;
LibWeb: Stub out Fetch::Infrastructure::Body::clone()
2022-09-25 19:25:53 +01:00
// 3. Return a body whose stream is out2 and other members are copied from body.
LibWeb: Implement a slightly better ad-hoc Body::clone method Just creating a stream on the JS heap isn't enough, as we will later crash when trying to read from that stream as it hasn't been properly initialized. Instead, until we have teeing implemented (which is a rather huge part of the Streams spec), create streams using proper AOs that do initialize the stream.
2024-01-27 09:16:09 -05:00
return Body::create(realm.vm(), *out2, m_source, m_length);
LibWeb: Stub out Fetch::Infrastructure::Body::clone()
2022-09-25 19:25:53 +01:00
}
LibWeb/Fetch: Implement Body's "fully read" function from the spec Required by XHR's reliance on Fetch.
2023-02-28 18:12:44 +00:00
// https://fetch.spec.whatwg.org/#body-fully-read
LibWeb: Propagate OOM in Body::fully_read() through its error callback Fetched bodies can be on the order of gigabytes, so rather than crashing when we hit OOM here, we can simply invoke the error callback with a DOM exception. We use "UnknownError" here as the spec directly supports this for OOM errors: UnknownError: The operation failed for an unknown transient reason (e.g. out of memory). This is still an ad-hoc implementation. We should be using streams, and we do have the AOs available to do so. But they need to be massaged to be compatible with callers of Body::fully_read. And once we do use streams, this function will become infallible - so making it infallible here is at least a step in the right direction.
2024-04-26 14:57:40 -04:00
void Body::fully_read(JS::Realm& realm, Web::Fetch::Infrastructure::Body::ProcessBodyCallback process_body, Web::Fetch::Infrastructure::Body::ProcessBodyErrorCallback process_body_error, TaskDestination task_destination) const
LibWeb: Stub out Fetch::Infrastructure::Body::fully_read_as_promise()
2022-09-25 19:27:02 +01:00
{
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
HTML::TemporaryExecutionContext execution_context { realm, HTML::TemporaryExecutionContext::CallbacksEnabled::Yes };
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
// 1. If taskDestination is null, then set taskDestination to the result of starting a new parallel queue.
if (task_destination.has<Empty>())
task_destination = HTML::ParallelQueue::create();
LibWeb/Fetch: Implement Body's "fully read" function from the spec Required by XHR's reliance on Fetch.
2023-02-28 18:12:44 +00:00
// 2. Let successSteps given a byte sequence bytes be to queue a fetch task to run processBody given bytes, with taskDestination.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
auto success_steps = [&realm, process_body, task_destination](ByteBuffer bytes) {
queue_fetch_task(task_destination, GC::create_function(realm.heap(), [process_body, bytes = move(bytes)]() mutable {
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
process_body->function()(move(bytes));
LibWeb: Let queue_fetch_task() take a JS::HeapFunction Changes the signature of queue_fetch_task() from AK:Function to JS::HeapFunction to be more clear to the user of the function that this is what it uses internally.
2024-04-19 10:23:40 +02:00
}));
LibWeb/Fetch: Implement Body's "fully read" function from the spec Required by XHR's reliance on Fetch.
2023-02-28 18:12:44 +00:00
};
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
// 3. Let errorSteps optionally given an exception exception be to queue a fetch task to run processBodyError given
// exception, with taskDestination.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
auto error_steps = [&realm, process_body_error, task_destination](JS::Value exception) {
queue_fetch_task(task_destination, GC::create_function(realm.heap(), [process_body_error, exception]() {
LibWeb: Change Fetch's ProcessBodyError to accept a plain JS value This callback is meant to be triggered by streams, which does not always provide a WebIDL::DOMException. Pass a plain value instead. Of all the users of this callback, only one actually uses the value, and already converts the DOMException to a plain value.
2024-04-30 07:20:41 -04:00
process_body_error->function()(exception);
LibWeb: Let queue_fetch_task() take a JS::HeapFunction Changes the signature of queue_fetch_task() from AK:Function to JS::HeapFunction to be more clear to the user of the function that this is what it uses internally.
2024-04-19 10:23:40 +02:00
}));
LibWeb/Fetch: Implement Body's "fully read" function from the spec Required by XHR's reliance on Fetch.
2023-02-28 18:12:44 +00:00
};
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
// 4. Let reader be the result of getting a reader for body’s stream. If that threw an exception, then run errorSteps
// with that exception and return.
LibWeb: Add a 'get a reader' helper method on ReadableStream
2024-12-24 12:38:25 +13:00
auto reader = m_stream->get_a_reader();
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
if (reader.is_exception()) {
LibWeb: Remove `dom_` from `dom_exception_to_throw_completion` We're not converting `WebIDL::DOMException`, but `WebIDL::Exception` instead.
2024-11-04 14:37:27 +01:00
auto throw_completion = Bindings::exception_to_throw_completion(realm.vm(), reader.release_error());
LibJS: Make Completion.[[Value]] non-optional Instead, just use js_undefined() whenever the [[Value]] field is unused. This avoids a whole bunch of presence checks.
2025-04-04 18:11:45 +02:00
error_steps(throw_completion.release_value());
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
return;
}
LibWeb/Fetch: Implement Body's "fully read" function from the spec Required by XHR's reliance on Fetch.
2023-02-28 18:12:44 +00:00
// 5. Read all bytes from reader, given successSteps and errorSteps.
LibWeb: Implement Web::Fetch::Body::fully_read() closer to spec By actually using streams, they get marked as disturbed and the `.bodyUsed` API starts to work. Fixes at least 94 subtests in the WPT `fetch/api/request` test suite. Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-11-04 14:11:07 +01:00
reader.value()->read_all_bytes(GC::create_function(realm.heap(), move(success_steps)), GC::create_function(realm.heap(), move(error_steps)));
LibWeb: Stub out Fetch::Infrastructure::Body::fully_read_as_promise()
2022-09-25 19:27:02 +01:00
}
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
// https://fetch.spec.whatwg.org/#body-incrementally-read
void Body::incrementally_read(ProcessBodyChunkCallback process_body_chunk, ProcessEndOfBodyCallback process_end_of_body, ProcessBodyErrorCallback process_body_error, TaskDestination task_destination)
{
LibWeb: Make TemporaryExecutionContext take a Realm&
2024-10-24 20:39:18 +13:00
HTML::TemporaryExecutionContext const execution_context { m_stream->realm(), HTML::TemporaryExecutionContext::CallbacksEnabled::Yes };
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
// 1. If taskDestination is null, then set taskDestination to the result of starting a new parallel queue.
if (task_destination.has<Empty>())
task_destination = HTML::ParallelQueue::create();
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
// 2. Let reader be the result of getting a reader for body’s stream.
// NOTE: This operation will not throw an exception.
LibWeb: Add a 'get a reader' helper method on ReadableStream
2024-12-24 12:38:25 +13:00
auto reader = MUST(m_stream->get_a_reader());
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
// 3. Perform the incrementally-read loop given reader, taskDestination, processBodyChunk, processEndOfBody, and processBodyError.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
VERIFY(!task_destination.has<Empty>());
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
incrementally_read_loop(reader, task_destination.get<GC::Ref<JS::Object>>(), process_body_chunk, process_end_of_body, process_body_error);
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
}
// https://fetch.spec.whatwg.org/#incrementally-read-loop
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
void Body::incrementally_read_loop(Streams::ReadableStreamDefaultReader& reader, TaskDestination task_destination, ProcessBodyChunkCallback process_body_chunk, ProcessEndOfBodyCallback process_end_of_body, ProcessBodyErrorCallback process_body_error)
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
{
auto& realm = reader.realm();
// 1. Let readRequest be the following read request:
LibJS+LibWeb: Use realm.create<T> instead of heap.allocate<T> The main motivation behind this is to remove JS specifics of the Realm from the implementation of the Heap. As a side effect of this change, this is a bit nicer to read than the previous approach, and in my opinion, also makes it a little more clear that this method is specific to a JavaScript Realm.
2024-11-14 05:50:17 +13:00
auto read_request = realm.create<IncrementalReadLoopReadRequest>(*this, reader, task_destination, process_body_chunk, process_end_of_body, process_body_error);
LibWeb: Implement the concept incrementally read a body
2024-05-17 22:06:27 +02:00
// 2. Read a chunk from reader given readRequest.
reader.read_a_chunk(read_request);
}
LibWeb: Implement 'Byte sequence as body' AO
2022-10-30 14:39:32 +00:00
// https://fetch.spec.whatwg.org/#byte-sequence-as-a-body
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
GC::Ref<Body> byte_sequence_as_body(JS::Realm& realm, ReadonlyBytes bytes)
LibWeb: Implement 'Byte sequence as body' AO
2022-10-30 14:39:32 +00:00
{
// To get a byte sequence bytes as a body, return the body of the result of safely extracting bytes.
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
auto [body, _] = safely_extract_body(realm, bytes);
LibWeb: Implement 'Byte sequence as body' AO
2022-10-30 14:39:32 +00:00
return body;
}
LibWeb: Add definitions from '2.2.4. Bodies' in the Fetch spec ...at least the ones that don't require an implementation of the Streams spec :^)
2022-07-12 18:04:24 +01:00
}
Reference in a new issue Copy permalink