Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-10-19 15:43:20 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Libraries/LibWeb/Fetch/Fetching/Fetching.cpp

2849 lines
164 KiB
C++
Raw Normal View History

LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
/*
LibWeb/Fetch: Port infrastructure to new String
2023-03-02 23:26:35 +00:00
* Copyright (c) 2022-2023, Linus Groh <linusg@serenityos.org>
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
* Copyright (c) 2023, Luke Wilde <lukew@serenityos.org>
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
* Copyright (c) 2023, Sam Atkins <atkinssj@serenityos.org>
LibWeb/Fetch: Implement the "set the Sec-Fetch-Dest header" AO
2024-04-29 20:54:14 +01:00
* Copyright (c) 2024, Jamie Mansfield <jmansfield@cadixdev.org>
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
* Copyright (c) 2025, Shannon Booth <shannon@serenityos.org>
LibWeb: Implement AO populate request from client
2025-08-06 21:27:17 +02:00
* Copyright (c) 2025, Kenneth Myhra <kennethmyhra@serenityos.org>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#include <AK/Base64.h>
#include <AK/Debug.h>
#include <AK/ScopeGuard.h>
LibJS+LibWeb: Move the macro to convert ENOMEM to an exception to LibJS Move the macro to LibJS and change it to return a throw completion instead of a WebIDL exception. This will let us use this macro within LibJS to handle OOM conditions.
2023-01-07 12:14:54 -05:00
#include <LibJS/Runtime/Completion.h>
RequestServer: Retrieve timing info from curl and pipe it to LibWeb This timing info will be used to create a PerformanceResourceTiming entry.
2025-02-26 13:28:21 +00:00
#include <LibRequests/RequestTimingInfo.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Bindings/MainThreadVM.h>
LibWeb: Move host_defined_intrinsics out of line
2024-11-08 04:03:55 +13:00
#include <LibWeb/Bindings/PrincipalHostDefined.h>
LibWeb: Enforce Content Security Policy on Fetch requests
2024-11-25 17:29:27 +00:00
#include <LibWeb/ContentSecurityPolicy/BlockingAlgorithms.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Cookie/Cookie.h>
#include <LibWeb/DOM/Document.h>
LibWeb: Rename URL platform object to DOMURL Along with putting functions in the URL namespace into a DOMURL namespace. This is done as LibWeb is in an awkward situation where it needs two URL classes. AK::URL is the general purpose URL class which is all that is needed in 95% of cases. URL in the Web namespace is needed predominantly for interfacing with the javascript interfaces. Because of two URLs in the same namespace, AK::URL has had to be used throughout LibWeb. If we move AK::URL into a URL namespace, this becomes more painful - where ::URL::URL is required to specify the constructor (and something like ::URL::create_with_url_or_path in other places). To fix this problem - rename the class in LibWeb implementing the URL IDL interface to DOMURL, along with moving the other Web URL related classes into this DOMURL folder. One could argue that this name also makes the situation a little more clear in LibWeb for why these two URL classes need be used in the first place.
2024-02-11 19:48:56 +13:00
#include <LibWeb/DOMURL/DOMURL.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Fetch/BodyInit.h>
#include <LibWeb/Fetch/Fetching/Checks.h>
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
#include <LibWeb/Fetch/Fetching/FetchedDataReceiver.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Fetch/Fetching/Fetching.h>
#include <LibWeb/Fetch/Fetching/PendingResponse.h>
#include <LibWeb/Fetch/Fetching/RefCountedFlag.h>
#include <LibWeb/Fetch/Infrastructure/FetchAlgorithms.h>
#include <LibWeb/Fetch/Infrastructure/FetchController.h>
#include <LibWeb/Fetch/Infrastructure/FetchParams.h>
LibWeb: Conform HTTP-network-or-cache to specs Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the fetch specs.
2024-07-18 21:31:42 +01:00
#include <LibWeb/Fetch/Infrastructure/FetchRecord.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Fetch/Infrastructure/FetchTimingInfo.h>
LibWeb/Fetch: Expose a minimised Content-Type to Resource Timing See: - https://github.com/whatwg/fetch/commit/931cd06
2024-05-19 13:23:38 +01:00
#include <LibWeb/Fetch/Infrastructure/HTTP/Headers.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Fetch/Infrastructure/HTTP/Methods.h>
#include <LibWeb/Fetch/Infrastructure/HTTP/Requests.h>
#include <LibWeb/Fetch/Infrastructure/HTTP/Responses.h>
#include <LibWeb/Fetch/Infrastructure/HTTP/Statuses.h>
#include <LibWeb/Fetch/Infrastructure/MimeTypeBlocking.h>
LibWeb: Move NetworkPartitionKey into a header This is used elsewhere, such as the HTMLLinkElement.
2024-05-31 18:44:49 +01:00
#include <LibWeb/Fetch/Infrastructure/NetworkPartitionKey.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Fetch/Infrastructure/NoSniffBlocking.h>
#include <LibWeb/Fetch/Infrastructure/PortBlocking.h>
#include <LibWeb/Fetch/Infrastructure/Task.h>
#include <LibWeb/Fetch/Infrastructure/URL.h>
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
#include <LibWeb/FileAPI/Blob.h>
#include <LibWeb/FileAPI/BlobURLStore.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/HTML/EventLoop/EventLoop.h>
#include <LibWeb/HTML/Scripting/Environments.h>
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
#include <LibWeb/HTML/Scripting/TemporaryExecutionContext.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/HTML/Window.h>
LibWeb: Ensure a Web::Page is associated with local Worker LoadRequests This is a hack on top of a hack because Workers don't *really* need to have a Web::Page at all, but the ResourceLoader infra that should be going away soon :tm: is not quite ready to axe that requirement for cookies.
2023-11-09 18:09:21 -07:00
#include <LibWeb/HTML/WorkerGlobalScope.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/HighResolutionTime/TimeOrigin.h>
#include <LibWeb/Loader/LoadRequest.h>
#include <LibWeb/Loader/ResourceLoader.h>
LibWeb: Implement should block mixed content request
2024-05-29 19:01:06 +01:00
#include <LibWeb/MixedContent/AbstractOperations.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/Platform/EventLoopPlugin.h>
#include <LibWeb/ReferrerPolicy/AbstractOperations.h>
LibWeb: Implement Resource Timing
2025-02-26 15:51:05 +00:00
#include <LibWeb/ResourceTiming/PerformanceResourceTiming.h>
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
#include <LibWeb/SRI/SRI.h>
LibWeb/Fetch: Implement the "set the Sec-Fetch-Dest header" AO
2024-04-29 20:54:14 +01:00
#include <LibWeb/SecureContexts/AbstractOperations.h>
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
#include <LibWeb/Streams/TransformStream.h>
#include <LibWeb/Streams/TransformStreamDefaultController.h>
LibWeb: Move TransformStream AOs into their own file The main streams AO file has gotten very large, and is a bit difficult to navigate. In an effort to improve DX, this migrates TransformStream AOs to their own file.
2025-04-17 20:02:23 -04:00
#include <LibWeb/Streams/TransformStreamOperations.h>
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
#include <LibWeb/Streams/Transformer.h>
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
#include <LibWeb/WebIDL/DOMException.h>
namespace Web::Fetch::Fetching {
WebContent: Enable in Windows CI
2025-06-28 03:32:38 -07:00
bool g_http_cache_enabled = false;
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
Everywhere: Fully qualify IsLvalueReference in TRY() macros If USING_AK_GLOBALLY is not defined, the name IsLvalueReference might not be available in the global namespace. Follow the pattern established in LibTest to fully qualify AK types in macros to avoid this problem.
2023-01-14 16:34:44 -07:00
#define TRY_OR_IGNORE(expression) \
({ \
AK+Everywhere: Do not implicitly copy variables in TRY macros For example, consider cases where we want to propagate errors only in specific instances: auto result = read_data(); // something like ErrorOr<ByteBuffer> if (result.is_error() && result.error().code() != EINTR) continue; auto bytes = TRY(result); The TRY invocation will currently copy the byte buffer when the expression (in this case, just a local variable) is stored into _temporary_result. This patch binds the expression to a reference to prevent such copies. In less trival invocations (such as TRY(some_function()), this will incur only temporary lifetime extensions, i.e. no functional change.
2023-02-09 13:27:43 -05:00
auto&& _temporary_result = (expression); \
Everywhere: Fully qualify IsLvalueReference in TRY() macros If USING_AK_GLOBALLY is not defined, the name IsLvalueReference might not be available in the global namespace. Follow the pattern established in LibTest to fully qualify AK types in macros to avoid this problem.
2023-01-14 16:34:44 -07:00
if (_temporary_result.is_error()) \
return; \
static_assert(!::AK::Detail::IsLvalueReference<decltype(_temporary_result.release_value())>, \
"Do not return a reference from a fallible expression"); \
_temporary_result.release_value(); \
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
})
// https://fetch.spec.whatwg.org/#concept-fetch
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ref<Infrastructure::FetchController>> fetch(JS::Realm& realm, Infrastructure::Request& request, Infrastructure::FetchAlgorithms const& algorithms, UseParallelQueue use_parallel_queue)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'fetch' with: request @ {}", &request);
auto& vm = realm.vm();
// 1. Assert: request’s mode is "navigate" or processEarlyHintsResponse is null.
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
VERIFY(request.mode() == Infrastructure::Request::Mode::Navigate || !algorithms.process_early_hints_response());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Let taskDestination be null.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
Infrastructure::TaskDestination task_destination;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. Let crossOriginIsolatedCapability be false.
auto cross_origin_isolated_capability = HTML::CanUseCrossOriginIsolatedAPIs::No;
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 4. Populate request from client given request.
populate_request_from_client(realm, request);
// 5. If request’s client is non-null, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (request.client() != nullptr) {
// 1. Set taskDestination to request’s client’s global object.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
task_destination = GC::Ref { request.client()->global_object() };
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Set crossOriginIsolatedCapability to request’s client’s cross-origin isolated capability.
cross_origin_isolated_capability = request.client()->cross_origin_isolated_capability();
}
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 6. If useParallelQueue is true, then set taskDestination to the result of starting a new parallel queue.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
if (use_parallel_queue == UseParallelQueue::Yes)
task_destination = HTML::ParallelQueue::create();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 7. Let timingInfo be a new fetch timing info whose start time and post-redirect start time are the coarsened
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// shared current time given crossOriginIsolatedCapability, and render-blocking is set to request’s
// render-blocking.
auto timing_info = Infrastructure::FetchTimingInfo::create(vm);
auto now = HighResolutionTime::coarsened_shared_current_time(cross_origin_isolated_capability == HTML::CanUseCrossOriginIsolatedAPIs::Yes);
timing_info->set_start_time(now);
timing_info->set_post_redirect_start_time(now);
timing_info->set_render_blocking(request.render_blocking());
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 8. Let fetchParams be a new fetch params whose request is request, timing info is timingInfo, process request
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// body chunk length is processRequestBodyChunkLength, process request end-of-body is processRequestEndOfBody,
// process early hints response is processEarlyHintsResponse, process response is processResponse, process
// response consume body is processResponseConsumeBody, process response end-of-body is processResponseEndOfBody,
// task destination is taskDestination, and cross-origin isolated capability is crossOriginIsolatedCapability.
auto fetch_params = Infrastructure::FetchParams::create(vm, request, timing_info);
LibWeb: Fix a few const-ness issues
2023-02-25 10:44:51 -07:00
fetch_params->set_algorithms(algorithms);
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
fetch_params->set_task_destination(task_destination);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
fetch_params->set_cross_origin_isolated_capability(cross_origin_isolated_capability);
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 9. If request’s body is a byte sequence, then set request’s body to request’s body as a body.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (auto const* buffer = request.body().get_pointer<ByteBuffer>())
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
request.set_body(Infrastructure::byte_sequence_as_body(realm, buffer->bytes()));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 10. If all of the following conditions are true:
LibWeb/Fetch: Add missing fetch step for preloaded resources This is currently no-op and a FIXME exists to implement the "consume a preloaded resource" AO.
2024-07-12 19:35:06 +01:00
if (
// - request’s URL’s scheme is an HTTP(S) scheme
Infrastructure::is_http_or_https_scheme(request.url().scheme())
// - request’s mode is "same-origin", "cors", or "no-cors"
&& (request.mode() == Infrastructure::Request::Mode::SameOrigin || request.mode() == Infrastructure::Request::Mode::CORS || request.mode() == Infrastructure::Request::Mode::NoCORS)
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// - request’s client is not null, and request’s client’s global object is a Window object
&& request.client() && is<HTML::Window>(request.client()->global_object())
LibWeb/Fetch: Add missing fetch step for preloaded resources This is currently no-op and a FIXME exists to implement the "consume a preloaded resource" AO.
2024-07-12 19:35:06 +01:00
// - request’s method is `GET`
&& StringView { request.method() }.equals_ignoring_ascii_case("GET"sv)
// - request’s unsafe-request flag is not set or request’s header list is empty
&& (!request.unsafe_request() || request.header_list()->is_empty())) {
// 1. Assert: request’s origin is same origin with request’s client’s origin.
LibWeb+LibURL: Move HTML::Origin to URL::Origin While Origin is defined in the HTML spec - this leaves us with quite an awkward relationship as the URL spec makes use of AO's from what is defined in the HTML spec. To simplify this factoring, relocate Origin into LibURL.
2024-10-05 15:33:34 +13:00
VERIFY(request.origin().has<URL::Origin>() && request.origin().get<URL::Origin>().is_same_origin(request.client()->origin()));
LibWeb/Fetch: Add missing fetch step for preloaded resources This is currently no-op and a FIXME exists to implement the "consume a preloaded resource" AO.
2024-07-12 19:35:06 +01:00
// 2. Let onPreloadedResponseAvailable be an algorithm that runs the following step given a response
// response: set fetchParams’s preloaded response candidate to response.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto on_preloaded_response_available = GC::create_function(realm.heap(), [fetch_params](GC::Ref<Infrastructure::Response> response) {
LibWeb/Fetch: Add missing fetch step for preloaded resources This is currently no-op and a FIXME exists to implement the "consume a preloaded resource" AO.
2024-07-12 19:35:06 +01:00
fetch_params->set_preloaded_response_candidate(response);
});
// FIXME: 3. Let foundPreloadedResource be the result of invoking consume a preloaded resource for request’s
// window, given request’s URL, request’s destination, request’s mode, request’s credentials mode,
// request’s integrity metadata, and onPreloadedResponseAvailable.
auto found_preloaded_resource = false;
(void)on_preloaded_response_available;
// 4. If foundPreloadedResource is true and fetchParams’s preloaded response candidate is null, then set
// fetchParams’s preloaded response candidate to "pending".
if (found_preloaded_resource && fetch_params->preloaded_response_candidate().has<Empty>())
fetch_params->set_preloaded_response_candidate(Infrastructure::FetchParams::PreloadedResponseCandidatePendingTag {});
}
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 11. If request’s header list does not contain `Accept`, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!request.header_list()->contains("Accept"sv.bytes())) {
// 1. Let value be `*/*`.
auto value = "*/*"sv;
LibWeb/Fetch: Use correct header for prefetch requests See: - https://github.com/whatwg/fetch/commit/a5560d2
2024-07-12 20:01:10 +01:00
// 2. If request’s initiator is "prefetch", then set value to the document `Accept` header value.
if (request.initiator() == Infrastructure::Request::Initiator::Prefetch) {
value = document_accept_header_value;
}
// 3. Otherwise, the user agent should set value to the first matching statement, if any, switching on request’s destination:
else if (request.destination().has_value()) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
switch (*request.destination()) {
// -> "document"
// -> "frame"
// -> "iframe"
case Infrastructure::Request::Destination::Document:
case Infrastructure::Request::Destination::Frame:
case Infrastructure::Request::Destination::IFrame:
LibWeb/Fetch: Use correct header for prefetch requests See: - https://github.com/whatwg/fetch/commit/a5560d2
2024-07-12 20:01:10 +01:00
// the document `Accept` header value
value = document_accept_header_value;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
break;
// -> "image"
case Infrastructure::Request::Destination::Image:
// `image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5`
value = "image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5"sv;
break;
LibWeb/Fetch: Use "json" destination See: - https://github.com/SerenityOS/serenity/commit/da8d0d8 - https://github.com/whatwg/fetch/commit/49bff76 - https://github.com/whatwg/html/commit/37659e9
2024-05-03 20:44:06 +01:00
// -> "json"
case Infrastructure::Request::Destination::JSON:
// `application/json,*/*;q=0.5`
value = "application/json,*/*;q=0.5"sv;
break;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> "style"
case Infrastructure::Request::Destination::Style:
// `text/css,*/*;q=0.1`
value = "text/css,*/*;q=0.1"sv;
break;
default:
break;
}
}
LibWeb/Fetch: Use correct header for prefetch requests See: - https://github.com/whatwg/fetch/commit/a5560d2
2024-07-12 20:01:10 +01:00
// 4. Append (`Accept`, value) to request’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Accept"sv, value.bytes());
request.header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 12. If request’s header list does not contain `Accept-Language`, then user agents should append
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// (`Accept-Language, an appropriate header value) to request’s header list.
if (!request.header_list()->contains("Accept-Language"sv.bytes())) {
LibWeb: Make preferred languages configurable This also changes fetch to use the preferred languages for the Accept-Language header.
2024-07-23 21:10:24 +01:00
StringBuilder accept_language;
accept_language.join(","sv, ResourceLoader::the().preferred_languages());
auto header = Infrastructure::Header::from_string_pair("Accept-Language"sv, accept_language.string_view());
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
request.header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 13. If request’s internal priority is null, then use request’s priority, initiator, destination, and
// render-blocking in an implementation-defined manner to set request’s internal priority to an
// implementation-defined object.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: The user-agent-defined object could encompass stream weight and dependency for HTTP/2, and equivalent
// information used to prioritize dispatch and processing of HTTP/1 fetches.
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 14. If request is a subresource request, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (request.is_subresource_request()) {
LibWeb: Append fetch record to client's fetch group when request is a ...subresource.
2024-07-22 20:46:39 +02:00
// 1. Let record be a new fetch record whose request is request and controller is fetchParams’s controller.
auto record = Infrastructure::FetchRecord::create(vm, request, fetch_params->controller());
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 2. Append record to request’s client’s fetch group’s fetch records.
LibWeb: Append fetch record to client's fetch group when request is a ...subresource.
2024-07-22 20:46:39 +02:00
request.client()->fetch_group().append(record);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 15. Run main fetch given fetchParams.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
(void)TRY(main_fetch(realm, fetch_params));
LibWeb: Align Fetching chapter's "To fetch" with latest spec changes
2025-08-07 11:41:58 +02:00
// 16. Return fetchParams’s controller.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return fetch_params->controller();
}
LibWeb: Implement AO populate request from client
2025-08-06 21:27:17 +02:00
// https://fetch.spec.whatwg.org/#populate-request-from-client
void populate_request_from_client(JS::Realm const& realm, Infrastructure::Request& request)
{
auto& heap = realm.heap();
// 1. If request’s traversable for user prompts is "client":
auto const* traversable_for_user_prompts = request.traversable_for_user_prompts().get_pointer<Infrastructure::Request::TraversableForUserPrompts>();
if (traversable_for_user_prompts && *traversable_for_user_prompts == Infrastructure::Request::TraversableForUserPrompts::Client) {
// 1. Set request’s traversable for user prompts to "no-traversable".
request.set_traversable_for_user_prompts(Infrastructure::Request::TraversableForUserPrompts::NoTraversable);
// 2. If request’s client is non-null:
if (request.client()) {
// 1. Let global be request’s client’s global object.
auto& global = request.client()->global_object();
// 2. If global is a Window object and global’s navigable is not null, then set request’s traversable for
// user prompts to global’s navigable’s traversable navigable.
if (auto const* window = as_if<HTML::Window>(global)) {
if (window->navigable())
request.set_traversable_for_user_prompts(window->navigable()->traversable_navigable());
}
}
}
// 2. If request’s origin is "client":
auto const* origin = request.origin().get_pointer<Infrastructure::Request::Origin>();
if (origin && *origin == Infrastructure::Request::Origin::Client) {
// 1. Assert: request’s client is non-null.
VERIFY(request.client());
// 2. Set request’s origin to request’s client’s origin.
request.set_origin(request.client()->origin());
}
// 3. If request’s policy container is "client":
auto const* policy_container = request.policy_container().get_pointer<Infrastructure::Request::PolicyContainer>();
if (policy_container && *policy_container == Infrastructure::Request::PolicyContainer::Client) {
// 1. If request’s client is non-null, then set request’s policy container to a clone of request’s client’s
// policy container.
if (request.client())
request.set_policy_container(request.client()->policy_container()->clone(heap));
// 2. Otherwise, set request’s policy container to a new policy container.
else
request.set_policy_container(heap.allocate<HTML::PolicyContainer>(heap));
}
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// https://fetch.spec.whatwg.org/#concept-main-fetch
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ptr<PendingResponse>> main_fetch(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params, Recursive recursive)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'main fetch' with: fetch_params @ {}", &fetch_params);
auto& vm = realm.vm();
// 1. Let request be fetchParams’s request.
auto request = fetch_params.request();
// 2. Let response be null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<Infrastructure::Response> response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. If request’s local-URLs-only flag is set and request’s current URL is not local, then set response to a
// network error.
if (request->local_urls_only() && !Infrastructure::is_local_url(request->current_url()))
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
response = Infrastructure::Response::network_error(vm, "Request with 'local-URLs-only' flag must have a local URL"_string);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Report CSP violations for request
2024-11-28 11:57:04 +00:00
// 4. Run report Content Security Policy violations for request.
ContentSecurityPolicy::report_content_security_policy_violations_for_request(realm, request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// FIXME: 5. Upgrade request to a potentially trustworthy URL, if appropriate.
LibWeb: Upgrade mixed requests to potentially trustworthy URLs (if appropriate)
2024-05-29 19:27:27 +01:00
// 6. Upgrade a mixed content request to a potentially trustworthy URL, if appropriate.
MixedContent::upgrade_a_mixed_content_request_to_a_potentially_trustworthy_url_if_appropriate(request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Enforce Integrity Policy on Fetch requests
2025-08-07 22:41:54 +02:00
// 7. If should request be blocked due to a bad port, should fetching request be blocked as mixed content, should
// request be blocked by Content Security Policy, or should request be blocked by Integrity Policy Policy
// returns blocked, then set response to a network error.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (Infrastructure::block_bad_port(request) == Infrastructure::RequestOrResponseBlocking::Blocked
LibWeb: Implement should block mixed content request
2024-05-29 19:01:06 +01:00
|| MixedContent::should_fetching_request_be_blocked_as_mixed_content(request) == Infrastructure::RequestOrResponseBlocking::Blocked
LibWeb: Enforce Integrity Policy on Fetch requests
2025-08-07 22:41:54 +02:00
|| ContentSecurityPolicy::should_request_be_blocked_by_content_security_policy(realm, request) == ContentSecurityPolicy::Directives::Directive::Result::Blocked
|| ContentSecurityPolicy::should_request_be_blocked_by_integrity_policy(request) == ContentSecurityPolicy::Directives::Directive::Result::Blocked) {
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
response = Infrastructure::Response::network_error(vm, "Request was blocked"_string);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 8. If request’s referrer policy is the empty string, then set request’s referrer policy to request’s policy
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// container’s referrer policy.
LibWeb: Consistently use the EmptyString state of ReferrerPolicy We previously used an empty optional to denote that a ReferrerPolicy is in the default empty string state. However, later additions added an explicit EmptyString state. This patch moves all users to the explicit state, and stops using `Optional<ReferrerPolicy>` everywhere except for when an option not being passed from JavaScript has meaning.
2024-03-05 09:35:25 -07:00
if (request->referrer_policy() == ReferrerPolicy::ReferrerPolicy::EmptyString) {
LibWeb: Make PolicyContainer GC allocated This is required to store Content Security Policies, as their Directives are implemented as subclasses with overridden virtual functions. Thus, they cannot be stored as generic Directive classes, as it'll lose the ability to call overridden functions when they are copied.
2024-11-25 14:30:12 +00:00
VERIFY(request->policy_container().has<GC::Ref<HTML::PolicyContainer>>());
request->set_referrer_policy(request->policy_container().get<GC::Ref<HTML::PolicyContainer>>()->referrer_policy);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 9. If request’s referrer is not "no-referrer", then set request’s referrer to the result of invoking determine
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// request’s referrer.
// NOTE: As stated in Referrer Policy, user agents can provide the end user with options to override request’s
// referrer to "no-referrer" or have it expose less sensitive information.
auto const* referrer = request->referrer().get_pointer<Infrastructure::Request::Referrer>();
if (!referrer || *referrer != Infrastructure::Request::Referrer::NoReferrer) {
auto determined_referrer = ReferrerPolicy::determine_requests_referrer(request);
if (determined_referrer.has_value())
request->set_referrer(*determined_referrer);
else
request->set_referrer(Infrastructure::Request::Referrer::NoReferrer);
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 10. Set request’s current URL’s scheme to "https" if all of the following conditions are true:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (
// - request’s current URL’s scheme is "http"
request->current_url().scheme() == "http"sv
// - request’s current URL’s host is a domain
LibURL: Promote Host to a proper class This lets us move a few Host-related functions (like serialization and checks for what the Host is) into Host instead of having them dotted around the codebase. For now, the interface is still very Variant-like, to avoid having to change quite so much in one go.
2024-11-27 15:12:17 +00:00
&& request->current_url().host().has_value() && request->current_url().host()->is_domain()
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// FIXME: - Matching request’s current URL’s host per Known HSTS Host Domain Name Matching results in either a
// superdomain match with an asserted includeSubDomains directive or a congruent match (with or without an
// asserted includeSubDomains directive) [HSTS]; or DNS resolution for the request finds a matching HTTPS RR
// per section 9.5 of [SVCB].
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
&& false) {
AK: Port URL scheme from DeprecatedString to String
2023-08-12 16:52:41 +12:00
request->current_url().set_scheme("https"_string);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto get_response = GC::create_function(vm.heap(), [&realm, &vm, &fetch_params, request]() -> WebIDL::ExceptionOr<GC::Ref<PendingResponse>> {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'main fetch' get_response() function");
LibWeb: Allow CORS requests from opaque origins to resource:// URLs JavaScript module requests (in a non-worker context) always have CORS enabled. However, CORS requests are only allowed for same-origin or HTTP/S requests. This patch extends this to allow resource:// requests from opaque origins (e.g. about: URLs). We must also set the Access-Control-Allow-Origin header to "null" to ensure that the response is accepted by the CORS checks. This does not affect requesting resource:// URLs from resource:// URLs as those are same-origin and skip CORS checks. This ultimately enables requesting resource:// JS modules from the about:settings page.
2025-04-23 11:14:15 -04:00
auto const* origin = request->origin().get_pointer<URL::Origin>();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> fetchParams’s preloaded response candidate is not null
if (!fetch_params.preloaded_response_candidate().has<Empty>()) {
// 1. Wait until fetchParams’s preloaded response candidate is not "pending".
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
HTML::main_thread_event_loop().spin_until(GC::create_function(vm.heap(), [&] {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return !fetch_params.preloaded_response_candidate().has<Infrastructure::FetchParams::PreloadedResponseCandidatePendingTag>();
LibWeb: Port EventLoop::spin_XXX to HeapFunction
2024-10-31 05:23:43 +13:00
}));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Assert: fetchParams’s preloaded response candidate is a response.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
VERIFY(fetch_params.preloaded_response_candidate().has<GC::Ref<Infrastructure::Response>>());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. Return fetchParams’s preloaded response candidate.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
return PendingResponse::create(vm, request, fetch_params.preloaded_response_candidate().get<GC::Ref<Infrastructure::Response>>());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
// -> request’s current URL’s origin is same origin with request’s origin, and request’s response tainting is "basic"
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> request’s current URL’s scheme is "data"
// -> request’s mode is "navigate" or "websocket"
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
if (
LibWeb: Allow CORS requests from opaque origins to resource:// URLs JavaScript module requests (in a non-worker context) always have CORS enabled. However, CORS requests are only allowed for same-origin or HTTP/S requests. This patch extends this to allow resource:// requests from opaque origins (e.g. about: URLs). We must also set the Access-Control-Allow-Origin header to "null" to ensure that the response is accepted by the CORS checks. This does not affect requesting resource:// URLs from resource:// URLs as those are same-origin and skip CORS checks. This ultimately enables requesting resource:// JS modules from the about:settings page.
2025-04-23 11:14:15 -04:00
(origin && request->current_url().origin().is_same_origin(*origin) && request->response_tainting() == Infrastructure::Request::ResponseTainting::Basic)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
|| request->current_url().scheme() == "data"sv
|| (request->mode() == Infrastructure::Request::Mode::Navigate || request->mode() == Infrastructure::Request::Mode::WebSocket)) {
// 1. Set request’s response tainting to "basic".
request->set_response_tainting(Infrastructure::Request::ResponseTainting::Basic);
// 2. Return the result of running scheme fetch given fetchParams.
return scheme_fetch(realm, fetch_params);
// NOTE: HTML assigns any documents and workers created from URLs whose scheme is "data" a unique
// opaque origin. Service workers can only be created from URLs whose scheme is an HTTP(S) scheme.
}
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> request’s mode is "same-origin"
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
if (request->mode() == Infrastructure::Request::Mode::SameOrigin) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// Return a network error.
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'same-origin' mode must have same URL and request origin"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> request’s mode is "no-cors"
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
if (request->mode() == Infrastructure::Request::Mode::NoCORS) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. If request’s redirect mode is not "follow", then return a network error.
if (request->redirect_mode() != Infrastructure::Request::RedirectMode::Follow)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'no-cors' mode must have redirect mode set to 'follow'"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Set request’s response tainting to "opaque".
request->set_response_tainting(Infrastructure::Request::ResponseTainting::Opaque);
// 3. Return the result of running scheme fetch given fetchParams.
return scheme_fetch(realm, fetch_params);
}
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> request’s current URL’s scheme is not an HTTP(S) scheme
LibWeb: Allow CORS requests from opaque origins to resource:// URLs JavaScript module requests (in a non-worker context) always have CORS enabled. However, CORS requests are only allowed for same-origin or HTTP/S requests. This patch extends this to allow resource:// requests from opaque origins (e.g. about: URLs). We must also set the Access-Control-Allow-Origin header to "null" to ensure that the response is accepted by the CORS checks. This does not affect requesting resource:// URLs from resource:// URLs as those are same-origin and skip CORS checks. This ultimately enables requesting resource:// JS modules from the about:settings page.
2025-04-23 11:14:15 -04:00
// AD-HOC: We allow CORS requests for resource:// URLs from opaque origins to enable requesting JS modules from internal pages.
if (!Infrastructure::is_http_or_https_scheme(request->current_url().scheme())
&& !(origin && origin->is_opaque() && request->current_url().scheme() == "resource"sv)) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: At this point all other request modes have been handled. Ensure we're not lying in the error message :^)
VERIFY(request->mode() == Infrastructure::Request::Mode::CORS);
// Return a network error.
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'cors' mode must have URL with HTTP or HTTPS scheme"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> request’s use-CORS-preflight flag is set
// -> request’s unsafe-request flag is set and either request’s method is not a CORS-safelisted method or
// CORS-unsafe request-header names with request’s header list is not empty
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
if (
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
request->use_cors_preflight()
|| (request->unsafe_request()
&& (!Infrastructure::is_cors_safelisted_method(request->method())
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
|| !Infrastructure::get_cors_unsafe_header_names(request->header_list()).is_empty()))) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. Set request’s response tainting to "cors".
request->set_response_tainting(Infrastructure::Request::ResponseTainting::CORS);
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
auto returned_pending_response = PendingResponse::create(vm, request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Let corsWithPreflightResponse be the result of running HTTP fetch given fetchParams and true.
auto cors_with_preflight_response = TRY(http_fetch(realm, fetch_params, MakeCORSPreflight::Yes));
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
cors_with_preflight_response->when_loaded([returned_pending_response](GC::Ref<Infrastructure::Response> cors_with_preflight_response) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'main fetch' cors_with_preflight_response load callback");
// 3. If corsWithPreflightResponse is a network error, then clear cache entries using request.
if (cors_with_preflight_response->is_network_error()) {
// FIXME: Clear cache entries
}
// 4. Return corsWithPreflightResponse.
returned_pending_response->resolve(cors_with_preflight_response);
});
return returned_pending_response;
}
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> Otherwise
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
// 1. Set request’s response tainting to "cors".
request->set_response_tainting(Infrastructure::Request::ResponseTainting::CORS);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Make the `main fetch` response callback a bit easier to read Each `if` branch in this callback returns a value, so let's add a bit of whitespace between them to be easier on the eyes.
2025-04-23 11:07:04 -04:00
// 2. Return the result of running HTTP fetch given fetchParams.
return http_fetch(realm, fetch_params);
LibWeb: Port EventLoop::spin_XXX to HeapFunction
2024-10-31 05:23:43 +13:00
});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (recursive == Recursive::Yes) {
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 12. If response is null, then set response to the result of running the steps corresponding to the first
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// matching statement:
LibWeb/Fetch: Actually check if the response is null in recursive fetch The condition for checking if there was already a response in recursive fetch was accidentally flipped, always causing a null deref. This made redirects crash for example.
2023-03-07 18:32:07 +00:00
auto pending_response = !response
LibWeb: Port EventLoop::spin_XXX to HeapFunction
2024-10-31 05:23:43 +13:00
? TRY(get_response->function()())
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
: PendingResponse::create(vm, request, *response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 13. If recursive is true, then return response.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return pending_response;
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 11. If recursive is false, then run the remaining steps in parallel.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
Platform::EventLoopPlugin::the().deferred_invoke(GC::create_function(realm.heap(), [&realm, &vm, &fetch_params, request, response, get_response] {
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 12. If response is null, then set response to the result of running the steps corresponding to the first
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// matching statement:
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
auto pending_response = PendingResponse::create(vm, request, Infrastructure::Response::create(vm));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!response) {
LibWeb: Port EventLoop::spin_XXX to HeapFunction
2024-10-31 05:23:43 +13:00
auto pending_response_or_error = get_response->function()();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (pending_response_or_error.is_error())
return;
pending_response = pending_response_or_error.release_value();
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
pending_response->when_loaded([&realm, &vm, &fetch_params, request, response, response_was_null = !response](GC::Ref<Infrastructure::Response> resolved_response) mutable {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'main fetch' pending_response load callback");
if (response_was_null)
response = resolved_response;
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 14. If response is not a network error and response is not a filtered response, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!response->is_network_error() && !is<Infrastructure::FilteredResponse>(*response)) {
// 1. If request’s response tainting is "cors", then:
if (request->response_tainting() == Infrastructure::Request::ResponseTainting::CORS) {
LibWeb: Support for Access-Control-Expose-Headers in Fetch This adds the headers named in Access-Control-Expose-Headers to the response's CORS-exposed header-name list which allows those headers to be accessed from JS.
2023-08-01 21:40:30 +12:00
// 1. Let headerNames be the result of extracting header list values given
// `Access-Control-Expose-Headers` and response’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header_names_or_failure = Infrastructure::extract_header_list_values("Access-Control-Expose-Headers"sv.bytes(), response->header_list());
LibWeb: Support for Access-Control-Expose-Headers in Fetch This adds the headers named in Access-Control-Expose-Headers to the response's CORS-exposed header-name list which allows those headers to be accessed from JS.
2023-08-01 21:40:30 +12:00
auto header_names = header_names_or_failure.has<Vector<ByteBuffer>>() ? header_names_or_failure.get<Vector<ByteBuffer>>() : Vector<ByteBuffer> {};
// 2. If request’s credentials mode is not "include" and headerNames contains `*`, then set
// response’s CORS-exposed header-name list to all unique header names in response’s header
// list.
if (request->credentials_mode() != Infrastructure::Request::CredentialsMode::Include && header_names.contains_slow("*"sv.bytes())) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto unique_header_names = response->header_list()->unique_names();
LibWeb: Support for Access-Control-Expose-Headers in Fetch This adds the headers named in Access-Control-Expose-Headers to the response's CORS-exposed header-name list which allows those headers to be accessed from JS.
2023-08-01 21:40:30 +12:00
response->set_cors_exposed_header_name_list(move(unique_header_names));
}
// 3. Otherwise, if headerNames is not null or failure, then set response’s CORS-exposed
// header-name list to headerNames.
else if (!header_names.is_empty()) {
response->set_cors_exposed_header_name_list(move(header_names));
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 2. Set response to the following filtered response with response as its internal response, depending
// on request’s response tainting:
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
response = [&]() -> GC::Ref<Infrastructure::Response> {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
switch (request->response_tainting()) {
// -> "basic"
case Infrastructure::Request::ResponseTainting::Basic:
// basic filtered response
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Responses
2024-04-26 13:42:39 -04:00
return Infrastructure::BasicFilteredResponse::create(vm, *response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> "cors"
case Infrastructure::Request::ResponseTainting::CORS:
// CORS filtered response
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Responses
2024-04-26 13:42:39 -04:00
return Infrastructure::CORSFilteredResponse::create(vm, *response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// -> "opaque"
case Infrastructure::Request::ResponseTainting::Opaque:
// opaque filtered response
return Infrastructure::OpaqueFilteredResponse::create(vm, *response);
default:
VERIFY_NOT_REACHED();
}
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Responses
2024-04-26 13:42:39 -04:00
}();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 15. Let internalResponse be response, if response is a network error, and response’s internal response
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// otherwise.
auto internal_response = response->is_network_error()
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
? GC::Ref { *response }
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
: static_cast<Infrastructure::FilteredResponse&>(*response).internal_response();
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 16. If internalResponse’s URL list is empty, then set it to a clone of request’s URL list.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: A response’s URL list can be empty (for example, when the response represents an about URL).
if (internal_response->url_list().is_empty())
internal_response->set_url_list(request->url_list());
LibWeb: Update Fetch's compute the redirect-taint concept
2025-08-10 19:41:30 +02:00
// 17. Set internalResponse’s redirect taint to request’s redirect-taint.
internal_response->set_redirect_taint(request->redirect_taint());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 18. If request’s timing allow failed flag is unset, then set internalResponse’s timing allow passed flag.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!request->timing_allow_failed())
internal_response->set_timing_allow_passed(true);
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 19. If response is not a network error and any of the following returns blocked
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!response->is_network_error() && (
LibWeb: Implement should block mixed content response to request
2024-05-29 19:40:01 +01:00
// - should internalResponse to request be blocked as mixed content
MixedContent::should_response_to_request_be_blocked_as_mixed_content(request, internal_response) == Infrastructure::RequestOrResponseBlocking::Blocked
LibWeb: Enforce Content Security Policy of Fetch responses
2024-11-28 12:30:36 +00:00
// - should internalResponse to request be blocked by Content Security Policy
|| ContentSecurityPolicy::should_response_to_request_be_blocked_by_content_security_policy(realm, internal_response, request) == ContentSecurityPolicy::Directives::Directive::Result::Blocked
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// - should internalResponse to request be blocked due to its MIME type
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
|| Infrastructure::should_response_to_request_be_blocked_due_to_its_mime_type(internal_response, request) == Infrastructure::RequestOrResponseBlocking::Blocked
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// - should internalResponse to request be blocked due to nosniff
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
|| Infrastructure::should_response_to_request_be_blocked_due_to_nosniff(internal_response, request) == Infrastructure::RequestOrResponseBlocking::Blocked)) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// then set response and internalResponse to a network error.
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
response = internal_response = Infrastructure::Response::network_error(vm, "Response was blocked"_string);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 20. If response’s type is "opaque", internalResponse’s status is 206, internalResponse’s range-requested
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// flag is set, and request’s header list does not contain `Range`, then set response and
// internalResponse to a network error.
// NOTE: Traditionally, APIs accept a ranged response even if a range was not requested. This prevents a
// partial response from an earlier ranged request being provided to an API that did not make a range
// request.
if (response->type() == Infrastructure::Response::Type::Opaque
&& internal_response->status() == 206
&& internal_response->range_requested()
&& !request->header_list()->contains("Range"sv.bytes())) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
response = internal_response = Infrastructure::Response::network_error(vm, "Response has status 206 and 'range-requested' flag set, but request has no 'Range' header"_string);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 21. If response is not a network error and either request’s method is `HEAD` or `CONNECT`, or
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// internalResponse’s status is a null body status, set internalResponse’s body to null and disregard
// any enqueuing toward it (if any).
// NOTE: This standardizes the error handling for servers that violate HTTP.
if (!response->is_network_error() && (StringView { request->method() }.is_one_of("HEAD"sv, "CONNECT"sv) || Infrastructure::is_null_body_status(internal_response->status())))
internal_response->set_body({});
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 22. If request’s integrity metadata is not the empty string, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!request->integrity_metadata().is_empty()) {
// 1. Let processBodyError be this step: run fetch response handover given fetchParams and a network
// error.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto process_body_error = GC::create_function(vm.heap(), [&realm, &vm, &fetch_params](JS::Value) {
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
fetch_response_handover(realm, fetch_params, Infrastructure::Response::network_error(vm, "Response body could not be processed"_string));
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. If response’s body is null, then run processBodyError and abort these steps.
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
if (!response->body()) {
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
process_body_error->function()({});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
// 3. Let processBody given bytes be these steps:
LibWeb: Do not move heap functions into other heap functions in Fetch In particular, the processBody callback here *can't* move the processBodyError callback. It is needed a few lines after. Passing by value is safe and intended here.
2024-04-30 07:24:37 -04:00
auto process_body = GC::create_function(vm.heap(), [&realm, request, response, &fetch_params, process_body_error](ByteBuffer bytes) {
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
// 1. If bytes do not match request’s integrity metadata, then run processBodyError and abort these steps.
if (!TRY_OR_IGNORE(SRI::do_bytes_match_metadata_list(bytes, request->integrity_metadata()))) {
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
process_body_error->function()({});
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
return;
}
// 2. Set response’s body to bytes as a body.
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
response->set_body(Infrastructure::byte_sequence_as_body(realm, bytes));
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
// 3. Run fetch response handover given fetchParams and response.
LibWeb: Propagate OOM in Body::fully_read() through its error callback Fetched bodies can be on the order of gigabytes, so rather than crashing when we hit OOM here, we can simply invoke the error callback with a DOM exception. We use "UnknownError" here as the spec directly supports this for OOM errors: UnknownError: The operation failed for an unknown transient reason (e.g. out of memory). This is still an ad-hoc implementation. We should be using streams, and we do have the AOs available to do so. But they need to be massaged to be compatible with callers of Body::fully_read. And once we do use streams, this function will become infallible - so making it infallible here is at least a step in the right direction.
2024-04-26 14:57:40 -04:00
fetch_response_handover(realm, fetch_params, *response);
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Implement integrity-metadata part of fetch algorithm Specifically, this makes `<link>` elements with an `integrity` attribute actually work. Previously, we would load their resource, and then drop it on the floor without actually using it. The Subresource Integrity code is in `LibWeb/SRI`, since SRI is the name of the recommendation spec: https://www.w3.org/TR/SRI/ However, the Fetch spec links to the editor's draft, which varies significantly from the recommendation, and so that is what the code is based on and what the spec comments link to: https://w3c.github.io/webappsec-subresource-integrity/ Fixes #18408
2023-04-20 16:52:01 +01:00
// 4. Fully read response’s body given processBody and processBodyError.
LibWeb: Propagate OOM in Body::fully_read() through its error callback Fetched bodies can be on the order of gigabytes, so rather than crashing when we hit OOM here, we can simply invoke the error callback with a DOM exception. We use "UnknownError" here as the spec directly supports this for OOM errors: UnknownError: The operation failed for an unknown transient reason (e.g. out of memory). This is still an ad-hoc implementation. We should be using streams, and we do have the AOs available to do so. But they need to be massaged to be compatible with callers of Body::fully_read. And once we do use streams, this function will become infallible - so making it infallible here is at least a step in the right direction.
2024-04-26 14:57:40 -04:00
response->body()->fully_read(realm, process_body, process_body_error, fetch_params.task_destination());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Add FIXME: for new step 6 of Fetch's "main fetch" This step was added in this commit: https://github.com/whatwg/fetch/commit/2d78995db8dbbe1c9ab4a9c6765d7d6603216bff
2023-04-19 15:49:15 +01:00
// 23. Otherwise, run fetch response handover given fetchParams and response.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
else {
LibWeb: Propagate OOM in Body::fully_read() through its error callback Fetched bodies can be on the order of gigabytes, so rather than crashing when we hit OOM here, we can simply invoke the error callback with a DOM exception. We use "UnknownError" here as the spec directly supports this for OOM errors: UnknownError: The operation failed for an unknown transient reason (e.g. out of memory). This is still an ad-hoc implementation. We should be using streams, and we do have the AOs available to do so. But they need to be massaged to be compatible with callers of Body::fully_read. And once we do use streams, this function will become infallible - so making it infallible here is at least a step in the right direction.
2024-04-26 14:57:40 -04:00
fetch_response_handover(realm, fetch_params, *response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
});
LibWeb: Make EventLoopPlugin::deferred_invoke take a HeapFunction
2024-10-31 02:39:29 +13:00
}));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
return GC::Ptr<PendingResponse> {};
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// https://fetch.spec.whatwg.org/#request-determine-the-environment
static GC::Ptr<HTML::Environment> determine_the_environment(GC::Ref<Infrastructure::Request> request)
{
// 1. If request’s reserved client is non-null, then return request’s reserved client.
if (request->reserved_client())
return request->reserved_client();
// 2. If request’s client is non-null, then return request’s client.
if (request->client())
return request->client();
// 3. Return null.
return {};
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// https://fetch.spec.whatwg.org/#fetch-finale
LibWeb: Propagate OOM in Body::fully_read() through its error callback Fetched bodies can be on the order of gigabytes, so rather than crashing when we hit OOM here, we can simply invoke the error callback with a DOM exception. We use "UnknownError" here as the spec directly supports this for OOM errors: UnknownError: The operation failed for an unknown transient reason (e.g. out of memory). This is still an ad-hoc implementation. We should be using streams, and we do have the AOs available to do so. But they need to be massaged to be compatible with callers of Body::fully_read. And once we do use streams, this function will become infallible - so making it infallible here is at least a step in the right direction.
2024-04-26 14:57:40 -04:00
void fetch_response_handover(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params, Infrastructure::Response& response)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'fetch response handover' with: fetch_params @ {}, response @ {}", &fetch_params, &response);
auto& vm = realm.vm();
// 1. Let timingInfo be fetchParams’s timing info.
auto timing_info = fetch_params.timing_info();
// 2. If response is not a network error and fetchParams’s request’s client is a secure context, then set
// timingInfo’s server-timing headers to the result of getting, decoding, and splitting `Server-Timing` from
// response’s header list.
// The user agent may decide to expose `Server-Timing` headers to non-secure contexts requests as well.
LibJS+LibWeb: Wrap raw JS::Cell*/& fields in GCPtr/NonnullGCPtr
2023-02-26 16:09:02 -07:00
auto client = fetch_params.request()->client();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!response.is_network_error() && client != nullptr && HTML::is_secure_context(*client)) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto server_timing_headers = response.header_list()->get_decode_and_split("Server-Timing"sv.bytes());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (server_timing_headers.has_value())
timing_info->set_server_timing_headers(server_timing_headers.release_value());
}
// 3. Let processResponseEndOfBody be the following steps:
auto process_response_end_of_body = [&vm, &response, &fetch_params, timing_info] {
// 1. Let unsafeEndTime be the unsafe shared current time.
auto unsafe_end_time = HighResolutionTime::unsafe_shared_current_time();
// 2. If fetchParams’s request’s destination is "document", then set fetchParams’s controller’s full timing
// info to fetchParams’s timing info.
if (fetch_params.request()->destination() == Infrastructure::Request::Destination::Document)
fetch_params.controller()->set_full_timing_info(fetch_params.timing_info());
// 3. Set fetchParams’s controller’s report timing steps to the following steps given a global object global:
LibWeb: Make reference to global in report timing steps non-const Marking a resource timing entry requires calling non-const methods on the global object to append to the performance buffer.
2025-02-26 15:02:49 +00:00
fetch_params.controller()->set_report_timing_steps([&vm, &response, &fetch_params, timing_info, unsafe_end_time](JS::Object& global) mutable {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. If fetchParams’s request’s URL’s scheme is not an HTTP(S) scheme, then return.
if (!Infrastructure::is_http_or_https_scheme(fetch_params.request()->url().scheme()))
return;
// 2. Set timingInfo’s end time to the relative high resolution time given unsafeEndTime and global.
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
// Spec Issue: Using relative time here is incorrect, as end time is converted to relative time by Resource Timing,
// causing it to take a relative time of an already relative time, effectively make it always a negative
// value approximately the value of the time origin.
timing_info->set_end_time(unsafe_end_time);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. Let cacheState be response’s cache state.
auto cache_state = response.cache_state();
// 4. Let bodyInfo be response’s body info.
auto body_info = response.body_info();
// 5. If response’s timing allow passed flag is not set, then set timingInfo to the result of creating an
// opaque timing info for timingInfo, set bodyInfo to a new response body info, and set cacheState to
// the empty string.
// NOTE: This covers the case of response being a network error.
if (!response.timing_allow_passed()) {
timing_info = Infrastructure::create_opaque_timing_info(vm, timing_info);
body_info = Infrastructure::Response::BodyInfo {};
cache_state = {};
}
LibWeb/Fetch: Share a conditional in fetch response handover See: - https://github.com/whatwg/fetch/commit/aaada1f
2024-05-19 13:34:15 +01:00
// 6. Let responseStatus be 0.
auto response_status = 0;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Update Fetch's compute the redirect-taint concept
2025-08-10 19:41:30 +02:00
// 7. If fetchParams’s request’s mode is not "navigate" or response’s redirect taint is "same-origin":
if (fetch_params.request()->mode() != Infrastructure::Request::Mode::Navigate || response.redirect_taint() == Infrastructure::RedirectTaint::SameOrigin) {
LibWeb/Fetch: Share a conditional in fetch response handover See: - https://github.com/whatwg/fetch/commit/aaada1f
2024-05-19 13:34:15 +01:00
// 1. Set responseStatus to response’s status.
response_status = response.status();
// 2. Let mimeType be the result of extracting a MIME type from response’s header list.
LibWeb/Fetch: Expose a minimised Content-Type to Resource Timing See: - https://github.com/whatwg/fetch/commit/931cd06
2024-05-19 13:23:38 +01:00
auto mime_type = response.header_list()->extract_mime_type();
LibWeb/Fetch: Share a conditional in fetch response handover See: - https://github.com/whatwg/fetch/commit/aaada1f
2024-05-19 13:34:15 +01:00
// 3. If mimeType is non-null, then set bodyInfo’s content type to the result of minimizing a supported MIME type given mimeType.
LibWeb/Fetch: Expose a minimised Content-Type to Resource Timing See: - https://github.com/whatwg/fetch/commit/931cd06
2024-05-19 13:23:38 +01:00
if (mime_type.has_value())
body_info.content_type = MimeSniff::minimise_a_supported_mime_type(mime_type.value());
}
LibWeb: Implement Resource Timing
2025-02-26 15:51:05 +00:00
// 8. If fetchParams’s request’s initiator type is not null, then mark resource timing given timingInfo,
// request’s URL, request’s initiator type, global, cacheState, bodyInfo, and responseStatus.
if (fetch_params.request()->initiator_type().has_value()) {
ResourceTiming::PerformanceResourceTiming::mark_resource_timing(timing_info, fetch_params.request()->url().to_string(), Infrastructure::initiator_type_to_string(fetch_params.request()->initiator_type().value()), global, cache_state, body_info, response_status);
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
});
// 4. Let processResponseEndOfBodyTask be the following steps:
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto process_response_end_of_body_task = GC::create_function(vm.heap(), [&fetch_params, &response] {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. Set fetchParams’s request’s done flag.
fetch_params.request()->set_done(true);
// 2. If fetchParams’s process response end-of-body is non-null, then run fetchParams’s process response
// end-of-body given response.
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
if (fetch_params.algorithms()->process_response_end_of_body())
(fetch_params.algorithms()->process_response_end_of_body())(response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. If fetchParams’s request’s initiator type is non-null and fetchParams’s request’s client’s global
// object is fetchParams’s task destination, then run fetchParams’s controller’s report timing steps
// given fetchParams’s request’s client’s global object.
LibJS+LibWeb: Wrap raw JS::Cell*/& fields in GCPtr/NonnullGCPtr
2023-02-26 16:09:02 -07:00
auto client = fetch_params.request()->client();
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto const* task_destination_global_object = fetch_params.task_destination().get_pointer<GC::Ref<JS::Object>>();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (client != nullptr && task_destination_global_object != nullptr) {
if (fetch_params.request()->initiator_type().has_value() && &client->global_object() == task_destination_global_object->ptr())
fetch_params.controller()->report_timing(client->global_object());
}
LibWeb: Let queue_fetch_task() take a JS::HeapFunction Changes the signature of queue_fetch_task() from AK:Function to JS::HeapFunction to be more clear to the user of the function that this is what it uses internally.
2024-04-19 10:23:40 +02:00
});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 5. Queue a fetch task to run processResponseEndOfBodyTask with fetchParams’s task destination.
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
Infrastructure::queue_fetch_task(fetch_params.controller(), fetch_params.task_destination(), move(process_response_end_of_body_task));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
};
// 4. If fetchParams’s process response is non-null, then queue a fetch task to run fetchParams’s process response
// given response, with fetchParams’s task destination.
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
if (fetch_params.algorithms()->process_response()) {
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
Infrastructure::queue_fetch_task(fetch_params.controller(), fetch_params.task_destination(), GC::create_function(vm.heap(), [&fetch_params, &response]() {
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
fetch_params.algorithms()->process_response()(response);
LibWeb: Let queue_fetch_task() take a JS::HeapFunction Changes the signature of queue_fetch_task() from AK:Function to JS::HeapFunction to be more clear to the user of the function that this is what it uses internally.
2024-04-19 10:23:40 +02:00
}));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Handover the fetch response's internal body data upon completion These is a normative change to the Fetch spec. See: https://github.com/whatwg/fetch/commit/9003266 https://github.com/whatwg/fetch/commit/b5a587b
2023-05-26 09:42:26 -04:00
// 5. Let internalResponse be response, if response is a network error; otherwise response’s internal response.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto internal_response = response.is_network_error() ? GC::Ref { response } : response.unsafe_response();
LibWeb: Handover the fetch response's internal body data upon completion These is a normative change to the Fetch spec. See: https://github.com/whatwg/fetch/commit/9003266 https://github.com/whatwg/fetch/commit/b5a587b
2023-05-26 09:42:26 -04:00
// 6. If internalResponse’s body is null, then run processResponseEndOfBody.
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
if (!internal_response->body()) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
process_response_end_of_body();
}
LibWeb: Handover the fetch response's internal body data upon completion These is a normative change to the Fetch spec. See: https://github.com/whatwg/fetch/commit/9003266 https://github.com/whatwg/fetch/commit/b5a587b
2023-05-26 09:42:26 -04:00
// 7. Otherwise:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
else {
LibWeb: Make TemporaryExecutionContext take a Realm&
2024-10-24 20:39:18 +13:00
HTML::TemporaryExecutionContext const execution_context { realm, HTML::TemporaryExecutionContext::CallbacksEnabled::Yes };
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
// 1. Let transformStream be a new TransformStream.
LibJS+LibWeb: Use realm.create<T> instead of heap.allocate<T> The main motivation behind this is to remove JS specifics of the Realm from the implementation of the Heap. As a side effect of this change, this is a bit nicer to read than the previous approach, and in my opinion, also makes it a little more clear that this method is specific to a JavaScript Realm.
2024-11-14 05:50:17 +13:00
auto transform_stream = realm.create<Streams::TransformStream>(realm);
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
// 2. Let identityTransformAlgorithm be an algorithm which, given chunk, enqueues chunk in transformStream.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto identity_transform_algorithm = GC::create_function(realm.heap(), [&realm, transform_stream](JS::Value chunk) -> GC::Ref<WebIDL::Promise> {
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
MUST(Streams::transform_stream_default_controller_enqueue(*transform_stream->controller(), chunk));
return WebIDL::create_resolved_promise(realm, JS::js_undefined());
});
// 3. Set up transformStream with transformAlgorithm set to identityTransformAlgorithm and flushAlgorithm set
// to processResponseEndOfBody.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto flush_algorithm = GC::create_function(realm.heap(), [&realm, process_response_end_of_body]() -> GC::Ref<WebIDL::Promise> {
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
process_response_end_of_body();
return WebIDL::create_resolved_promise(realm, JS::js_undefined());
});
LibWeb/Streams: Move "set up transform stream" to TransformStream This is not marked as an AO in the spec, and is a publically exported API exposed on TransformStream.
2024-12-08 17:35:07 +13:00
transform_stream->set_up(identity_transform_algorithm, flush_algorithm);
LibWeb: Integrate TransformStream into fetch_response_handover()
2024-04-27 10:00:47 +02:00
// 4. Set internalResponse’s body’s stream to the result of internalResponse’s body’s stream piped through transformStream.
LibWeb/Streams: Actually implement the piped through steps This mistakenly implemented the 'piped to' operation on ReadableStream. No functional difference as the caller was doing the extra work already of 'piped through' vs 'piped to'.
2024-12-26 12:02:07 +13:00
internal_response->body()->set_stream(internal_response->body()->stream()->piped_through(transform_stream));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Handover the fetch response's internal body data upon completion These is a normative change to the Fetch spec. See: https://github.com/whatwg/fetch/commit/9003266 https://github.com/whatwg/fetch/commit/b5a587b
2023-05-26 09:42:26 -04:00
// 8. If fetchParams’s process response consume body is non-null, then:
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
if (fetch_params.algorithms()->process_response_consume_body()) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. Let processBody given nullOrBytes be this step: run fetchParams’s process response consume body given
// response and nullOrBytes.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto process_body = GC::create_function(vm.heap(), [&fetch_params, &response](ByteBuffer null_or_bytes) {
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
(fetch_params.algorithms()->process_response_consume_body())(response, null_or_bytes);
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Let processBodyError be this step: run fetchParams’s process response consume body given response and
// failure.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto process_body_error = GC::create_function(vm.heap(), [&fetch_params, &response](JS::Value) {
LibWeb/Fetch: Use `JS::HeapFunction` for callback in `FetchAlgorithms` In FetchAlgorithms, it is common for callbacks to capture realms. This can indirectly keep objects alive that hold FetchController with these callbacks. This creates a cyclic dependency. However, when JS::HeapFunction is used, this is not a problem, as captured by callbacks values do not create new roots.
2023-08-18 18:26:08 +02:00
(fetch_params.algorithms()->process_response_consume_body())(response, Infrastructure::FetchAlgorithms::ConsumeBodyFailureTag {});
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
});
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Handover the fetch response's internal body data upon completion These is a normative change to the Fetch spec. See: https://github.com/whatwg/fetch/commit/9003266 https://github.com/whatwg/fetch/commit/b5a587b
2023-05-26 09:42:26 -04:00
// 3. If internalResponse's body is null, then queue a fetch task to run processBody given null, with
// fetchParams’s task destination.
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
if (!internal_response->body()) {
LibWeb: Add "parallel queue" and allow it as fetch task destination Note that it's not actually executing tasks in parallel, it's still throwing them on the HTML event loop task queue, each with its own unique task source. This makes our fetch implementation a lot more robust when HTTP caching is enabled, and you can now click links on https://terminal.shop/ without hitting TODO assertions in fetch.
2025-07-16 12:29:26 +02:00
Infrastructure::queue_fetch_task(fetch_params.controller(), fetch_params.task_destination(), GC::create_function(vm.heap(), [process_body]() {
LibWeb: Fix GC leaks in Fetch::Infrastructure::Body::fully_read() By making this function accept the success and error steps as HeapFunction rather than SafeFunction, we break a bunch of strong GC cycles.
2024-04-23 10:25:20 +02:00
process_body->function()({});
LibWeb: Let queue_fetch_task() take a JS::HeapFunction Changes the signature of queue_fetch_task() from AK:Function to JS::HeapFunction to be more clear to the user of the function that this is what it uses internally.
2024-04-19 10:23:40 +02:00
}));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Handover the fetch response's internal body data upon completion These is a normative change to the Fetch spec. See: https://github.com/whatwg/fetch/commit/9003266 https://github.com/whatwg/fetch/commit/b5a587b
2023-05-26 09:42:26 -04:00
// 4. Otherwise, fully read internalResponse body given processBody, processBodyError, and fetchParams’s task
LibWeb: Fully read body if there is one in fetch response handover Required by XHR's reliance on Fetch.
2023-02-28 18:48:59 +00:00
// destination.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
else {
LibWeb: Propagate OOM in Body::fully_read() through its error callback Fetched bodies can be on the order of gigabytes, so rather than crashing when we hit OOM here, we can simply invoke the error callback with a DOM exception. We use "UnknownError" here as the spec directly supports this for OOM errors: UnknownError: The operation failed for an unknown transient reason (e.g. out of memory). This is still an ad-hoc implementation. We should be using streams, and we do have the AOs available to do so. But they need to be massaged to be compatible with callers of Body::fully_read. And once we do use streams, this function will become infallible - so making it infallible here is at least a step in the right direction.
2024-04-26 14:57:40 -04:00
internal_response->body()->fully_read(realm, process_body, process_body_error, fetch_params.task_destination());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
}
// https://fetch.spec.whatwg.org/#concept-scheme-fetch
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ref<PendingResponse>> scheme_fetch(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'scheme fetch' with: fetch_params @ {}", &fetch_params);
auto& vm = realm.vm();
// 1. If fetchParams is canceled, then return the appropriate network error for fetchParams.
if (fetch_params.is_canceled())
LibWeb/Fetch: Store Response error message as a String{,View} variant The majority of error strings are StringView literals, and it seems silly to require heap-allocating strings for these. This idea is stolen from a recent change in fd1fbad :^)
2023-03-03 18:02:43 +00:00
return PendingResponse::create(vm, fetch_params.request(), Infrastructure::Response::appropriate_network_error(vm, fetch_params));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Let request be fetchParams’s request.
auto request = fetch_params.request();
// 3. Switch on request’s current URL’s scheme and run the associated steps:
// -> "about"
if (request->current_url().scheme() == "about"sv) {
// If request’s current URL’s path is the string "blank", then return a new response whose status message is
// `OK`, header list is « (`Content-Type`, `text/html;charset=utf-8`) », and body is the empty byte sequence as
// a body.
// NOTE: URLs such as "about:config" are handled during navigation and result in a network error in the context
// of fetching.
LibWeb+LibURL: Use URL paths directly for comparison This matches the text of the spec a little more closely in many cases and is also more efficient than serializing the URL path.
2024-08-05 15:14:00 +12:00
if (request->current_url().paths().size() == 1 && request->current_url().paths()[0] == "blank"sv) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto response = Infrastructure::Response::create(vm);
response->set_status_message(MUST(ByteBuffer::copy("OK"sv.bytes())));
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Content-Type"sv, "text/html;charset=utf-8"sv);
response->header_list()->append(move(header));
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
response->set_body(Infrastructure::byte_sequence_as_body(realm, ""sv.bytes()));
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
return PendingResponse::create(vm, request, response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
Ladybird+LibWeb: Add about scheme support for internal pages
2024-01-12 19:40:41 +01:00
// FIXME: This is actually wrong, see note above.
return TRY(nonstandard_resource_loader_file_or_http_network_fetch(realm, fetch_params));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// -> "blob"
else if (request->current_url().scheme() == "blob"sv) {
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 1. Let blobURLEntry be request’s current URL’s blob URL entry.
LibWeb: Use URL's 'blob URL entry' for blob fetches Performing a lookup in the blob URL registry does not work in the case of a web worker - as the registry is not shared between processes. However - the URL itself passed to a worker has the blob attached to it, which we can pull out of the URL on a fetch.
2024-05-05 20:39:13 +12:00
auto const& blob_url_entry = request->current_url().blob_url_entry();
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 2. If request’s method is not `GET` or blobURLEntry is null, then return a network error. [FILEAPI]
LibWeb/LibURL/LibIPC: Extend createObjectURL to also accept MediaSources This required some changes in LibURL & LibIPC since it has its own definition of an BlobURLEntry. For now, we don't have a concrete usage of MediaSource in LibURL so it is defined as an empty struct. This removes one FIXME in an idl file.
2025-08-18 00:37:34 +02:00
if (request->method() != "GET"sv.bytes() || !blob_url_entry.has_value())
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request has an invalid 'blob:' URL"_string));
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 3. Let requestEnvironment be the result of determining the environment given request.
auto request_environment = determine_the_environment(request);
// 4. Let isTopLevelNavigation be true if request’s destination is "document"; otherwise, false.
bool is_top_level_navigation = request->destination() == Infrastructure::Request::Destination::Document;
// 5. If isTopLevelNavigation is false and requestEnvironment is null, then return a network error.
if (!is_top_level_navigation && !request_environment)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request is missing fetch client"_string));
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 6. Let navigationOrEnvironment be the string "navigation" if isTopLevelNavigation is true; otherwise, requestEnvironment.
auto navigation_or_environment = [&]() -> Variant<FileAPI::NavigationEnvironment, GC::Ref<HTML::Environment>> {
if (is_top_level_navigation)
return FileAPI::NavigationEnvironment {};
return GC::Ref { *request_environment };
}();
// 7. Let blob be the result of obtaining a blob object given blobURLEntry and navigationOrEnvironment.
LibWeb/LibURL/LibIPC: Extend createObjectURL to also accept MediaSources This required some changes in LibURL & LibIPC since it has its own definition of an BlobURLEntry. For now, we don't have a concrete usage of MediaSource in LibURL so it is defined as an empty struct. This removes one FIXME in an idl file.
2025-08-18 00:37:34 +02:00
auto maybe_blob_object = FileAPI::obtain_a_blob_object(blob_url_entry.value(), navigation_or_environment);
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 8. If blob is not a Blob object, then return a network error.
LibWeb/LibURL/LibIPC: Extend createObjectURL to also accept MediaSources This required some changes in LibURL & LibIPC since it has its own definition of an BlobURLEntry. For now, we don't have a concrete usage of MediaSource in LibURL so it is defined as an empty struct. This removes one FIXME in an idl file.
2025-08-18 00:37:34 +02:00
if (!maybe_blob_object.has_value())
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Failed to obtain a Blob object from 'blob:' URL"_string));
URL::BlobURLEntry::Blob* blob_object;
if (blob_object = maybe_blob_object.value().get_pointer<URL::BlobURLEntry::Blob>(); !blob_object)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Failed to obtain a Blob object from 'blob:' URL"_string));
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
auto const blob = FileAPI::Blob::create(realm, blob_object->data, blob_object->type);
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 9. Let response be a new response.
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
auto response = Infrastructure::Response::create(vm);
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 10. Let fullLength be blob’s size.
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
auto full_length = blob->size();
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 11. Let serializedFullLength be fullLength, serialized and isomorphic encoded.
AK: Make String::number() infallible This API will always succeed in creating a String representing the provided number in base-10.
2024-10-14 10:05:01 +02:00
auto serialized_full_length = String::number(full_length);
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 12. Let type be blob’s type.
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
auto const& type = blob->type();
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 13. If request’s header list does not contain `Range`:
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
if (!request->header_list()->contains("Range"sv.bytes())) {
// 1. Let bodyWithType be the result of safely extracting blob.
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
auto body_with_type = safely_extract_body(realm, blob->raw_bytes());
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 2. Set response’s status message to `OK`.
response->set_status_message(MUST(ByteBuffer::copy("OK"sv.bytes())));
// 3. Set response’s body to bodyWithType’s body.
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
response->set_body(body_with_type.body);
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 4. Set response’s header list to « (`Content-Length`, serializedFullLength), (`Content-Type`, type) ».
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto content_length_header = Infrastructure::Header::from_string_pair("Content-Length"sv, serialized_full_length);
response->header_list()->append(move(content_length_header));
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto content_type_header = Infrastructure::Header::from_string_pair("Content-Type"sv, type);
response->header_list()->append(move(content_type_header));
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
}
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 14. Otherwise:
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
else {
// 1. Set response’s range-requested flag.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
response->set_range_requested(true);
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 2. Let rangeHeader be the result of getting `Range` from request’s header list.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
auto const range_header = request->header_list()->get("Range"sv.bytes()).value_or(ByteBuffer {});
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 3. Let rangeValue be the result of parsing a single range header value given rangeHeader and true.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
auto maybe_range_value = Infrastructure::parse_single_range_header_value(range_header, true);
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 4. If rangeValue is failure, then return a network error.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
if (!maybe_range_value.has_value())
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Failed to parse single range header value"_string));
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 5. Let (rangeStart, rangeEnd) be rangeValue.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
auto& [range_start, range_end] = maybe_range_value.value();
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 6. If rangeStart is null:
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
if (!range_start.has_value()) {
VERIFY(range_end.has_value());
// 1. Set rangeStart to fullLength − rangeEnd.
range_start = full_length - *range_end;
// 2. Set rangeEnd to rangeStart + rangeEnd − 1.
range_end = *range_start + *range_end - 1;
}
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 7. Otherwise:
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
else {
// 1. If rangeStart is greater than or equal to fullLength, then return a network error.
if (*range_start >= full_length)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "rangeStart is greater than or equal to fullLength"_string));
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
// 2. If rangeEnd is null or rangeEnd is greater than or equal to fullLength, then set rangeEnd to fullLength − 1.
if (!range_end.has_value() || *range_end >= full_length)
range_end = full_length - 1;
}
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 8. Let slicedBlob be the result of invoking slice blob given blob, rangeStart, rangeEnd + 1, and type.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
auto sliced_blob = TRY(blob->slice(*range_start, *range_end + 1, type));
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 9. Let slicedBodyWithType be the result of safely extracting slicedBlob.
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
auto sliced_body_with_type = safely_extract_body(realm, sliced_blob->raw_bytes());
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 10. Set response’s body to slicedBodyWithType’s body.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
response->set_body(sliced_body_with_type.body);
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
// 11. Let serializedSlicedLength be slicedBlob’s size, serialized and isomorphic encoded.
LibWeb/Fetch: Implement blob range section of scheme fetch specification
2024-11-18 17:34:30 -06:00
auto serialized_sliced_length = String::number(sliced_blob->size());
// 12. Let contentRange be the result of invoking build a content range given rangeStart, rangeEnd, and fullLength.
auto content_range = Infrastructure::build_content_range(*range_start, *range_end, full_length);
// 13. Set response’s status to 206.
response->set_status(206);
// 14. Set response’s status message to `Partial Content`.
response->set_status_message(MUST(ByteBuffer::copy("Partial Content"sv.bytes())));
// 15. Set response’s header list to «
// (`Content-Length`, serializedSlicedLength),
auto content_length_header = Infrastructure::Header::from_string_pair("Content-Length"sv, serialized_sliced_length);
response->header_list()->append(move(content_length_header));
// (`Content-Type`, type),
auto content_type_header = Infrastructure::Header::from_string_pair("Content-Type"sv, type);
response->header_list()->append(move(content_type_header));
// (`Content-Range`, contentRange) ».
auto content_range_header = Infrastructure::Header::from_string_pair("Content-Range"sv, content_range);
response->header_list()->append(move(content_range_header));
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
}
LibWeb: Partition Blob URL fetches by Storage Key This was a security mechanism introduced in the fetch spec, with supporting AOs added to the FileAPI spec.
2025-01-19 19:02:18 +13:00
// 15. Return response.
LibWeb: Begin support for requesting blob URLs with Fetch infrastructure This does not yet implement requests with a Range header.
2023-08-01 18:53:39 -04:00
return PendingResponse::create(vm, request, response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// -> "data"
else if (request->current_url().scheme() == "data"sv) {
// 1. Let dataURLStruct be the result of running the data: URL processor on request’s current URL.
LibURL+LibWeb: Move data URL processing to LibWeb's fetch infrastructure This is a fetching AO and is only used by LibWeb in the context of fetch tasks. Move it to LibWeb with other fetch methods. The main reason for this is that it requires the use of other LibWeb AOs such as the forgiving Base64 decoder and MIME sniffing. These AOs aren't available within LibURL.
2024-03-24 08:37:33 -04:00
auto data_url_struct = Infrastructure::process_data_url(request->current_url());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. If dataURLStruct is failure, then return a network error.
AK: Decode data URLs to separate class (and parse like every other URL) Parsing 'data:' URLs took it's own route. It never set standard URL fields like path, query or fragment (except for scheme) and instead gave us separate methods called `data_payload()`, `data_mime_type()`, and `data_payload_is_base64()`. Because parsing 'data:' didn't use standard fields, running the following JS code: new URL('#a', 'data:text/plain,hello').toString() not only cleared the path as URLParser doesn't check for data from data_payload() function (making the result be 'data:#a'), but it also crashes the program because we forbid having an empty MIME type when we serialize to string. With this change, 'data:' URLs will be parsed like every other URLs. To decode the 'data:' URL contents, one needs to call process_data_url() on a URL, which will return a struct containing MIME type with already decoded data! :^)
2023-07-06 19:11:58 +02:00
if (data_url_struct.is_error())
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Failed to process 'data:' URL"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. Let mimeType be dataURLStruct’s MIME type, serialized.
LibWeb: Make more MimeSniff::MimeType APIs infallible
2024-10-14 11:06:43 +02:00
auto const& mime_type = data_url_struct.value().mime_type.serialized();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 4. Return a new response whose status message is `OK`, header list is « (`Content-Type`, mimeType) », and
// body is dataURLStruct’s body as a body.
auto response = Infrastructure::Response::create(vm);
response->set_status_message(MUST(ByteBuffer::copy("OK"sv.bytes())));
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Content-Type"sv, mime_type);
response->header_list()->append(move(header));
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
response->set_body(Infrastructure::byte_sequence_as_body(realm, data_url_struct.value().body));
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
return PendingResponse::create(vm, request, response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// -> "file"
LibWeb: Restrict fetching file: and resource: URLs to internal pages They are now blocked on pages which: - Don't have an opaque origin (should be only user-initiated or about:) - Aren't other file: pages - Aren't other resource: pages
2024-06-10 18:06:37 +02:00
// AD-HOC: "resource"
else if (request->current_url().scheme() == "file"sv || request->current_url().scheme() == "resource"sv) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// For now, unfortunate as it is, file: URLs are left as an exercise for the reader.
// When in doubt, return a network error.
LibWeb+LibURL: Move HTML::Origin to URL::Origin While Origin is defined in the HTML spec - this leaves us with quite an awkward relationship as the URL spec makes use of AO's from what is defined in the HTML spec. To simplify this factoring, relocate Origin into LibURL.
2024-10-05 15:33:34 +13:00
if (request->origin().has<URL::Origin>() && (request->origin().get<URL::Origin>().is_opaque() || request->origin().get<URL::Origin>().scheme() == "file"sv || request->origin().get<URL::Origin>().scheme() == "resource"sv))
LibWeb: Restrict fetching file: and resource: URLs to internal pages They are now blocked on pages which: - Don't have an opaque origin (should be only user-initiated or about:) - Aren't other file: pages - Aren't other resource: pages
2024-06-10 18:06:37 +02:00
return TRY(nonstandard_resource_loader_file_or_http_network_fetch(realm, fetch_params));
else
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'file:' or 'resource:' URL blocked"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// -> HTTP(S) scheme
else if (Infrastructure::is_http_or_https_scheme(request->current_url().scheme())) {
// Return the result of running HTTP fetch given fetchParams.
return http_fetch(realm, fetch_params);
}
// 4. Return a network error.
auto message = request->current_url().scheme() == "about"sv
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
? "Request has invalid 'about:' URL, only 'about:blank' can be fetched"_string
: "Request URL has invalid scheme, must be one of 'about', 'blob', 'data', 'file', 'http', or 'https'"_string;
LibWeb/Fetch: Port infrastructure to new String
2023-03-02 23:26:35 +00:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, move(message)));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// https://fetch.spec.whatwg.org/#concept-http-fetch
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ref<PendingResponse>> http_fetch(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params, MakeCORSPreflight make_cors_preflight)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP fetch' with: fetch_params @ {}, make_cors_preflight = {}",
&fetch_params, make_cors_preflight == MakeCORSPreflight::Yes ? "Yes"sv : "No"sv);
auto& vm = realm.vm();
// 1. Let request be fetchParams’s request.
auto request = fetch_params.request();
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 2. Let response and internalResponse be null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<Infrastructure::Response> response;
GC::Ptr<Infrastructure::Response> internal_response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 3. If request’s service-workers mode is "all", then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (request->service_workers_mode() == Infrastructure::Request::ServiceWorkersMode::All) {
// 1. Let requestForServiceWorker be a clone of request.
LibJS: Make Heap::allocate<T>() infallible Stop worrying about tiny OOMs. Work towards #20449. While going through these, I also changed the function signature in many places where returning ThrowCompletionOr<T> is no longer necessary.
2023-08-13 13:05:26 +02:00
auto request_for_service_worker = request->clone(realm);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. If requestForServiceWorker’s body is non-null, then:
if (!request_for_service_worker->body().has<Empty>()) {
// FIXME: 1. Let transformStream be a new TransformStream.
// FIXME: 2. Let transformAlgorithm given chunk be these steps:
// FIXME: 3. Set up transformStream with transformAlgorithm set to transformAlgorithm.
// FIXME: 4. Set requestForServiceWorker’s body’s stream to the result of requestForServiceWorker’s body’s stream
// piped through transformStream.
}
// 3. Let serviceWorkerStartTime be the coarsened shared current time given fetchParams’s cross-origin isolated
// capability.
auto service_worker_start_time = HighResolutionTime::coarsened_shared_current_time(fetch_params.cross_origin_isolated_capability() == HTML::CanUseCrossOriginIsolatedAPIs::Yes);
// FIXME: 4. Set response to the result of invoking handle fetch for requestForServiceWorker, with fetchParams’s
// controller and fetchParams’s cross-origin isolated capability.
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 5. If response is non-null, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (response) {
// 1. Set fetchParams’s timing info’s final service worker start time to serviceWorkerStartTime.
fetch_params.timing_info()->set_final_service_worker_start_time(service_worker_start_time);
// 2. If request’s body is non-null, then cancel request’s body with undefined.
if (!request->body().has<Empty>()) {
// FIXME: Implement cancelling streams
}
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 3. Set internalResponse to response, if response is not a filtered response; otherwise to response’s
// internal response.
internal_response = !is<Infrastructure::FilteredResponse>(*response)
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
? GC::Ref { *response }
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
: static_cast<Infrastructure::FilteredResponse const&>(*response).internal_response();
// 4. If one of the following is true
if (
// - response’s type is "error"
response->type() == Infrastructure::Response::Type::Error
// - request’s mode is "same-origin" and response’s type is "cors"
|| (request->mode() == Infrastructure::Request::Mode::SameOrigin && response->type() == Infrastructure::Response::Type::CORS)
// - request’s mode is not "no-cors" and response’s type is "opaque"
|| (request->mode() != Infrastructure::Request::Mode::NoCORS && response->type() == Infrastructure::Response::Type::Opaque)
// - request’s redirect mode is not "manual" and response’s type is "opaqueredirect"
|| (request->redirect_mode() != Infrastructure::Request::RedirectMode::Manual && response->type() == Infrastructure::Response::Type::OpaqueRedirect)
// - request’s redirect mode is not "follow" and response’s URL list has more than one item.
|| (request->redirect_mode() != Infrastructure::Request::RedirectMode::Follow && response->url_list().size() > 1)) {
// then return a network error.
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Invalid request/response state combination"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<PendingResponse> pending_actual_response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
auto returned_pending_response = PendingResponse::create(vm, request);
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 4. If response is null, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (!response) {
// 1. If makeCORSPreflight is true and one of these conditions is true:
// NOTE: This step checks the CORS-preflight cache and if there is no suitable entry it performs a
// CORS-preflight fetch which, if successful, populates the cache. The purpose of the CORS-preflight
// fetch is to ensure the fetched resource is familiar with the CORS protocol. The cache is there to
// minimize the number of CORS-preflight fetches.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<PendingResponse> pending_preflight_response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (make_cors_preflight == MakeCORSPreflight::Yes && (
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// - There is no method cache entry match for request’s method using request, and either request’s
// method is not a CORS-safelisted method or request’s use-CORS-preflight flag is set.
// FIXME: We currently have no cache, so there will always be no method cache entry.
(!Infrastructure::is_cors_safelisted_method(request->method()) || request->use_cors_preflight())
// - There is at least one item in the CORS-unsafe request-header names with request’s header list for
// which there is no header-name cache entry match using request.
// FIXME: We currently have no cache, so there will always be no header-name cache entry.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
|| !Infrastructure::get_cors_unsafe_header_names(request->header_list()).is_empty())) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 1. Let preflightResponse be the result of running CORS-preflight fetch given request.
pending_preflight_response = TRY(cors_preflight_fetch(realm, request));
// NOTE: Step 2 is performed in pending_preflight_response's load callback below.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto fetch_main_content = [request = GC::make_root(request), realm = GC::make_root(realm), fetch_params = GC::make_root(fetch_params)]() -> WebIDL::ExceptionOr<GC::Ref<PendingResponse>> {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 2. If request’s redirect mode is "follow", then set request’s service-workers mode to "none".
// NOTE: Redirects coming from the network (as opposed to from a service worker) are not to be exposed to a
// service worker.
if (request->redirect_mode() == Infrastructure::Request::RedirectMode::Follow)
request->set_service_workers_mode(Infrastructure::Request::ServiceWorkersMode::None);
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 3. Set response and internalResponse to the result of running HTTP-network-or-cache fetch given fetchParams.
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return http_network_or_cache_fetch(*realm, *fetch_params);
};
if (pending_preflight_response) {
pending_actual_response = PendingResponse::create(vm, request);
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
pending_preflight_response->when_loaded([returned_pending_response, pending_actual_response, fetch_main_content = move(fetch_main_content)](GC::Ref<Infrastructure::Response> preflight_response) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP fetch' pending_preflight_response load callback");
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 2. If preflightResponse is a network error, then return preflightResponse.
if (preflight_response->is_network_error()) {
returned_pending_response->resolve(preflight_response);
return;
}
auto pending_main_content_response = TRY_OR_IGNORE(fetch_main_content());
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
pending_main_content_response->when_loaded([pending_actual_response](GC::Ref<Infrastructure::Response> main_content_response) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP fetch' pending_main_content_response load callback");
pending_actual_response->resolve(main_content_response);
});
});
} else {
pending_actual_response = TRY(fetch_main_content());
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
} else {
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
pending_actual_response = PendingResponse::create(vm, request, Infrastructure::Response::create(vm));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
pending_actual_response->when_loaded([&realm, &vm, &fetch_params, request, response, internal_response, returned_pending_response, response_was_null = !response](GC::Ref<Infrastructure::Response> resolved_actual_response) mutable {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP fetch' pending_actual_response load callback");
if (response_was_null) {
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
response = internal_response = resolved_actual_response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 4. If request’s response tainting is "cors" and a CORS check for request and response returns failure,
// then return a network error.
// NOTE: As the CORS check is not to be applied to responses whose status is 304 or 407, or responses from
// a service worker for that matter, it is applied here.
if (request->response_tainting() == Infrastructure::Request::ResponseTainting::CORS
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Requests
2024-04-26 13:35:10 -04:00
&& !cors_check(request, *response)) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "Request with 'cors' response tainting failed CORS check"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
// 5. If the TAO check for request and response returns failure, then set request’s timing allow failed flag.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Requests
2024-04-26 13:35:10 -04:00
if (!tao_check(request, *response))
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
request->set_timing_allow_failed(true);
}
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 5. If either request’s response tainting or response’s type is "opaque", and the cross-origin resource
// policy check with request’s origin, request’s client, request’s destination, and internalResponse returns
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// blocked, then return a network error.
// NOTE: The cross-origin resource policy check runs for responses coming from the network and responses coming
// from the service worker. This is different from the CORS check, as request’s client and the service
// worker can have different embedder policies.
if ((request->response_tainting() == Infrastructure::Request::ResponseTainting::Opaque || response->type() == Infrastructure::Response::Type::Opaque)
&& false // FIXME: "and the cross-origin resource policy check with request’s origin, request’s client, request’s destination, and actualResponse returns blocked"
) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "Response was blocked by cross-origin resource policy check"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<PendingResponse> inner_pending_response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 6. If internalResponse’s status is a redirect status:
if (Infrastructure::is_redirect_status(internal_response->status())) {
// FIXME: 1. If internalResponse’s status is not 303, request’s body is non-null, and the connection uses HTTP/2,
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// then user agents may, and are even encouraged to, transmit an RST_STREAM frame.
// NOTE: 303 is excluded as certain communities ascribe special status to it.
// 2. Switch on request’s redirect mode:
switch (request->redirect_mode()) {
// -> "error"
case Infrastructure::Request::RedirectMode::Error:
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 1. Set response to a network error.
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
response = Infrastructure::Response::network_error(vm, "Request with 'error' redirect mode received redirect response"_string);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
break;
// -> "manual"
case Infrastructure::Request::RedirectMode::Manual:
LibWeb/Fetch: Use a basic filtered response for redirect navigations Match following change in the spec: https://github.com/whatwg/fetch/commit/8f109835dcff90d19caed4b551a0da32d9d0f57e
2023-04-09 03:42:31 +03:00
// 1. If request’s mode is "navigate", then set fetchParams’s controller’s next manual redirect steps
// to run HTTP-redirect fetch given fetchParams and response.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (request->mode() == Infrastructure::Request::Mode::Navigate) {
Everywhere: Remove unnecessary mutable attributes from lambdas These lambdas were marked mutable as they captured a Ptr wrapper class by value, which then only returned const-qualified references to the value they point from the previous const pointer operators. Nothing is actually mutating in the lambdas state here, and now that the Ptr operators don't add extra const qualifiers these can be removed.
2022-11-19 01:09:53 +00:00
fetch_params.controller()->set_next_manual_redirect_steps([&realm, &fetch_params, response] {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
(void)http_redirect_fetch(realm, fetch_params, *response);
});
}
LibWeb/Fetch: Use a basic filtered response for redirect navigations Match following change in the spec: https://github.com/whatwg/fetch/commit/8f109835dcff90d19caed4b551a0da32d9d0f57e
2023-04-09 03:42:31 +03:00
// 2. Otherwise, set response to an opaque-redirect filtered response whose internal response is
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// internalResponse.
LibWeb/Fetch: Use a basic filtered response for redirect navigations Match following change in the spec: https://github.com/whatwg/fetch/commit/8f109835dcff90d19caed4b551a0da32d9d0f57e
2023-04-09 03:42:31 +03:00
else {
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
response = Infrastructure::OpaqueRedirectFilteredResponse::create(vm, *internal_response);
LibWeb/Fetch: Use a basic filtered response for redirect navigations Match following change in the spec: https://github.com/whatwg/fetch/commit/8f109835dcff90d19caed4b551a0da32d9d0f57e
2023-04-09 03:42:31 +03:00
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
break;
// -> "follow"
case Infrastructure::Request::RedirectMode::Follow:
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 1. Set response to the result of running HTTP-redirect fetch given fetchParams and response.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
inner_pending_response = TRY_OR_IGNORE(http_redirect_fetch(realm, fetch_params, *response));
break;
default:
VERIFY_NOT_REACHED();
}
}
LibWeb: Prefer GCPtr<T> over Optional<NonnullGCPtr<T>>
2024-04-07 15:40:20 +02:00
if (inner_pending_response) {
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
inner_pending_response->when_loaded([returned_pending_response](GC::Ref<Infrastructure::Response> response) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP fetch' inner_pending_response load callback");
returned_pending_response->resolve(response);
});
} else {
returned_pending_response->resolve(*response);
}
});
LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-03 20:53:05 +01:00
// 7. Return response.
// NOTE: Typically internalResponse’s body’s stream is still being enqueued to after returning.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return returned_pending_response;
}
// https://fetch.spec.whatwg.org/#concept-http-redirect-fetch
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ptr<PendingResponse>> http_redirect_fetch(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params, Infrastructure::Response& response)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP-redirect fetch' with: fetch_params @ {}, response = {}", &fetch_params, &response);
auto& vm = realm.vm();
// 1. Let request be fetchParams’s request.
auto request = fetch_params.request();
LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-05 19:36:18 +01:00
// 2. Let internalResponse be response, if response is not a filtered response; otherwise response’s internal
// response.
auto internal_response = !is<Infrastructure::FilteredResponse>(response)
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
? GC::Ref { response }
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
: static_cast<Infrastructure::FilteredResponse const&>(response).internal_response();
LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-05 19:36:18 +01:00
// 3. Let locationURL be internalResponse’s location URL given request’s current URL’s fragment.
auto location_url_or_error = internal_response->location_url(request->current_url().fragment());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 4. If locationURL is null, then return response.
if (!location_url_or_error.is_error() && !location_url_or_error.value().has_value())
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
return PendingResponse::create(vm, request, response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 5. If locationURL is failure, then return a network error.
if (location_url_or_error.is_error())
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request redirect URL is invalid"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto location_url = location_url_or_error.release_value().release_value();
// 6. If locationURL’s scheme is not an HTTP(S) scheme, then return a network error.
if (!Infrastructure::is_http_or_https_scheme(location_url.scheme()))
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request redirect URL must have HTTP or HTTPS scheme"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 7. If request’s redirect count is 20, then return a network error.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (request->redirect_count() == 20)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request has reached maximum redirect count of 20"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 8. Increase request’s redirect count by 1.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
request->set_redirect_count(request->redirect_count() + 1);
LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-05 19:36:18 +01:00
// 9. If request’s mode is "cors", locationURL includes credentials, and request’s origin is not same origin with
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// locationURL’s origin, then return a network error.
if (request->mode() == Infrastructure::Request::Mode::CORS
&& location_url.includes_credentials()
LibWeb+LibURL: Move HTML::Origin to URL::Origin While Origin is defined in the HTML spec - this leaves us with quite an awkward relationship as the URL spec makes use of AO's from what is defined in the HTML spec. To simplify this factoring, relocate Origin into LibURL.
2024-10-05 15:33:34 +13:00
&& request->origin().has<URL::Origin>()
LibWeb+LibURL: Consolidate Origin parsing and serialization into LibURL Because of the previous awkward factoring of Origin we had two implementations of Origin serializing and creation. Move the implementation of DOMURL::url_origin into URL::origin, and instead use the implemenation of URL::Origin::serialize for serialization (replacing URL::serialize_origin). This happens to fix 8 URL subtests as the two implemenations had diverged, and URL::serialize_origin was previously missing the spec changes of: whatwg/url@eee49fd and whatwg/url@fff33c3
2024-10-05 17:03:51 +13:00
&& !request->origin().get<URL::Origin>().is_same_origin(location_url.origin())) {
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'cors' mode and different URL and request origin must not include credentials in redirect URL"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 10. If request’s response tainting is "cors" and locationURL includes credentials, then return a network error.
// NOTE: This catches a cross-origin resource redirecting to a same-origin URL.
if (request->response_tainting() == Infrastructure::Request::ResponseTainting::CORS && location_url.includes_credentials())
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'cors' response tainting must not include credentials in redirect URL"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-05 19:36:18 +01:00
// 11. If internalResponse’s status is not 303, request’s body is non-null, and request’s body’s source is null, then
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// return a network error.
LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-05 19:36:18 +01:00
if (internal_response->status() != 303
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
&& !request->body().has<Empty>()
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
&& request->body().get<GC::Ref<Infrastructure::Body>>()->source().has<Empty>()) {
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request has body but no body source"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 12. If one of the following is true
if (
LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec The spec and implementation's comments had diverged a little, this brings them in line :)
2024-06-05 19:36:18 +01:00
// - internalResponse’s status is 301 or 302 and request’s method is `POST`
((internal_response->status() == 301 || internal_response->status() == 302) && request->method() == "POST"sv.bytes())
// - internalResponse’s status is 303 and request’s method is not `GET` or `HEAD`
|| (internal_response->status() == 303 && !(request->method() == "GET"sv.bytes() || request->method() == "HEAD"sv.bytes()))
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// then:
) {
// 1. Set request’s method to `GET` and request’s body to null.
request->set_method(MUST(ByteBuffer::copy("GET"sv.bytes())));
request->set_body({});
static constexpr Array request_body_header_names {
"Content-Encoding"sv,
"Content-Language"sv,
"Content-Location"sv,
"Content-Type"sv
};
// 2. For each headerName of request-body-header name, delete headerName from request’s header list.
for (auto header_name : request_body_header_names.span())
request->header_list()->delete_(header_name.bytes());
}
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 13. If request’s current URL’s origin is not same origin with locationURL’s origin, then for each headerName of
// CORS non-wildcard request-header name, delete headerName from request’s header list.
// NOTE: I.e., the moment another origin is seen after the initial request, the `Authorization` header is removed.
LibWeb+LibURL: Consolidate Origin parsing and serialization into LibURL Because of the previous awkward factoring of Origin we had two implementations of Origin serializing and creation. Move the implementation of DOMURL::url_origin into URL::origin, and instead use the implemenation of URL::Origin::serialize for serialization (replacing URL::serialize_origin). This happens to fix 8 URL subtests as the two implemenations had diverged, and URL::serialize_origin was previously missing the spec changes of: whatwg/url@eee49fd and whatwg/url@fff33c3
2024-10-05 17:03:51 +13:00
if (!request->current_url().origin().is_same_origin(location_url.origin())) {
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
static constexpr Array cors_non_wildcard_request_header_names {
"Authorization"sv
};
for (auto header_name : cors_non_wildcard_request_header_names)
request->header_list()->delete_(header_name.bytes());
}
// 14. If request’s body is non-null, then set request’s body to the body of the result of safely extracting
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// request’s body’s source.
// NOTE: request’s body’s source’s nullity has already been checked.
if (!request->body().has<Empty>()) {
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto const& source = request->body().get<GC::Ref<Infrastructure::Body>>()->source();
LibWeb: Fix typo in BodyInitOrReadableBytes type alias
2022-11-09 22:37:07 +00:00
// NOTE: BodyInitOrReadableBytes is a superset of Body::SourceType
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto converted_source = source.has<ByteBuffer>()
LibWeb: Fix typo in BodyInitOrReadableBytes type alias
2022-11-09 22:37:07 +00:00
? BodyInitOrReadableBytes { source.get<ByteBuffer>() }
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
: BodyInitOrReadableBytes { source.get<GC::Root<FileAPI::Blob>>() };
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
auto [body, _] = safely_extract_body(realm, converted_source);
request->set_body(body);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 15. Let timingInfo be fetchParams’s timing info.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto timing_info = fetch_params.timing_info();
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 16. Set timingInfo’s redirect end time and post-redirect start time to the coarsened shared current time given
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// fetchParams’s cross-origin isolated capability.
auto now = HighResolutionTime::coarsened_shared_current_time(fetch_params.cross_origin_isolated_capability() == HTML::CanUseCrossOriginIsolatedAPIs::Yes);
timing_info->set_redirect_end_time(now);
timing_info->set_post_redirect_start_time(now);
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 17. If timingInfo’s redirect start time is 0, then set timingInfo’s redirect start time to timingInfo’s start
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// time.
if (timing_info->redirect_start_time() == 0)
timing_info->set_redirect_start_time(timing_info->start_time());
LibWeb/Fetch: Remove Authorization header upon cross-origin redirect This is a change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/9004f4e
2022-12-07 18:48:40 +00:00
// 18. Append locationURL to request’s URL list.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
request->url_list().append(location_url);
LibWeb: Implement "Set request’s referrer policy on redirect" AO
2024-06-05 20:15:56 +01:00
// 19. Invoke set request’s referrer policy on redirect on request and internalResponse.
ReferrerPolicy::set_request_referrer_policy_on_redirect(request, internal_response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Pass recursive=false to manual navigation redirect Implement https://github.com/whatwg/fetch/commit/ca10f49748f2289433ce75dcf1f2fcf451f4f24f Fixes the issue I found while working on navigation: https://github.com/whatwg/fetch/issues/1629
2023-04-14 20:42:12 +03:00
// 20. Let recursive be true.
auto recursive = Recursive::Yes;
// 21. If request’s redirect mode is "manual", then:
if (request->redirect_mode() == Infrastructure::Request::RedirectMode::Manual) {
// 1. Assert: request’s mode is "navigate".
VERIFY(request->mode() == Infrastructure::Request::Mode::Navigate);
// 2. Set recursive to false.
recursive = Recursive::No;
}
// 22. Return the result of running main fetch given fetchParams and recursive.
return main_fetch(realm, fetch_params, recursive);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
class CachePartition : public RefCounted<CachePartition> {
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
public:
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// https://httpwg.org/specs/rfc9111.html#constructing.responses.from.caches
LibWeb/Fetch: Return a cloned cached response body Otherwise we will fully read from the cached response and invalidate it's stream, invalidating it for the next time it is read from. Fixes a crash when reloading linegoup.lol after two reloads.
2024-12-26 14:58:27 +13:00
GC::Ptr<Infrastructure::Response> select_response(JS::Realm& realm, URL::URL const& url, ReadonlyBytes method, Vector<Infrastructure::Header> const& headers, Vector<GC::Ptr<Infrastructure::Response>>& initial_set_of_stored_responses) const
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
{
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// When presented with a request, a cache MUST NOT reuse a stored response unless:
// - the presented target URI (Section 7.1 of [HTTP]) and that of the stored response match, and
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
auto it = m_cache.find(url);
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
if (it == m_cache.end()) {
dbgln("\033[31;1mHTTP CACHE MISS!\033[0m {}", url);
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
return {};
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
}
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
auto const& cached_response = it->value;
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// - the request method associated with the stored response allows it to be used for the presented request, and
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
if (method != cached_response->method()) {
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
dbgln("\033[31;1mHTTP CACHE MISS!\033[0m (Bad method) {}", url);
return {};
}
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// FIXME: - request header fields nominated by the stored response (if any) match those presented (see Section 4.1), and
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
(void)headers;
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// FIXME: - the stored response does not contain the no-cache directive (Section 5.2.2.4), unless it is successfully validated (Section 4.3), and
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
LibWeb: Protect HTTP cache entries from garbage collector One day we'll have an eviction strategy, too, but for now let's not allow these to get collected. Co-Authored-By: Gingeh <39150378+Gingeh@users.noreply.github.com>
2024-12-25 15:52:50 +01:00
initial_set_of_stored_responses.append(*cached_response);
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
// FIXME: - the stored response is one of the following:
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// + fresh (see Section 4.2), or
// + allowed to be served stale (see Section 4.2.4), or
// + successfully validated (see Section 4.3).
dbgln("\033[32;1mHTTP CACHE HIT!\033[0m {}", url);
LibWeb/Fetch: Return a cloned cached response body Otherwise we will fully read from the cached response and invalidate it's stream, invalidating it for the next time it is read from. Fixes a crash when reloading linegoup.lol after two reloads.
2024-12-26 14:58:27 +13:00
return cached_response->clone(realm);
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
}
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
void store_response(JS::Realm& realm, Infrastructure::Request const& http_request, Infrastructure::Response const& response)
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
{
if (!is_cacheable(http_request, response))
return;
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
auto cached_response = Infrastructure::Response::create(realm.vm());
store_header_and_trailer_fields(response, *cached_response->header_list());
cached_response->set_body(response.body()->clone(realm));
cached_response->set_body_info(response.body_info());
cached_response->set_method(MUST(ByteBuffer::copy(http_request.method())));
cached_response->set_status(response.status());
cached_response->url_list().append(http_request.current_url());
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
m_cache.set(http_request.current_url(), move(cached_response));
}
// https://httpwg.org/specs/rfc9111.html#freshening.responses
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
void freshen_stored_responses_upon_validation(Infrastructure::Response const& response, Vector<GC::Ptr<Infrastructure::Response>>& initial_set_of_stored_responses)
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
{
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
// When a cache receives a 304 (Not Modified) response, it needs to identify stored
// responses that are suitable for updating with the new information provided, and then do so.
// The initial set of stored responses to update are those that could have been
// chosen for that request — i.e., those that meet the requirements in Section 4,
// except the last requirement to be fresh, able to be served stale, or just validated.
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
for (auto stored_response : initial_set_of_stored_responses) {
// Then, that initial set of stored responses is further filtered by the first match of:
// - FIXME: If the new response contains one or more strong validators (see Section 8.8.1 of [HTTP]),
// then each of those strong validators identifies a selected representation for update.
// All the stored responses in the initial set with one of those same strong validators
// are identified for update.
// If none of the initial set contains at least one of the same strong validators,
// then the cache MUST NOT use the new response to update any stored responses.
// - FIXME: If the new response contains no strong validators but does contain one or more weak validators,
// and those validators correspond to one of the initial set's stored responses,
// then the most recent of those matching stored responses is identified for update.
// - FIXME: If the new response does not include any form of validator (such as where a client generates an
// `If-Modified-Since` request from a source other than the `Last-Modified` response header field),
// and there is only one stored response in the initial set, and that stored response also lacks a validator,
// then that stored response is identified for update.
// For each stored response identified, the cache MUST update its header fields
// with the header fields provided in the 304 (Not Modified) response, as per Section 3.2.
update_stored_header_fields(response, stored_response->header_list());
}
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
}
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
private:
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// https://httpwg.org/specs/rfc9111.html#storing.fields
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
bool is_exempted_for_storage(StringView header_name)
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
{
// Caches MUST include all received response header fields — including unrecognized ones — when storing a response;
// this assures that new HTTP header fields can be successfully deployed. However, the following exceptions are made:
// - The Connection header field and fields whose names are listed in it are required by Section 7.6.1 of [HTTP]
// to be removed before forwarding the message. This MAY be implemented by doing so before storage.
// - Likewise, some fields' semantics require them to be removed before forwarding the message, and this MAY be
// implemented by doing so before storage; see Section 7.6.1 of [HTTP] for some examples.
// FIXME: - The no-cache (Section 5.2.2.4) and private (Section 5.2.2.7) cache directives can have arguments that
// prevent storage of header fields by all caches and shared caches, respectively.
// FIXME: - Header fields that are specific to the proxy that a cache uses when forwarding a request MUST NOT be stored,
// unless the cache incorporates the identity of the proxy into the cache key.
// Effectively, this is limited to Proxy-Authenticate (Section 11.7.1 of [HTTP]), Proxy-Authentication-Info (Section 11.7.3 of [HTTP]), and Proxy-Authorization (Section 11.7.2 of [HTTP]).
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
return header_name.is_one_of_ignoring_ascii_case(
"Connection"sv,
"Proxy-Connection"sv,
"Keep-Alive"sv,
"TE"sv,
"Transfer-Encoding"sv,
"Upgrade"sv);
}
// https://httpwg.org/specs/rfc9111.html#update
bool is_exempted_for_updating(StringView header_name)
{
// Caches are required to update a stored response's header fields from another
// (typically newer) response in several situations; for example, see Sections 3.4, 4.3.4, and 4.3.5.
// When doing so, the cache MUST add each header field in the provided response to the stored response,
// replacing field values that are already present, with the following exceptions:
// - Header fields excepted from storage in Section 3.1,
return is_exempted_for_storage(header_name)
// - Header fields that the cache's stored response depends upon, as described below,
|| false
// - Header fields that are automatically processed and removed by the recipient, as described below, and
|| false
// - The Content-Length header field.
|| header_name.equals_ignoring_ascii_case("Content-Length"sv);
// In some cases, caches (especially in user agents) store the results of processing
// the received response, rather than the response itself, and updating header fields
// that affect that processing can result in inconsistent behavior and security issues.
// Caches in this situation MAY omit these header fields from updating stored responses
// on an exceptional basis but SHOULD limit such omission to those fields necessary to
// assure integrity of the stored response.
// For example, a browser might decode the content coding of a response while it is being received,
// creating a disconnect between the data it has stored and the response's original metadata.
// Updating that stored metadata with a different Content-Encoding header field would be problematic.
// Likewise, a browser might store a post-parse HTML tree rather than the content received in the response;
// updating the Content-Type header field would not be workable in this case because any assumptions about
// the format made in parsing would now be invalid.
// Furthermore, some fields are automatically processed and removed by the HTTP implementation,
// such as the Content-Range header field. Implementations MAY automatically omit such header fields from updates,
// even when the processing does not actually occur.
// Note that the Content-* prefix is not a signal that a header field is omitted from update; it is a convention for MIME header fields, not HTTP.
}
// https://httpwg.org/specs/rfc9111.html#update
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
void update_stored_header_fields(Infrastructure::Response const& response, Infrastructure::HeaderList& headers)
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
{
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
for (auto& header : *response.header_list()) {
auto name = StringView(header.name);
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
if (is_exempted_for_updating(name))
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
continue;
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
headers.delete_(header.name);
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
}
for (auto& header : *response.header_list()) {
auto name = StringView(header.name);
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
if (is_exempted_for_updating(name))
continue;
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
headers.append(Infrastructure::Header::copy(header));
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
}
}
// https://httpwg.org/specs/rfc9111.html#storing.fields
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
void store_header_and_trailer_fields(Infrastructure::Response const& response, Web::Fetch::Infrastructure::HeaderList& headers)
LibWeb/Fetch: Update cached responses when revalidating Responses returned from `http_network_or_cache_fetch` were copied directly from the cache, which is incorrect, since revalidation may later modify the response, or even invalidate it, such as when the `Access-Control-Allow-Origin` header is changed. This fixes WPT test [wpt/cors/304.htm](http://wpt.live/cors/304.htm)
2024-10-22 10:35:01 +02:00
{
for (auto& header : *response.header_list()) {
auto name = StringView(header.name);
if (is_exempted_for_storage(name))
continue;
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
headers.append(Infrastructure::Header::copy(header));
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
}
}
// https://httpwg.org/specs/rfc9111.html#response.cacheability
static bool is_cacheable(Infrastructure::Request const& request, Infrastructure::Response const& response)
{
// A cache MUST NOT store a response to a request unless:
// - AD-HOC: For now, we simply don't cache responses without a simple ByteBuffer body.
if (!response.body() || !response.body()->source().has<ByteBuffer>())
return false;
// - the request method is understood by the cache;
if (request.method() != "GET"sv.bytes() && request.method() != "HEAD"sv.bytes())
return false;
// - the response status code is final (see Section 15 of [HTTP]);
if (response.status() < 200)
return false;
// - if the response status code is 206 or 304,
// or the must-understand cache directive (see Section 5.2.2.3) is present:
// the cache understands the response status code;
if (response.status() == 206 || response.status() == 304) {
// FIXME: Implement must-understand cache directive
}
// - the no-store cache directive is not present in the response (see Section 5.2.2.5);
if (request.cache_mode() == Infrastructure::Request::CacheMode::NoStore)
return false;
// FIXME: - if the cache is shared: the private response directive is either not present
// or allows a shared cache to store a modified response; see Section 5.2.2.7);
// FIXME: - if the cache is shared: the Authorization header field is not present in the
// request (see Section 11.6.2 of [HTTP]) or a response directive is present
// that explicitly allows shared caching (see Section 3.5); and
// FIXME: - the response contains at least one of the following:
// + a public response directive (see Section 5.2.2.9);
// + a private response directive, if the cache is not shared (see Section 5.2.2.7);
// + an Expires header field (see Section 5.3);
// + a max-age response directive (see Section 5.2.2.1);
// + if the cache is shared: an s-maxage response directive (see Section 5.2.2.10);
// + a cache extension that allows it to be cached (see Section 5.2.3); or
// + a status code that is defined as heuristically cacheable (see Section 4.2.2).
return true;
}
LibWeb: Protect HTTP cache entries from garbage collector One day we'll have an eviction strategy, too, but for now let's not allow these to get collected. Co-Authored-By: Gingeh <39150378+Gingeh@users.noreply.github.com>
2024-12-25 15:52:50 +01:00
HashMap<URL::URL, GC::Root<Infrastructure::Response>> m_cache;
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
};
class HTTPCache {
public:
LibWeb: Move NetworkPartitionKey into a header This is used elsewhere, such as the HTMLLinkElement.
2024-05-31 18:44:49 +01:00
CachePartition& get(Infrastructure::NetworkPartitionKey const& key)
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
{
return *m_cache.ensure(key, [] {
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
return adopt_ref(*new CachePartition);
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
});
}
static HTTPCache& the()
{
static HTTPCache s_cache;
return s_cache;
}
private:
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
HashMap<Infrastructure::NetworkPartitionKey, NonnullRefPtr<CachePartition>> m_cache;
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
};
// https://fetch.spec.whatwg.org/#determine-the-http-cache-partition
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
static RefPtr<CachePartition> determine_the_http_cache_partition(Infrastructure::Request const& request)
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
{
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
if (!g_http_cache_enabled)
return nullptr;
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
// 1. Let key be the result of determining the network partition key given request.
LibWeb: Move NetworkPartitionKey into a header This is used elsewhere, such as the HTMLLinkElement.
2024-05-31 18:44:49 +01:00
auto key = Infrastructure::determine_the_network_partition_key(request);
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
// 2. If key is null, then return null.
if (!key.has_value())
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
return nullptr;
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
// 3. Return the unique HTTP cache associated with key. [HTTP-CACHING]
return HTTPCache::the().get(key.value());
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// https://fetch.spec.whatwg.org/#concept-http-network-or-cache-fetch
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ref<PendingResponse>> http_network_or_cache_fetch(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params, IsAuthenticationFetch is_authentication_fetch, IsNewConnectionFetch is_new_connection_fetch)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP-network-or-cache fetch' with: fetch_params @ {}, is_authentication_fetch = {}, is_new_connection_fetch = {}",
&fetch_params, is_authentication_fetch == IsAuthenticationFetch::Yes ? "Yes"sv : "No"sv, is_new_connection_fetch == IsNewConnectionFetch::Yes ? "Yes"sv : "No"sv);
auto& vm = realm.vm();
// 1. Let request be fetchParams’s request.
auto request = fetch_params.request();
// 2. Let httpFetchParams be null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<Infrastructure::FetchParams const> http_fetch_params;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. Let httpRequest be null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<Infrastructure::Request> http_request;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 4. Let response be null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<Infrastructure::Response> response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 5. Let storedResponse be null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<Infrastructure::Response> stored_response;
LibGC: Rename MarkedVector => RootVector Let's try to make it a bit more clear that this is a Vector of GC roots.
2024-12-26 14:32:52 +01:00
GC::RootVector<GC::Ptr<Infrastructure::Response>> initial_set_of_stored_responses(realm.heap());
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 6. Let httpCache be null.
// (Typeless until we actually implement it, needed for checks below)
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
RefPtr<CachePartition> http_cache;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 7. Let the revalidatingFlag be unset.
auto revalidating_flag = RefCountedFlag::create(false);
auto include_credentials = IncludeCredentials::No;
// 8. Run these steps, but abort when fetchParams is canceled:
// NOTE: There's an 'if aborted' check after this anyway, so not doing this is fine and only incurs a small delay.
// For now, support for aborting fetch requests is limited anyway as ResourceLoader doesn't support it.
auto aborted = false;
{
ScopeGuard set_aborted = [&] {
if (fetch_params.is_canceled())
aborted = true;
};
LibWeb: Replace request's "window" with "traversable for user prompts" User prompts are not tied to specific Windows or the client's Window. They are tied to a traversable navigable (browser tab).
2025-08-07 10:54:40 +02:00
// 1. If request’s traversable for user prompts is "no-traversable" and request’s redirect mode is "error",
// then set httpFetchParams to fetchParams and httpRequest to request.
if (request->traversable_for_user_prompts().has<Infrastructure::Request::TraversableForUserPrompts>()
&& request->traversable_for_user_prompts().get<Infrastructure::Request::TraversableForUserPrompts>() == Infrastructure::Request::TraversableForUserPrompts::NoTraversable
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
&& request->redirect_mode() == Infrastructure::Request::RedirectMode::Error) {
http_fetch_params = fetch_params;
http_request = request;
}
// 2. Otherwise:
else {
// 1. Set httpRequest to a clone of request.
// NOTE: Implementations are encouraged to avoid teeing request’s body’s stream when request’s body’s
// source is null as only a single body is needed in that case. E.g., when request’s body’s source
// is null, redirects and authentication will end up failing the fetch.
LibJS: Make Heap::allocate<T>() infallible Stop worrying about tiny OOMs. Work towards #20449. While going through these, I also changed the function signature in many places where returning ThrowCompletionOr<T> is no longer necessary.
2023-08-13 13:05:26 +02:00
http_request = request->clone(realm);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Set httpFetchParams to a copy of fetchParams.
// 3. Set httpFetchParams’s request to httpRequest.
LibWeb: Fix a few const-ness issues
2023-02-25 10:44:51 -07:00
auto new_http_fetch_params = Infrastructure::FetchParams::create(vm, *http_request, fetch_params.timing_info());
new_http_fetch_params->set_algorithms(fetch_params.algorithms());
new_http_fetch_params->set_task_destination(fetch_params.task_destination());
new_http_fetch_params->set_cross_origin_isolated_capability(fetch_params.cross_origin_isolated_capability());
new_http_fetch_params->set_preloaded_response_candidate(fetch_params.preloaded_response_candidate());
http_fetch_params = new_http_fetch_params;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 3. Let includeCredentials be true if one of
if (
// - request’s credentials mode is "include"
request->credentials_mode() == Infrastructure::Request::CredentialsMode::Include
// - request’s credentials mode is "same-origin" and request’s response tainting is "basic"
|| (request->credentials_mode() == Infrastructure::Request::CredentialsMode::SameOrigin
&& request->response_tainting() == Infrastructure::Request::ResponseTainting::Basic)
// is true; otherwise false.
) {
include_credentials = IncludeCredentials::Yes;
} else {
include_credentials = IncludeCredentials::No;
}
// 4. If Cross-Origin-Embedder-Policy allows credentials with request returns false, then set
// includeCredentials to false.
if (!request->cross_origin_embedder_policy_allows_credentials())
include_credentials = IncludeCredentials::No;
// 5. Let contentLength be httpRequest’s body’s length, if httpRequest’s body is non-null; otherwise null.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto content_length = http_request->body().has<GC::Ref<Infrastructure::Body>>()
? http_request->body().get<GC::Ref<Infrastructure::Body>>()->length()
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
: Optional<u64> {};
// 6. Let contentLengthHeaderValue be null.
auto content_length_header_value = Optional<ByteBuffer> {};
// 7. If httpRequest’s body is null and httpRequest’s method is `POST` or `PUT`, then set
// contentLengthHeaderValue to `0`.
if (http_request->body().has<Empty>() && StringView { http_request->method() }.is_one_of("POST"sv, "PUT"sv))
content_length_header_value = MUST(ByteBuffer::copy("0"sv.bytes()));
// 8. If contentLength is non-null, then set contentLengthHeaderValue to contentLength, serialized and
// isomorphic encoded.
if (content_length.has_value())
AK: Make String::number() infallible This API will always succeed in creating a String representing the provided number in base-10.
2024-10-14 10:05:01 +02:00
content_length_header_value = MUST(ByteBuffer::copy(String::number(*content_length).bytes()));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 9. If contentLengthHeaderValue is non-null, then append (`Content-Length`, contentLengthHeaderValue) to
// httpRequest’s header list.
if (content_length_header_value.has_value()) {
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("Content-Length"sv.bytes())),
.value = content_length_header_value.release_value(),
};
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Conform HTTP-network-or-cache to specs Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the fetch specs.
2024-07-18 21:31:42 +01:00
// 10. If contentLength is non-null and httpRequest’s keepalive is true, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (content_length.has_value() && http_request->keepalive()) {
LibWeb: Conform HTTP-network-or-cache to specs Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the fetch specs.
2024-07-18 21:31:42 +01:00
// 1. Let inflightKeepaliveBytes be 0.
u64 inflight_keep_alive_bytes = 0;
// 2. Let group be httpRequest’s client’s fetch group.
auto& group = http_request->client()->fetch_group();
// 3. Let inflightRecords be the set of fetch records in group whose request’s keepalive is true and done flag is unset.
LibWeb: Make ESO "fetch group" weakly reference the fetch records Otherwise we end up holding on to every fetch record indefinitely. Found by analyzing GC heap graphs on Discord.
2025-07-29 21:23:30 +02:00
GC::RootVector<GC::Ref<Infrastructure::FetchRecord>> in_flight_records(vm.heap());
for (auto& fetch_record : group) {
if (fetch_record.request()->keepalive() && !fetch_record.request()->done())
LibWeb: Conform HTTP-network-or-cache to specs Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the fetch specs.
2024-07-18 21:31:42 +01:00
in_flight_records.append(fetch_record);
}
// 4. For each fetchRecord of inflightRecords:
for (auto const& fetch_record : in_flight_records) {
// 1. Let inflightRequest be fetchRecord’s request.
auto const& in_flight_request = fetch_record->request();
// 2. Increment inflightKeepaliveBytes by inflightRequest’s body’s length.
inflight_keep_alive_bytes += in_flight_request->body().visit(
[](Empty) -> u64 { return 0; },
[](ByteBuffer const& buffer) -> u64 { return buffer.size(); },
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
[](GC::Ref<Infrastructure::Body> body) -> u64 {
LibWeb: Conform HTTP-network-or-cache to specs Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the fetch specs.
2024-07-18 21:31:42 +01:00
return body->length().has_value() ? body->length().value() : 0;
});
}
// 5. If the sum of contentLength and inflightKeepaliveBytes is greater than 64 kibibytes, then return a network error.
LibWeb/Fetch: Add a constant for the keepalive maximum size
2024-08-11 12:42:53 +01:00
if ((content_length.value() + inflight_keep_alive_bytes) > keepalive_maximum_size)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Keepalive request exceeded maximum allowed size of 64 KiB"_string));
LibWeb: Conform HTTP-network-or-cache to specs Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the fetch specs.
2024-07-18 21:31:42 +01:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: The above limit ensures that requests that are allowed to outlive the environment settings object
// and contain a body, have a bounded size and are not allowed to stay alive indefinitely.
}
// 11. If httpRequest’s referrer is a URL, then:
AK+LibURL: Move AK::URL into a new URL library This URL library ends up being a relatively fundamental base library of the system, as LibCore depends on LibURL. This change has two main benefits: * Moving AK back more towards being an agnostic library that can be used between the kernel and userspace. URL has never really fit that description - and is not used in the kernel. * URL _should_ depend on LibUnicode, as it needs punnycode support. However, it's not really possible to do this inside of AK as it can't depend on any external library. This change brings us a little closer to being able to do that, but unfortunately we aren't there quite yet, as the code generators depend on LibCore.
2024-03-18 16:22:27 +13:00
if (http_request->referrer().has<URL::URL>()) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. Let referrerValue be httpRequest’s referrer, serialized and isomorphic encoded.
Userland: Avoid some conversions from rvalue strings to StringView These are all actually fine, there is no UAF here. But once e.g. `ByteString::view() &&` is deleted, these instances won't compile.
2024-04-03 21:51:34 -04:00
auto referrer_string = http_request->referrer().get<URL::URL>().serialize();
auto referrer_value = TRY_OR_THROW_OOM(vm, ByteBuffer::copy(referrer_string.bytes()));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Append (`Referer`, referrerValue) to httpRequest’s header list.
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("Referer"sv.bytes())),
.value = move(referrer_value),
};
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 12. Append a request `Origin` header for httpRequest.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Requests
2024-04-26 13:35:10 -04:00
http_request->add_origin_header();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb/Fetch: Append the Fetch metadata headers
2024-04-29 20:57:34 +01:00
// 13. Append the Fetch metadata headers for httpRequest.
append_fetch_metadata_headers_for_request(*http_request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 14. FIXME If httpRequest’s initiator is "prefetch", then set a structured field value
// given (`Sec-Purpose`, the token prefetch) in httpRequest’s header list.
// 15. If httpRequest’s header list does not contain `User-Agent`, then user agents should append
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// (`User-Agent`, default `User-Agent` value) to httpRequest’s header list.
if (!http_request->header_list()->contains("User-Agent"sv.bytes())) {
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("User-Agent"sv.bytes())),
LibWeb: Honor User-Agent spoofing in Fetch headers This makes spoofing consistent between legacy ResourceLoader loads, Fetch loads, and the JavaScript `navigator` APIs.
2023-12-27 11:43:14 +01:00
.value = Infrastructure::default_user_agent_value(),
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
};
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 16. If httpRequest’s cache mode is "default" and httpRequest’s header list contains `If-Modified-Since`,
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// `If-None-Match`, `If-Unmodified-Since`, `If-Match`, or `If-Range`, then set httpRequest’s cache mode to
// "no-store".
if (http_request->cache_mode() == Infrastructure::Request::CacheMode::Default
&& (http_request->header_list()->contains("If-Modified-Since"sv.bytes())
|| http_request->header_list()->contains("If-None-Match"sv.bytes())
|| http_request->header_list()->contains("If-Unmodified-Since"sv.bytes())
|| http_request->header_list()->contains("If-Match"sv.bytes())
|| http_request->header_list()->contains("If-Range"sv.bytes()))) {
http_request->set_cache_mode(Infrastructure::Request::CacheMode::NoStore);
}
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 17. If httpRequest’s cache mode is "no-cache", httpRequest’s prevent no-cache cache-control header
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// modification flag is unset, and httpRequest’s header list does not contain `Cache-Control`, then append
// (`Cache-Control`, `max-age=0`) to httpRequest’s header list.
if (http_request->cache_mode() == Infrastructure::Request::CacheMode::NoCache
&& !http_request->prevent_no_cache_cache_control_header_modification()
&& !http_request->header_list()->contains("Cache-Control"sv.bytes())) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Cache-Control"sv, "max-age=0"sv);
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 18. If httpRequest’s cache mode is "no-store" or "reload", then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (http_request->cache_mode() == Infrastructure::Request::CacheMode::NoStore
|| http_request->cache_mode() == Infrastructure::Request::CacheMode::Reload) {
// 1. If httpRequest’s header list does not contain `Pragma`, then append (`Pragma`, `no-cache`) to
// httpRequest’s header list.
if (!http_request->header_list()->contains("Pragma"sv.bytes())) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Pragma"sv, "no-cache"sv);
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 2. If httpRequest’s header list does not contain `Cache-Control`, then append
// (`Cache-Control`, `no-cache`) to httpRequest’s header list.
if (!http_request->header_list()->contains("Cache-Control"sv.bytes())) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Cache-Control"sv, "no-cache"sv);
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 19. If httpRequest’s header list contains `Range`, then append (`Accept-Encoding`, `identity`) to
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// httpRequest’s header list.
// NOTE: This avoids a failure when handling content codings with a part of an encoded response.
// Additionally, many servers mistakenly ignore `Range` headers if a non-identity encoding is accepted.
if (http_request->header_list()->contains("Range"sv.bytes())) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Accept-Encoding"sv, "identity"sv);
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 20. Modify httpRequest’s header list per HTTP. Do not append a given header if httpRequest’s header list
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// contains that header’s name.
// NOTE: It would be great if we could make this more normative somehow. At this point headers such as
// `Accept-Encoding`, `Connection`, `DNT`, and `Host`, are to be appended if necessary.
// `Accept`, `Accept-Charset`, and `Accept-Language` must not be included at this point.
// NOTE: `Accept` and `Accept-Language` are already included (unless fetch() is used, which does not include
// the latter by default), and `Accept-Charset` is a waste of bytes. See HTTP header layer division for
// more details.
LibWeb+LibWebView+WebContent: Replace DNT with GPC Global Privacy Control aims to be a replacement for Do Not Track. DNT ended up not being a great solution, as it wasn't enforced by law. This actually resulted in the DNT header serving as an extra fingerprinting data point. GPC is becoming enforced by law in USA states such as California and Colorado. CA is further working on a bill which requires that browsers implement such an opt-out preference signal (OOPS): https://cppa.ca.gov/announcements/2025/20250911.html This patch replaces DNT with GPC and hooks up the associated settings.
2025-04-02 09:30:34 -04:00
//
// https://w3c.github.io/gpc/#the-sec-gpc-header-field-for-http-requests
if (ResourceLoader::the().enable_global_privacy_control() && !http_request->header_list()->contains("Sec-GPC"sv.bytes())) {
auto header = Infrastructure::Header::from_string_pair("Sec-GPC"sv, "1"sv);
LibWeb: Support sending DNT header with requests
2024-07-02 20:29:43 +01:00
http_request->header_list()->append(move(header));
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 21. If includeCredentials is true, then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (include_credentials == IncludeCredentials::Yes) {
// 1. If the user agent is not configured to block cookies for httpRequest (see section 7 of [COOKIES]),
// then:
if (true) {
// 1. Let cookies be the result of running the "cookie-string" algorithm (see section 5.4 of [COOKIES])
// with the user agent’s cookie store and httpRequest’s current URL.
auto cookies = ([&] {
LibWeb: Receive cookies through principal_host_defined_page Previously we depended on an associated document on the ESO to get to the page, but Workers do not have documents. However, we can simply get to the page with `principal_host_defined_page`, removing the issue.
2025-09-07 15:31:25 +01:00
auto& page = Bindings::principal_host_defined_page(HTML::principal_realm(realm));
return page.client().page_did_request_cookie(http_request->current_url(), Cookie::Source::Http);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
})();
// 2. If cookies is not the empty string, then append (`Cookie`, cookies) to httpRequest’s header list.
if (!cookies.is_empty()) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Cookie"sv, cookies);
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
// 2. If httpRequest’s header list does not contain `Authorization`, then:
if (!http_request->header_list()->contains("Authorization"sv.bytes())) {
// 1. Let authorizationValue be null.
LibWeb/Fetch: Port infrastructure to new String
2023-03-02 23:26:35 +00:00
auto authorization_value = Optional<String> {};
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. If there’s an authentication entry for httpRequest and either httpRequest’s use-URL-credentials
// flag is unset or httpRequest’s current URL does not include credentials, then set
// authorizationValue to authentication entry.
if (false // FIXME: "If there’s an authentication entry for httpRequest"
&& (!http_request->use_url_credentials() || !http_request->current_url().includes_credentials())) {
// FIXME: "set authorizationValue to authentication entry."
}
// 3. Otherwise, if httpRequest’s current URL does include credentials and isAuthenticationFetch is
// true, set authorizationValue to httpRequest’s current URL, converted to an `Authorization` value.
else if (http_request->current_url().includes_credentials() && is_authentication_fetch == IsAuthenticationFetch::Yes) {
auto const& url = http_request->current_url();
LibURL+LibWeb: Do not percent decode in password/username getters Doing it is not part of the spec. Whenever needed, the spec will explicitly percent decode the username and password. This fixes some URL WPT tests.
2024-08-04 22:02:02 +12:00
auto payload = MUST(String::formatted("{}:{}", URL::percent_decode(url.username()), URL::percent_decode(url.password())));
LibWeb/Fetch: Port infrastructure to new String
2023-03-02 23:26:35 +00:00
authorization_value = TRY_OR_THROW_OOM(vm, encode_base64(payload.bytes()));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 4. If authorizationValue is non-null, then append (`Authorization`, authorizationValue) to
// httpRequest’s header list.
if (authorization_value.has_value()) {
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header = Infrastructure::Header::from_string_pair("Authorization"sv, *authorization_value);
http_request->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
}
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// FIXME: 22. If there’s a proxy-authentication entry, use it as appropriate.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: This intentionally does not depend on httpRequest’s credentials mode.
LibWeb: Start adding infrastructure for an HTTP Cache
2024-05-21 10:38:10 -06:00
// 23. Set httpCache to the result of determining the HTTP cache partition, given httpRequest.
http_cache = determine_the_http_cache_partition(*http_request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 24. If httpCache is null, then set httpRequest’s cache mode to "no-store".
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
if (!http_cache)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
http_request->set_cache_mode(Infrastructure::Request::CacheMode::NoStore);
LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec One step was added since we last visited this AO.
2024-05-21 09:17:23 -06:00
// 25. If httpRequest’s cache mode is neither "no-store" nor "reload", then:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (http_request->cache_mode() != Infrastructure::Request::CacheMode::NoStore
&& http_request->cache_mode() != Infrastructure::Request::CacheMode::Reload) {
LibWeb/Fetch: Remove redundant timingInfo variables This is an editorial change in the Fetch spec. See: https://github.com/whatwg/fetch/commit/e8b67b0
2022-12-07 19:16:37 +00:00
// 1. Set storedResponse to the result of selecting a response from the httpCache, possibly needing
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// validation, as per the "Constructing Responses from Caches" chapter of HTTP Caching [HTTP-CACHING],
// if any.
// NOTE: As mandated by HTTP, this still takes the `Vary` header into account.
LibWeb/Fetch: Return a cloned cached response body Otherwise we will fully read from the cached response and invalidate it's stream, invalidating it for the next time it is read from. Fixes a crash when reloading linegoup.lol after two reloads.
2024-12-26 14:58:27 +13:00
stored_response = http_cache->select_response(realm, http_request->current_url(), http_request->method(), *http_request->header_list(), initial_set_of_stored_responses);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. If storedResponse is non-null, then:
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
if (stored_response) {
LibWeb/Fetch: Implement logic to process a response from HTTP's cache Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-21 20:16:45 -04:00
// 1. If cache mode is "default", storedResponse is a stale-while-revalidate response,
// and httpRequest’s client is non-null, then:
if (http_request->cache_mode() == Infrastructure::Request::CacheMode::Default
&& stored_response->is_stale_while_revalidate()
&& http_request->client() != nullptr) {
// 1. Set response to storedResponse.
response = stored_response;
// 2. Set response’s cache state to "local".
response->set_cache_state(Infrastructure::Response::CacheState::Local);
// 3. Let revalidateRequest be a clone of request.
auto revalidate_request = request->clone(realm);
// 4. Set revalidateRequest’s cache mode set to "no-cache".
revalidate_request->set_cache_mode(Infrastructure::Request::CacheMode::NoCache);
// 5. Set revalidateRequest’s prevent no-cache cache-control header modification flag.
revalidate_request->set_prevent_no_cache_cache_control_header_modification(true);
// 6. Set revalidateRequest’s service-workers mode set to "none".
revalidate_request->set_service_workers_mode(Infrastructure::Request::ServiceWorkersMode::None);
// 7. In parallel, run main fetch given a new fetch params whose request is revalidateRequest.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
Platform::EventLoopPlugin::the().deferred_invoke(GC::create_function(realm.heap(), [&vm, &realm, revalidate_request, fetch_params = GC::Ref(fetch_params)] {
LibWeb/Fetch: Implement logic to process a response from HTTP's cache Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-21 20:16:45 -04:00
(void)main_fetch(realm, Infrastructure::FetchParams::create(vm, revalidate_request, fetch_params->timing_info()));
LibWeb: Make EventLoopPlugin::deferred_invoke take a HeapFunction
2024-10-31 02:39:29 +13:00
}));
LibWeb/Fetch: Implement logic to process a response from HTTP's cache Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-21 20:16:45 -04:00
}
// 2. Otherwise:
else {
// 1. If storedResponse is a stale response, then set the revalidatingFlag.
if (stored_response->is_stale())
revalidating_flag->set_value(true);
// 2. If the revalidatingFlag is set and httpRequest’s cache mode is neither "force-cache" nor "only-if-cached", then:
if (revalidating_flag->value()
&& http_request->cache_mode() != Infrastructure::Request::CacheMode::ForceCache
&& http_request->cache_mode() != Infrastructure::Request::CacheMode::OnlyIfCached) {
// 1. If storedResponse’s header list contains `ETag`, then append (`If-None-Match`, `ETag`'s value) to httpRequest’s header list.
if (auto etag = stored_response->header_list()->get("ETag"sv.bytes()); etag.has_value()) {
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
http_request->header_list()->append(Infrastructure::Header::from_string_pair("If-None-Match"sv, *etag));
LibWeb/Fetch: Implement logic to process a response from HTTP's cache Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-21 20:16:45 -04:00
}
// 2. If storedResponse’s header list contains `Last-Modified`, then append (`If-Modified-Since`, `Last-Modified`'s value) to httpRequest’s header list.
if (auto last_modified = stored_response->header_list()->get("Last-Modified"sv.bytes()); last_modified.has_value()) {
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
http_request->header_list()->append(Infrastructure::Header::from_string_pair("If-Modified-Since"sv, *last_modified));
LibWeb/Fetch: Implement logic to process a response from HTTP's cache Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-21 20:16:45 -04:00
}
}
// 3. Otherwise, set response to storedResponse and set response’s cache state to "local".
else {
response = stored_response;
response->set_cache_state(Infrastructure::Response::CacheState::Local);
}
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
}
// 9. If aborted, then return the appropriate network error for fetchParams.
if (aborted)
LibWeb/Fetch: Store Response error message as a String{,View} variant The majority of error strings are StringView literals, and it seems silly to require heap-allocating strings for these. This idea is stolen from a recent change in fd1fbad :^)
2023-03-03 18:02:43 +00:00
return PendingResponse::create(vm, request, Infrastructure::Response::appropriate_network_error(vm, fetch_params));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
GC::Ptr<PendingResponse> pending_forward_response;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 10. If response is null, then:
if (!response) {
// 1. If httpRequest’s cache mode is "only-if-cached", then return a network error.
if (http_request->cache_mode() == Infrastructure::Request::CacheMode::OnlyIfCached)
LibWeb: Do not store network errors as a StringView This is very clearly a very dangerous API to have, and was causing a crash on Linux as a result of a stack use-after-free when visiting https://www.index.hr/. Fixes #3901
2025-04-02 20:51:45 +13:00
return PendingResponse::create(vm, request, Infrastructure::Response::network_error(vm, "Request with 'only-if-cached' cache mode doesn't have a cached response"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Let forwardResponse be the result of running HTTP-network fetch given httpFetchParams, includeCredentials,
// and isNewConnectionFetch.
LibWeb: Support loading file:// URLs via fetch (through ResourceLoader) This builds on the existing ad-hoc ResourceLoader code for HTTP fetches which works for files as well. This also includes a test that checks that stylesheets loaded with the "file" URL scheme actually work.
2023-03-22 23:56:11 +01:00
pending_forward_response = TRY(nonstandard_resource_loader_file_or_http_network_fetch(realm, *http_fetch_params, include_credentials, is_new_connection_fetch));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
} else {
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
pending_forward_response = PendingResponse::create(vm, request, Infrastructure::Response::create(vm));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
auto returned_pending_response = PendingResponse::create(vm, request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
pending_forward_response->when_loaded([&realm, &vm, &fetch_params, request, response, stored_response, initial_set_of_stored_responses, http_request, returned_pending_response, is_authentication_fetch, is_new_connection_fetch, revalidating_flag, include_credentials, response_was_null = !response, http_cache](GC::Ref<Infrastructure::Response> resolved_forward_response) mutable {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP-network-or-cache fetch' pending_forward_response load callback");
if (response_was_null) {
auto forward_response = resolved_forward_response;
// NOTE: TRACE is omitted as it is a forbidden method in Fetch.
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
auto method_is_unsafe = !(StringView { http_request->method() }.is_one_of("GET"sv, "HEAD"sv, "OPTIONS"sv));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. If httpRequest’s method is unsafe and forwardResponse’s status is in the range 200 to 399, inclusive,
// invalidate appropriate stored responses in httpCache, as per the "Invalidation" chapter of HTTP
// Caching, and set storedResponse to null.
if (method_is_unsafe && forward_response->status() >= 200 && forward_response->status() <= 399) {
// FIXME: "invalidate appropriate stored responses in httpCache, as per the "Invalidation" chapter of HTTP Caching"
stored_response = nullptr;
}
// 4. If the revalidatingFlag is set and forwardResponse’s status is 304, then:
if (revalidating_flag->value() && forward_response->status() == 304) {
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
dbgln("\033[34;1mHTTP CACHE REVALIDATE (304)\033[0m {}", http_request->current_url());
// 1. Update storedResponse’s header list using forwardResponse’s header list, as per the "Freshening
// Stored Responses upon Validation" chapter of HTTP Caching.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: This updates the stored response in cache as well.
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
http_cache->freshen_stored_responses_upon_validation(*forward_response, initial_set_of_stored_responses);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 2. Set response to storedResponse.
response = stored_response;
// 3. Set response’s cache state to "validated".
if (response)
response->set_cache_state(Infrastructure::Response::CacheState::Validated);
}
// 5. If response is null, then:
if (!response) {
// 1. Set response to forwardResponse.
response = forward_response;
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
// 2. Store httpRequest and forwardResponse in httpCache, as per the "Storing Responses in Caches" chapter of HTTP Caching.
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// NOTE: If forwardResponse is a network error, this effectively caches the network error, which is
// sometimes known as "negative caching".
// NOTE: The associated body info is stored in the cache alongside the response.
LibWeb: Implement very basic in-memory HTTP caching This patch adds a simple in-memory HTTP cache to each WebContent process. It's currently off by default (turn it on with --enable-http-cache) since the validation logic is lacking and incomplete.
2024-06-22 18:53:11 +02:00
if (http_cache)
LibWeb/Fetch: Do not clone stored responses Reading the RFC9111 spec makes it clear that the stored response was not intended to be cloned. This is because there is a "clone response" operation that is used in other places, but never for stored responses.
2024-10-22 11:47:22 +02:00
http_cache->store_response(realm, *http_request, *forward_response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
}
// 11. Set response’s URL list to a clone of httpRequest’s URL list.
response->set_url_list(http_request->url_list());
// 12. If httpRequest’s header list contains `Range`, then set response’s range-requested flag.
if (http_request->header_list()->contains("Range"sv.bytes()))
response->set_range_requested(true);
// 13. Set response’s request-includes-credentials to includeCredentials.
response->set_request_includes_credentials(include_credentials == IncludeCredentials::Yes);
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
auto inner_pending_response = PendingResponse::create(vm, request, *response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 14. If response’s status is 401, httpRequest’s response tainting is not "cors", includeCredentials is true,
LibWeb: Replace request's "window" with "traversable for user prompts" User prompts are not tied to specific Windows or the client's Window. They are tied to a traversable navigable (browser tab).
2025-08-07 10:54:40 +02:00
// and request’s traversable for user prompts is a traversable navigable:
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
if (response->status() == 401
&& http_request->response_tainting() != Infrastructure::Request::ResponseTainting::CORS
&& include_credentials == IncludeCredentials::Yes
LibWeb: Replace request's "window" with "traversable for user prompts" User prompts are not tied to specific Windows or the client's Window. They are tied to a traversable navigable (browser tab).
2025-08-07 10:54:40 +02:00
&& request->traversable_for_user_prompts().has<GC::Ptr<HTML::TraversableNavigable>>()
LibWeb: Check presence of `WWW-Authenticate` header in fetch response If a HTTP 401 response we get does not contain a `WWW-Authenticate` header, we should not trigger the logic to ask the user for credentials and retry the request. This part is hinted at in a TODO / 'Needs testing' remark in the spec but needs to be fleshed out. Raised an upstream issue to do so: https://github.com/whatwg/fetch/issues/1766 This fixes login forms triggering an infinite fetch loop when providing incorrect credentials. Co-Authored-By: Victor Tran <vicr12345@gmail.com>
2024-08-13 11:47:15 +02:00
// AD-HOC: Require at least one WWW-Authenticate header to be set before automatically retrying an authenticated
// request (see rule 1 below). See: https://github.com/whatwg/fetch/issues/1766
&& request->header_list()->contains("WWW-Authenticate"sv.bytes())) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 1. Needs testing: multiple `WWW-Authenticate` headers, missing, parsing issues.
// (Red box in the spec, no-op)
// 2. If request’s body is non-null, then:
if (!request->body().has<Empty>()) {
// 1. If request’s body’s source is null, then return a network error.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
if (request->body().get<GC::Ref<Infrastructure::Body>>()->source().has<Empty>()) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "Request has body but no body source"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
// 2. Set request’s body to the body of the result of safely extracting request’s body’s source.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto const& source = request->body().get<GC::Ref<Infrastructure::Body>>()->source();
LibWeb: Fix typo in BodyInitOrReadableBytes type alias
2022-11-09 22:37:07 +00:00
// NOTE: BodyInitOrReadableBytes is a superset of Body::SourceType
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto converted_source = source.has<ByteBuffer>()
LibWeb: Fix typo in BodyInitOrReadableBytes type alias
2022-11-09 22:37:07 +00:00
? BodyInitOrReadableBytes { source.get<ByteBuffer>() }
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
: BodyInitOrReadableBytes { source.get<GC::Root<FileAPI::Blob>>() };
LibWeb: Remove exception handling from safely extracting response bodies The entire purpose of this AO is to avoid handling exceptions, which we can do now that the underlying AOs do not throw exceptions on OOM.
2024-11-04 16:06:01 +01:00
auto [body, _] = safely_extract_body(realm, converted_source);
request->set_body(body);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 3. If request’s use-URL-credentials flag is unset or isAuthenticationFetch is true, then:
if (!request->use_url_credentials() || is_authentication_fetch == IsAuthenticationFetch::Yes) {
// 1. If fetchParams is canceled, then return the appropriate network error for fetchParams.
if (fetch_params.is_canceled()) {
LibWeb/Fetch: Store Response error message as a String{,View} variant The majority of error strings are StringView literals, and it seems silly to require heap-allocating strings for these. This idea is stolen from a recent change in fd1fbad :^)
2023-03-03 18:02:43 +00:00
returned_pending_response->resolve(Infrastructure::Response::appropriate_network_error(vm, fetch_params));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
// FIXME: 2. Let username and password be the result of prompting the end user for a username and password,
// respectively, in request’s window.
dbgln("Fetch: Username/password prompt is not implemented, using empty strings. This request will probably fail.");
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 17:49:34 +03:30
auto username = ByteString::empty();
auto password = ByteString::empty();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 3. Set the username given request’s current URL and username.
LibURL: Make percent_encode return a String This simplifies a bunch of places which were needing to error check and convert from a ByteString to String.
2024-08-10 13:12:19 +12:00
request->current_url().set_username(username);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 4. Set the password given request’s current URL and password.
LibURL: Make percent_encode return a String This simplifies a bunch of places which were needing to error check and convert from a ByteString to String.
2024-08-10 13:12:19 +12:00
request->current_url().set_password(password);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
// 4. Set response to the result of running HTTP-network-or-cache fetch given fetchParams and true.
inner_pending_response = TRY_OR_IGNORE(http_network_or_cache_fetch(realm, fetch_params, IsAuthenticationFetch::Yes));
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
inner_pending_response->when_loaded([&realm, &vm, &fetch_params, request, returned_pending_response, is_authentication_fetch, is_new_connection_fetch](GC::Ref<Infrastructure::Response> response) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'HTTP network-or-cache fetch' inner_pending_response load callback");
// 15. If response’s status is 407, then:
if (response->status() == 407) {
LibWeb: Replace request's "window" with "traversable for user prompts" User prompts are not tied to specific Windows or the client's Window. They are tied to a traversable navigable (browser tab).
2025-08-07 10:54:40 +02:00
// 1. If request’s traversable for user prompts is "no-traversable", then return a network error.
if (request->traversable_for_user_prompts().has<Infrastructure::Request::TraversableForUserPrompts>()
&& request->traversable_for_user_prompts().get<Infrastructure::Request::TraversableForUserPrompts>() == Infrastructure::Request::TraversableForUserPrompts::NoTraversable) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "Request requires proxy authentication but has 'no-window' set"_string));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
// 2. Needs testing: multiple `Proxy-Authenticate` headers, missing, parsing issues.
// (Red box in the spec, no-op)
// 3. If fetchParams is canceled, then return the appropriate network error for fetchParams.
if (fetch_params.is_canceled()) {
LibWeb/Fetch: Store Response error message as a String{,View} variant The majority of error strings are StringView literals, and it seems silly to require heap-allocating strings for these. This idea is stolen from a recent change in fd1fbad :^)
2023-03-03 18:02:43 +00:00
returned_pending_response->resolve(Infrastructure::Response::appropriate_network_error(vm, fetch_params));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
// FIXME: 4. Prompt the end user as appropriate in request’s window and store the result as a
// proxy-authentication entry.
// NOTE: Remaining details surrounding proxy authentication are defined by HTTP.
// FIXME: 5. Set response to the result of running HTTP-network-or-cache fetch given fetchParams.
// (Doing this without step 4 would potentially lead to an infinite request cycle.)
}
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
auto inner_pending_response = PendingResponse::create(vm, request, *response);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 16. If all of the following are true
if (
// - response’s status is 421
response->status() == 421
// - isNewConnectionFetch is false
&& is_new_connection_fetch == IsNewConnectionFetch::No
// - request’s body is null, or request’s body is non-null and request’s body’s source is non-null
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
&& (request->body().has<Empty>() || !request->body().get<GC::Ref<Infrastructure::Body>>()->source().has<Empty>())
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// then:
) {
// 1. If fetchParams is canceled, then return the appropriate network error for fetchParams.
if (fetch_params.is_canceled()) {
LibWeb/Fetch: Store Response error message as a String{,View} variant The majority of error strings are StringView literals, and it seems silly to require heap-allocating strings for these. This idea is stolen from a recent change in fd1fbad :^)
2023-03-03 18:02:43 +00:00
returned_pending_response->resolve(Infrastructure::Response::appropriate_network_error(vm, fetch_params));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return;
}
// 2. Set response to the result of running HTTP-network-or-cache fetch given fetchParams,
// isAuthenticationFetch, and true.
inner_pending_response = TRY_OR_IGNORE(http_network_or_cache_fetch(realm, fetch_params, is_authentication_fetch, IsNewConnectionFetch::Yes));
}
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
inner_pending_response->when_loaded([returned_pending_response, is_authentication_fetch](GC::Ref<Infrastructure::Response> response) {
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
// 17. If isAuthenticationFetch is true, then create an authentication entry for request and the given
// realm.
if (is_authentication_fetch == IsAuthenticationFetch::Yes) {
// FIXME: "create an authentication entry for request and the given realm"
}
returned_pending_response->resolve(response);
});
});
});
// 18. Return response.
// NOTE: Typically response’s body’s stream is still being enqueued to after returning.
return returned_pending_response;
}
#if defined(WEB_FETCH_DEBUG)
static void log_load_request(auto const& load_request)
{
dbgln("Fetch: Invoking ResourceLoader");
dbgln("> {} {} HTTP/1.1", load_request.method(), load_request.url());
for (auto const& [name, value] : load_request.headers())
dbgln("> {}: {}", name, value);
dbgln(">");
for (auto line : StringView { load_request.body() }.split_view('\n', SplitBehavior::KeepEmpty))
dbgln("> {}", line);
}
static void log_response(auto const& status_code, auto const& headers, auto const& data)
{
dbgln("< HTTP/1.1 {}", status_code.value_or(0));
Fetch: Make sure we iterate over HeaderMap's headers() This fixes a build failure when built with CMake option '-DENABLE_ALL_THE_DEBUG_MACROS=ON'.
2024-06-09 19:50:23 +02:00
for (auto const& [name, value] : headers.headers())
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln("< {}: {}", name, value);
dbgln("<");
for (auto line : StringView { data }.split_view('\n', SplitBehavior::KeepEmpty))
dbgln("< {}", line);
}
#endif
// https://fetch.spec.whatwg.org/#concept-http-network-fetch
// Drop-in replacement for 'HTTP-network fetch', but obviously non-standard :^)
LibWeb: Support loading file:// URLs via fetch (through ResourceLoader) This builds on the existing ad-hoc ResourceLoader code for HTTP fetches which works for files as well. This also includes a test that checks that stylesheets loaded with the "file" URL scheme actually work.
2023-03-22 23:56:11 +01:00
// It also handles file:// URLs since those can also go through ResourceLoader.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ref<PendingResponse>> nonstandard_resource_loader_file_or_http_network_fetch(JS::Realm& realm, Infrastructure::FetchParams const& fetch_params, IncludeCredentials include_credentials, IsNewConnectionFetch is_new_connection_fetch)
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'non-standard HTTP-network fetch' with: fetch_params @ {}", &fetch_params);
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
auto fetch_timing_info = fetch_params.timing_info();
auto cross_origin_isolated_capability = fetch_params.cross_origin_isolated_capability();
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto& vm = realm.vm();
(void)include_credentials;
(void)is_new_connection_fetch;
auto request = fetch_params.request();
LibWeb: Get Page from principal realm during resource loading Fixes a crash for module loading for a shadow realm.
2024-11-25 10:37:28 +13:00
auto& page = Bindings::principal_host_defined_page(HTML::principal_realm(realm));
LibWeb: Make sure that fetch() response cookies get saved We have to provide the Web::Page* in order for cookies to get saved.
2023-01-15 10:10:11 +01:00
LibWeb/Fetch: Don't add cookies when creating ResourceLoader request Using LoadRequest::create_for_url_on_page will unconditionally add cookies as long as there's a page available. However, it is up to http_network_or_cache_fetch to determine if cookies should be added to the request. This was noticed when implementing CORS-preflight requests, where we sent cookies in OPTIONS requests.
2023-02-08 23:29:16 +00:00
// NOTE: Using LoadRequest::create_for_url_on_page here will unconditionally add cookies as long as there's a page available.
// However, it is up to http_network_or_cache_fetch to determine if cookies should be added to the request.
LoadRequest load_request;
load_request.set_url(request->current_url());
LibWeb: Make HTML::Window::page() return a Page&
2023-12-15 20:33:16 +01:00
load_request.set_page(page);
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 17:49:34 +03:30
load_request.set_method(ByteString::copy(request->method()));
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
for (auto const& header : *request->header_list())
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 17:49:34 +03:30
load_request.set_header(ByteString::copy(header.name), ByteString::copy(header.value));
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
if (auto const* body = request->body().get_pointer<GC::Ref<Infrastructure::Body>>()) {
LibWeb: Make `Fetch::Infrastructure::Body` be GC allocated Making the body GC-allocated allows us to avoid using `JS::Handle` for `m_stream` in its members.
2023-08-18 19:38:13 +02:00
TRY((*body)->source().visit(
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
[&](ByteBuffer const& byte_buffer) -> WebIDL::ExceptionOr<void> {
LibJS+LibWeb: Move the macro to convert ENOMEM to an exception to LibJS Move the macro to LibJS and change it to return a throw completion instead of a WebIDL exception. This will let us use this macro within LibJS to handle OOM conditions.
2023-01-07 12:14:54 -05:00
load_request.set_body(TRY_OR_THROW_OOM(vm, ByteBuffer::copy(byte_buffer)));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return {};
},
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
[&](GC::Root<FileAPI::Blob> const& blob_handle) -> WebIDL::ExceptionOr<void> {
LibWeb: Implement Blob::bytes() Implements https://w3c.github.io/FileAPI/#dom-blob-bytes.
2024-07-23 23:48:01 -07:00
load_request.set_body(TRY_OR_THROW_OOM(vm, ByteBuffer::copy(blob_handle->raw_bytes())));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return {};
},
[](Empty) -> WebIDL::ExceptionOr<void> {
return {};
}));
}
LibWeb: Register PendingResponse with a Request to keep it alive This was an oversight from when I converted PendingResponse and various other classes from being ref-counted to GC-allocated last minute - no one takes care to keep all of them alive. Some are on the stack, and some might be captured in another PendingResponse's JS::SafeFunction, but ultimately, we need a better solution. Since a PendingResponse is *always* the result of someone having created a Request, let's just let that keep a list of each PendingResponse that has been created for it, and visit them until they are resolved. After that, they can be GC'd with no complaints.
2022-11-01 19:35:38 +00:00
auto pending_response = PendingResponse::create(vm, request);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
if constexpr (WEB_FETCH_DEBUG) {
dbgln("Fetch: Invoking ResourceLoader");
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
log_load_request(load_request);
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
}
// FIXME: This check should be removed and all HTTP requests should go through the `ResourceLoader::load_unbuffered`
// path. The buffer option should then be supplied to the steps below that allow us to buffer data up to a
// user-agent-defined limit (or not). However, we will need to fully use stream operations throughout the
// fetch process to enable this (e.g. Body::fully_read must use streams for this to work).
if (request->buffer_policy() == Infrastructure::Request::BufferPolicy::DoNotBufferResponse) {
LibWeb: Make TemporaryExecutionContext take a Realm&
2024-10-24 20:39:18 +13:00
HTML::TemporaryExecutionContext execution_context { realm, HTML::TemporaryExecutionContext::CallbacksEnabled::Yes };
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibWeb: Define stream variable before using it This is an editoral change from the fetch spec. Since we already defined the stream before it being used this only re-numbers the spec steps. It also corrects a minor typo ('followings' to 'following') which was corrected in the same editoral spec change.
2025-08-06 20:18:18 +02:00
// 10. Let stream be a new ReadableStream.
LibJS+LibWeb: Use realm.create<T> instead of heap.allocate<T> The main motivation behind this is to remove JS specifics of the Realm from the implementation of the Heap. As a side effect of this change, this is a bit nicer to read than the previous approach, and in my opinion, also makes it a little more clear that this method is specific to a JavaScript Realm.
2024-11-14 05:50:17 +13:00
auto stream = realm.create<Streams::ReadableStream>(realm);
auto fetched_data_receiver = realm.create<FetchedDataReceiver>(fetch_params, stream);
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibWeb: Define stream variable before using it This is an editoral change from the fetch spec. Since we already defined the stream before it being used this only re-numbers the spec steps. It also corrects a minor typo ('followings' to 'following') which was corrected in the same editoral spec change.
2025-08-06 20:18:18 +02:00
// 11. Let pullAlgorithm be the following steps:
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto pull_algorithm = GC::create_function(realm.heap(), [&realm, fetched_data_receiver]() {
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
// 1. Let promise be a new promise.
auto promise = WebIDL::create_promise(realm);
// 2. Run the following steps in parallel:
// NOTE: This is handled by FetchedDataReceiver.
fetched_data_receiver->set_pending_promise(promise);
// 3. Return promise.
return promise;
});
LibWeb: Define stream variable before using it This is an editoral change from the fetch spec. Since we already defined the stream before it being used this only re-numbers the spec steps. It also corrects a minor typo ('followings' to 'following') which was corrected in the same editoral spec change.
2025-08-06 20:18:18 +02:00
// 12. Let cancelAlgorithm be an algorithm that aborts fetchParams’s controller with reason, given reason.
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto cancel_algorithm = GC::create_function(realm.heap(), [&realm, &fetch_params](JS::Value reason) {
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
fetch_params.controller()->abort(realm, reason);
return WebIDL::create_resolved_promise(realm, JS::js_undefined());
});
// 13. Set up stream with byte reading support with pullAlgorithm set to pullAlgorithm, cancelAlgorithm set to cancelAlgorithm.
LibWeb/Streams: Move ReadableStream functions out of AbstractOperations These are not defined in the abstract operations section of the spec and are the publically exported Stream APIs exposed on ReadableStream.
2024-12-08 17:28:44 +13:00
stream->set_up_with_byte_reading_support(pull_algorithm, cancel_algorithm);
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto on_headers_received = GC::create_function(vm.heap(), [&vm, request, pending_response, stream](HTTP::HeaderMap const& response_headers, Optional<u32> status_code, Optional<String> const& reason_phrase) {
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
(void)request;
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
if (pending_response->is_resolved()) {
// RequestServer will send us the response headers twice, the second time being for HTTP trailers. This
// fetch algorithm is not interested in trailers, so just drop them here.
return;
}
auto response = Infrastructure::Response::create(vm);
response->set_status(status_code.value_or(200));
LibRequests+LibWeb: Propagate HTTP reason phrase
2024-10-23 16:46:26 -05:00
if (reason_phrase.has_value())
response->set_status_message(MUST(ByteBuffer::copy(reason_phrase.value().bytes())));
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
if constexpr (WEB_FETCH_DEBUG) {
dbgln("Fetch: ResourceLoader load for '{}' {}: (status {})",
request->url(),
Infrastructure::is_ok_status(response->status()) ? "complete"sv : "failed"sv,
response->status());
log_response(status_code, response_headers, ReadonlyBytes {});
}
LibHTTP+RequestServer: Add HTTP::HeaderMap and use for response headers Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...> everywhere, we now encapsulate this in a class. Even better, the new class also allows keeping track of multiple headers with the same name! This will make it possible for HTTP responses to actually retain all their headers on the perilous journey from RequestServer to LibWeb.
2024-06-09 11:28:37 +02:00
for (auto const& [name, value] : response_headers.headers()) {
LibWeb: Avoid re-encoding response headers isomorphic encoding a value that has already been encoded will result in garbage data. `response_headers` is already encoded in ISO-8859-1/latin1, we cannot use `from_string_pair`, as it triggers ISO-8859-1/latin1 encoding. Follow-up of https://github.com/LadybirdBrowser/ladybird/pull/1893
2024-12-12 10:26:41 -08:00
auto header = Infrastructure::Header::from_latin1_pair(name, value);
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
response->header_list()->append(move(header));
}
// 14. Set response’s body to a new body whose stream is stream.
response->set_body(Infrastructure::Body::create(vm, stream));
// 17. Return response.
// NOTE: Typically response’s body’s stream is still being enqueued to after returning.
pending_response->resolve(response);
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
});
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
// 16. Run these steps in parallel:
// FIXME: 1. Run these steps, but abort when fetchParams is canceled:
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
auto on_data_received = GC::create_function(vm.heap(), [fetched_data_receiver](ReadonlyBytes bytes) {
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
// 1. If one or more bytes have been transmitted from response’s message body, then:
if (!bytes.is_empty()) {
// 1. Let bytes be the transmitted bytes.
// FIXME: 2. Let codings be the result of extracting header list values given `Content-Encoding` and response’s header list.
// FIXME: 3. Increase response’s body info’s encoded size by bytes’s length.
// FIXME: 4. Set bytes to the result of handling content codings given codings and bytes.
// FIXME: 5. Increase response’s body info’s decoded size by bytes’s length.
// FIXME: 6. If bytes is failure, then terminate fetchParams’s controller.
// 7. Append bytes to buffer.
fetched_data_receiver->on_data_received(bytes);
// FIXME: 8. If the size of buffer is larger than an upper limit chosen by the user agent, ask the user agent
// to suspend the ongoing fetch.
}
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
});
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
RequestServer: Retrieve timing info from curl and pipe it to LibWeb This timing info will be used to create a PerformanceResourceTiming entry.
2025-02-26 13:28:21 +00:00
auto on_complete = GC::create_function(vm.heap(), [&vm, &realm, pending_response, stream](bool success, Requests::RequestTimingInfo const&, Optional<StringView> error_message) {
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
dbgln("FIXME: Implement on_complete timing info for unbuffered requests");
LibWeb: Make TemporaryExecutionContext take a Realm&
2024-10-24 20:39:18 +13:00
HTML::TemporaryExecutionContext execution_context { realm, HTML::TemporaryExecutionContext::CallbacksEnabled::Yes };
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
// 16.1.1.2. Otherwise, if the bytes transmission for response’s message body is done normally and stream is readable,
// then close stream, and abort these in-parallel steps.
if (success) {
if (stream->is_readable())
stream->close();
}
// 16.1.2.2. Otherwise, if stream is readable, error stream with a TypeError.
else {
auto error = MUST(String::formatted("Load failed: {}", error_message));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
if (stream->is_readable())
stream->error(JS::TypeError::create(realm, error));
if (!pending_response->is_resolved())
pending_response->resolve(Infrastructure::Response::network_error(vm, error));
}
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
});
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
ResourceLoader::the().load_unbuffered(load_request, on_headers_received, on_data_received, on_complete);
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
} else {
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
auto on_load_success = GC::create_function(vm.heap(), [&realm, &vm, request, pending_response, fetch_timing_info, cross_origin_isolated_capability](ReadonlyBytes data, Requests::RequestTimingInfo const& timing_info, HTTP::HeaderMap const& response_headers, Optional<u32> status_code, Optional<String> const& reason_phrase) {
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
(void)request;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: ResourceLoader load for '{}' complete", request->url());
if constexpr (WEB_FETCH_DEBUG)
log_response(status_code, response_headers, data);
LibWeb/Fetch: Port infrastructure to new String
2023-03-02 23:26:35 +00:00
auto [body, _] = TRY_OR_IGNORE(extract_body(realm, data));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto response = Infrastructure::Response::create(vm);
response->set_status(status_code.value_or(200));
response->set_body(move(body));
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
auto body_info = response->body_info();
body_info.encoded_size = timing_info.encoded_body_size;
body_info.decoded_size = data.size();
response->set_body_info(body_info);
LibHTTP+RequestServer: Add HTTP::HeaderMap and use for response headers Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...> everywhere, we now encapsulate this in a class. Even better, the new class also allows keeping track of multiple headers with the same name! This will make it possible for HTTP responses to actually retain all their headers on the perilous journey from RequestServer to LibWeb.
2024-06-09 11:28:37 +02:00
for (auto const& [name, value] : response_headers.headers()) {
LibWeb: Avoid re-encoding response headers isomorphic encoding a value that has already been encoded will result in garbage data. `response_headers` is already encoded in ISO-8859-1/latin1, we cannot use `from_string_pair`, as it triggers ISO-8859-1/latin1 encoding. Follow-up of https://github.com/LadybirdBrowser/ladybird/pull/1893
2024-12-12 10:26:41 -08:00
auto header = Infrastructure::Header::from_latin1_pair(name, value);
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
response->header_list()->append(move(header));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibRequests+LibWeb: Propagate HTTP reason phrase
2024-10-23 16:46:26 -05:00
if (reason_phrase.has_value())
response->set_status_message(MUST(ByteBuffer::copy(reason_phrase.value().bytes())));
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
fetch_timing_info->update_final_timings(timing_info, cross_origin_isolated_capability);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
pending_response->resolve(response);
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
});
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
auto on_load_error = GC::create_function(vm.heap(), [&realm, &vm, request, pending_response, fetch_timing_info, cross_origin_isolated_capability](ByteString const& error, Requests::RequestTimingInfo const& timing_info, Optional<u32> status_code, Optional<String> const& reason_phrase, ReadonlyBytes data, HTTP::HeaderMap const& response_headers) {
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
(void)request;
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: ResourceLoader load for '{}' failed: {} (status {})", request->url(), error, status_code.value_or(0));
LibWeb: Log failed Fetch responses when WEB_FETCH_DEBUG is enabled We do the same for successful responses. Very useful for debugging issues on live websites.
2024-03-13 16:09:27 -04:00
if constexpr (WEB_FETCH_DEBUG)
log_response(status_code, response_headers, data);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
auto response = Infrastructure::Response::create(vm);
// FIXME: This is ugly, ResourceLoader should tell us.
if (status_code.value_or(0) == 0) {
LibWeb/Fetch: Pass error from ResourceLoader into network_error
2024-06-18 15:27:06 +02:00
response = Infrastructure::Response::network_error(vm, TRY_OR_IGNORE(String::from_byte_string(error)));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
} else {
LibWeb: Ensure fetch errors set their response types/codes appropriately If we fail to set the response type to an error, calling code will think the fetch was successful. We also should not default to an error code of 200, which would also indicate success.
2023-04-04 16:10:19 -04:00
response->set_type(Infrastructure::Response::Type::Error);
response->set_status(status_code.value_or(400));
LibWeb/Fetch: Include body and headers in Response for failed requests Fixes https://github.com/SerenityOS/serenity/issues/21290
2023-10-02 19:29:15 +02:00
auto [body, _] = TRY_OR_IGNORE(extract_body(realm, data));
Revert "LibWeb: Mark body stream with a TypeError if the request failed" This reverts commit 4d0301d2d2abae9b8e665fd49023eb4054d34eb7. This caused /html/dom/reflection-embedded.html to massively regress.
2025-04-20 19:06:10 -04:00
response->set_body(move(body));
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
auto body_info = response->body_info();
body_info.encoded_size = timing_info.encoded_body_size;
body_info.decoded_size = data.size();
response->set_body_info(body_info);
LibHTTP+RequestServer: Add HTTP::HeaderMap and use for response headers Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...> everywhere, we now encapsulate this in a class. Even better, the new class also allows keeping track of multiple headers with the same name! This will make it possible for HTTP responses to actually retain all their headers on the perilous journey from RequestServer to LibWeb.
2024-06-09 11:28:37 +02:00
for (auto const& [name, value] : response_headers.headers()) {
LibWeb: Avoid re-encoding response headers isomorphic encoding a value that has already been encoded will result in garbage data. `response_headers` is already encoded in ISO-8859-1/latin1, we cannot use `from_string_pair`, as it triggers ISO-8859-1/latin1 encoding. Follow-up of https://github.com/LadybirdBrowser/ladybird/pull/1893
2024-12-12 10:26:41 -08:00
auto header = Infrastructure::Header::from_latin1_pair(name, value);
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
response->header_list()->append(move(header));
LibWeb/Fetch: Include body and headers in Response for failed requests Fixes https://github.com/SerenityOS/serenity/issues/21290
2023-10-02 19:29:15 +02:00
}
LibRequests+LibWeb: Propagate HTTP reason phrase
2024-10-23 16:46:26 -05:00
if (reason_phrase.has_value())
response->set_status_message(MUST(ByteBuffer::copy(reason_phrase.value().bytes())));
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
LibWeb/Fetch: Update timing info with the timings received from RS
2025-02-26 15:46:41 +00:00
fetch_timing_info->update_final_timings(timing_info, cross_origin_isolated_capability);
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
pending_response->resolve(response);
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
});
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
LibWeb: Make ResourceLoader callbacks HeapFunctions
2024-10-31 06:12:14 +13:00
ResourceLoader::the().load(load_request, on_load_success, on_load_error);
LibWeb: Support unbuffered fetch requests Supporting unbuffered fetches is actually part of the fetch spec in its HTTP-network-fetch algorithm. We had previously implemented this method in a very ad-hoc manner as a simple wrapper around ResourceLoader. This is still the case, but we now implement a good amount of these steps according to spec, using ResourceLoader's unbuffered API. The response data is forwarded through to the fetch response using streams. This will eventually let us remove the use of ResourceLoader's buffered API, as all responses should just be streamed this way. The streams spec then supplies ways to wait for completion, thus allowing fully buffered responses. However, we have more work to do to make the other parts of our fetch implementation (namely, Body::fully_read) use streams before we can do this.
2024-05-26 08:03:29 -04:00
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
return pending_response;
}
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// https://fetch.spec.whatwg.org/#cors-preflight-fetch-0
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
WebIDL::ExceptionOr<GC::Ref<PendingResponse>> cors_preflight_fetch(JS::Realm& realm, Infrastructure::Request& request)
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
{
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'CORS-preflight fetch' with request @ {}", &request);
auto& vm = realm.vm();
// 1. Let preflight be a new request whose method is `OPTIONS`, URL list is a clone of request’s URL list, initiator is
// request’s initiator, destination is request’s destination, origin is request’s origin, referrer is request’s referrer,
// referrer policy is request’s referrer policy, mode is "cors", and response tainting is "cors".
auto preflight = Fetch::Infrastructure::Request::create(vm);
preflight->set_method(TRY_OR_THROW_OOM(vm, ByteBuffer::copy("OPTIONS"sv.bytes())));
preflight->set_url_list(request.url_list());
preflight->set_initiator(request.initiator());
preflight->set_destination(request.destination());
preflight->set_origin(request.origin());
preflight->set_referrer(request.referrer());
preflight->set_referrer_policy(request.referrer_policy());
preflight->set_mode(Infrastructure::Request::Mode::CORS);
preflight->set_response_tainting(Infrastructure::Request::ResponseTainting::CORS);
// 2. Append (`Accept`, `*/*`) to preflight’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto temp_header = Infrastructure::Header::from_string_pair("Accept"sv, "*/*"sv);
preflight->header_list()->append(move(temp_header));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 3. Append (`Access-Control-Request-Method`, request’s method) to preflight’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
temp_header = Infrastructure::Header::from_string_pair("Access-Control-Request-Method"sv, request.method());
preflight->header_list()->append(move(temp_header));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 4. Let headers be the CORS-unsafe request-header names with request’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto headers = Infrastructure::get_cors_unsafe_header_names(request.header_list());
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 5. If headers is not empty, then:
if (!headers.is_empty()) {
// 1. Let value be the items in headers separated from each other by `,`.
// NOTE: This intentionally does not use combine, as 0x20 following 0x2C is not the way this was implemented,
// for better or worse.
ByteBuffer value;
bool first = true;
for (auto const& header : headers) {
if (!first)
TRY_OR_THROW_OOM(vm, value.try_append(','));
TRY_OR_THROW_OOM(vm, value.try_append(header));
first = false;
}
// 2. Append (`Access-Control-Request-Headers`, value) to preflight’s header list.
temp_header = Infrastructure::Header {
.name = TRY_OR_THROW_OOM(vm, ByteBuffer::copy("Access-Control-Request-Headers"sv.bytes())),
.value = move(value),
};
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
preflight->header_list()->append(move(temp_header));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
}
// 6. Let response be the result of running HTTP-network-or-cache fetch given a new fetch params whose request is preflight.
// FIXME: The spec doesn't say anything about timing_info here, but FetchParams requires a non-null FetchTimingInfo object.
auto timing_info = Infrastructure::FetchTimingInfo::create(vm);
auto fetch_params = Infrastructure::FetchParams::create(vm, preflight, timing_info);
auto returned_pending_response = PendingResponse::create(vm, request);
auto preflight_response = TRY(http_network_or_cache_fetch(realm, fetch_params));
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-15 04:01:23 +13:00
preflight_response->when_loaded([&vm, &request, returned_pending_response](GC::Ref<Infrastructure::Response> response) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
dbgln_if(WEB_FETCH_DEBUG, "Fetch: Running 'CORS-preflight fetch' preflight_response load callback");
// 7. If a CORS check for request and response returns success and response’s status is an ok status, then:
// NOTE: The CORS check is done on request rather than preflight to ensure the correct credentials mode is used.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Requests
2024-04-26 13:35:10 -04:00
if (cors_check(request, response) && Infrastructure::is_ok_status(response->status())) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 1. Let methods be the result of extracting header list values given `Access-Control-Allow-Methods` and response’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto methods_or_failure = Infrastructure::extract_header_list_values("Access-Control-Allow-Methods"sv.bytes(), response->header_list());
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 2. Let headerNames be the result of extracting header list values given `Access-Control-Allow-Headers` and
// response’s header list.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto header_names_or_failure = Infrastructure::extract_header_list_values("Access-Control-Allow-Headers"sv.bytes(), response->header_list());
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
// 3. If either methods or headerNames is failure, return a network error.
if (methods_or_failure.has<Infrastructure::ExtractHeaderParseFailure>()) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "The Access-Control-Allow-Methods in the CORS-preflight response is syntactically invalid"_string));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
if (header_names_or_failure.has<Infrastructure::ExtractHeaderParseFailure>()) {
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "The Access-Control-Allow-Headers in the CORS-preflight response is syntactically invalid"_string));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
// NOTE: We treat "methods_or_failure" being `Empty` as empty Vector here.
auto methods = methods_or_failure.has<Vector<ByteBuffer>>() ? methods_or_failure.get<Vector<ByteBuffer>>() : Vector<ByteBuffer> {};
// NOTE: We treat "header_names_or_failure" being `Empty` as empty Vector here.
auto header_names = header_names_or_failure.has<Vector<ByteBuffer>>() ? header_names_or_failure.get<Vector<ByteBuffer>>() : Vector<ByteBuffer> {};
// 4. If methods is null and request’s use-CORS-preflight flag is set, then set methods to a new list containing request’s method.
// NOTE: This ensures that a CORS-preflight fetch that happened due to request’s use-CORS-preflight flag being set is cached.
if (methods.is_empty() && request.use_cors_preflight())
methods = Vector { TRY_OR_IGNORE(ByteBuffer::copy(request.method())) };
// 5. If request’s method is not in methods, request’s method is not a CORS-safelisted method, and request’s credentials mode
// is "include" or methods does not contain `*`, then return a network error.
if (!methods.contains_slow(request.method()) && !Infrastructure::is_cors_safelisted_method(request.method())) {
if (request.credentials_mode() == Infrastructure::Request::CredentialsMode::Include) {
Everywhere: Remove `sv` suffix from format string literals This prevents the compile-time checks that would catch errors in the format invocation (which would usually lead to a runtime crash).
2025-04-08 13:50:50 -04:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, TRY_OR_IGNORE(String::formatted("Non-CORS-safelisted method '{}' not found in the CORS-preflight response's Access-Control-Allow-Methods header (the header may be missing). '*' is not allowed as the main request includes credentials.", StringView { request.method() }))));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
if (!methods.contains_slow("*"sv.bytes())) {
Everywhere: Remove `sv` suffix from format string literals This prevents the compile-time checks that would catch errors in the format invocation (which would usually lead to a runtime crash).
2025-04-08 13:50:50 -04:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, TRY_OR_IGNORE(String::formatted("Non-CORS-safelisted method '{}' not found in the CORS-preflight response's Access-Control-Allow-Methods header and there was no '*' entry. The header may be missing.", StringView { request.method() }))));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
}
// 6. If one of request’s header list’s names is a CORS non-wildcard request-header name and is not a byte-case-insensitive match
// for an item in headerNames, then return a network error.
for (auto const& header : *request.header_list()) {
if (Infrastructure::is_cors_non_wildcard_request_header_name(header.name)) {
bool is_in_header_names = false;
for (auto const& allowed_header_name : header_names) {
Everywhere: Rename equals_ignoring_case => equals_ignoring_ascii_case Let's make it clear that these functions deal with ASCII case only.
2023-03-10 08:48:54 +01:00
if (StringView { allowed_header_name }.equals_ignoring_ascii_case(header.name)) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
is_in_header_names = true;
break;
}
}
if (!is_in_header_names) {
Everywhere: Remove `sv` suffix from format string literals This prevents the compile-time checks that would catch errors in the format invocation (which would usually lead to a runtime crash).
2025-04-08 13:50:50 -04:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, TRY_OR_IGNORE(String::formatted("Main request contains the header '{}' that is not specified in the CORS-preflight response's Access-Control-Allow-Headers header (the header may be missing). '*' does not capture this header.", StringView { header.name }))));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
}
}
// 7. For each unsafeName of the CORS-unsafe request-header names with request’s header list, if unsafeName is not a
// byte-case-insensitive match for an item in headerNames and request’s credentials mode is "include" or headerNames
// does not contain `*`, return a network error.
LibWeb: Remove OOM propagation from Fetch::Infrastructure::Headers
2024-04-26 13:24:20 -04:00
auto unsafe_names = Infrastructure::get_cors_unsafe_header_names(request.header_list());
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
for (auto const& unsafe_name : unsafe_names) {
bool is_in_header_names = false;
for (auto const& header_name : header_names) {
Everywhere: Rename equals_ignoring_case => equals_ignoring_ascii_case Let's make it clear that these functions deal with ASCII case only.
2023-03-10 08:48:54 +01:00
if (StringView { unsafe_name }.equals_ignoring_ascii_case(header_name)) {
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
is_in_header_names = true;
break;
}
}
if (!is_in_header_names) {
if (request.credentials_mode() == Infrastructure::Request::CredentialsMode::Include) {
Everywhere: Remove `sv` suffix from format string literals This prevents the compile-time checks that would catch errors in the format invocation (which would usually lead to a runtime crash).
2025-04-08 13:50:50 -04:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, TRY_OR_IGNORE(String::formatted("CORS-unsafe request-header '{}' not found in the CORS-preflight response's Access-Control-Allow-Headers header (the header may be missing). '*' is not allowed as the main request includes credentials.", StringView { unsafe_name }))));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
if (!header_names.contains_slow("*"sv.bytes())) {
Everywhere: Remove `sv` suffix from format string literals This prevents the compile-time checks that would catch errors in the format invocation (which would usually lead to a runtime crash).
2025-04-08 13:50:50 -04:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, TRY_OR_IGNORE(String::formatted("CORS-unsafe request-header '{}' not found in the CORS-preflight response's Access-Control-Allow-Headers header and there was no '*' entry. The header may be missing.", StringView { unsafe_name }))));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
return;
}
}
}
// FIXME: 8. Let max-age be the result of extracting header list values given `Access-Control-Max-Age` and response’s header list.
// FIXME: 9. If max-age is failure or null, then set max-age to 5.
// FIXME: 10. If max-age is greater than an imposed limit on max-age, then set max-age to the imposed limit.
// 11. If the user agent does not provide for a cache, then return response.
// NOTE: Since we don't currently have a cache, this is always true.
returned_pending_response->resolve(response);
return;
// FIXME: 12. For each method in methods for which there is a method cache entry match using request, set matching entry’s max-age
// to max-age.
// FIXME: 13. For each method in methods for which there is no method cache entry match using request, create a new cache entry
// with request, max-age, method, and null.
// FIXME: 14. For each headerName in headerNames for which there is a header-name cache entry match using request, set matching
// entry’s max-age to max-age.
// FIXME: 15. For each headerName in headerNames for which there is no header-name cache entry match using request, create a
// new cache entry with request, max-age, null, and headerName.
// FIXME: 16. Return response.
}
// 8. Otherwise, return a network error.
AK: Make "foo"_string infallible Stop worrying about tiny OOMs. Work towards #20405.
2023-08-07 11:12:38 +02:00
returned_pending_response->resolve(Infrastructure::Response::network_error(vm, "CORS-preflight check failed"_string));
LibWeb/Fetch: Implement CORS preflight The main things missing is the CORS preflight cache and making extract_header_list_values properly parse, validate and return split values for the Access-Control headers.
2023-02-08 23:35:52 +00:00
});
return returned_pending_response;
}
LibWeb/Fetch: Implement the "set the Sec-Fetch-Dest header" AO
2024-04-29 20:54:14 +01:00
// https://w3c.github.io/webappsec-fetch-metadata/#abstract-opdef-set-dest
void set_sec_fetch_dest_header(Infrastructure::Request& request)
{
// 1. Assert: r’s url is a potentially trustworthy URL.
VERIFY(SecureContexts::is_url_potentially_trustworthy(request.url()) == SecureContexts::Trustworthiness::PotentiallyTrustworthy);
// 2. Let header be a Structured Header whose value is a token.
// FIXME: This is handled below, as Serenity doesn't have APIs for RFC 8941.
// 3. If r’s destination is the empty string, set header’s value to the string "empty". Otherwise, set header’s value to r’s destination.
ByteBuffer header_value;
if (!request.destination().has_value()) {
header_value = MUST(ByteBuffer::copy("empty"sv.bytes()));
} else {
header_value = MUST(ByteBuffer::copy(Infrastructure::request_destination_to_string(request.destination().value()).bytes()));
}
// 4. Set a structured field value `Sec-Fetch-Dest`/header in r’s header list.
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("Sec-Fetch-Dest"sv.bytes())),
.value = move(header_value),
};
request.header_list()->append(move(header));
}
LibWeb/Fetch: Implement the "set the Sec-Fetch-Mode header" AO
2024-04-29 20:55:44 +01:00
// https://w3c.github.io/webappsec-fetch-metadata/#abstract-opdef-set-dest
void set_sec_fetch_mode_header(Infrastructure::Request& request)
{
// 1. Assert: r’s url is a potentially trustworthy URL.
VERIFY(SecureContexts::is_url_potentially_trustworthy(request.url()) == SecureContexts::Trustworthiness::PotentiallyTrustworthy);
// 2. Let header be a Structured Header whose value is a token.
// FIXME: This is handled below, as Serenity doesn't have APIs for RFC 8941.
// 3. Set header’s value to r’s mode.
auto header_value = MUST(ByteBuffer::copy(Infrastructure::request_mode_to_string(request.mode()).bytes()));
// 4. Set a structured field value `Sec-Fetch-Mode`/header in r’s header list.
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("Sec-Fetch-Mode"sv.bytes())),
.value = move(header_value),
};
request.header_list()->append(move(header));
}
LibWeb/Fetch: Implement the "set the Sec-Fetch-Site header" AO
2024-04-29 20:56:30 +01:00
// https://w3c.github.io/webappsec-fetch-metadata/#abstract-opdef-set-site
void set_sec_fetch_site_header(Infrastructure::Request& request)
{
// 1. Assert: r’s url is a potentially trustworthy URL.
VERIFY(SecureContexts::is_url_potentially_trustworthy(request.url()) == SecureContexts::Trustworthiness::PotentiallyTrustworthy);
// 2. Let header be a Structured Header whose value is a token.
// FIXME: This is handled below, as Serenity doesn't have APIs for RFC 8941.
// 3. Set header’s value to same-origin.
auto header_value = "same-origin"sv;
// FIXME: 4. If r is a navigation request that was explicitly caused by a user’s interaction with the user agent (by typing an address
// into the user agent directly, for example, or by clicking a bookmark, etc.), then set header’s value to none.
// 5. If header’s value is not none, then for each url in r’s url list:
if (!header_value.equals_ignoring_ascii_case("none"sv)) {
LibWeb: Set Sec-Fetch-Site header to same-site where appropriate This also fixes it looking at the request's current URL origin instead of the request's actual origin.
2025-01-28 19:10:32 +00:00
VERIFY(request.origin().has<URL::Origin>());
auto& request_origin = request.origin().get<URL::Origin>();
LibWeb/Fetch: Implement the "set the Sec-Fetch-Site header" AO
2024-04-29 20:56:30 +01:00
for (auto& url : request.url_list()) {
// 1. If url is same origin with r’s origin, continue.
LibWeb: Set Sec-Fetch-Site header to same-site where appropriate This also fixes it looking at the request's current URL origin instead of the request's actual origin.
2025-01-28 19:10:32 +00:00
if (url.origin().is_same_origin(request_origin))
LibWeb/Fetch: Implement the "set the Sec-Fetch-Site header" AO
2024-04-29 20:56:30 +01:00
continue;
// 2. Set header’s value to cross-site.
header_value = "cross-site"sv;
LibWeb: Set Sec-Fetch-Site header to same-site where appropriate This also fixes it looking at the request's current URL origin instead of the request's actual origin.
2025-01-28 19:10:32 +00:00
// 3. If r’s origin is not same site with url’s origin, then break.
if (!request_origin.is_same_site(url.origin()))
break;
LibWeb/Fetch: Implement the "set the Sec-Fetch-Site header" AO
2024-04-29 20:56:30 +01:00
LibWeb: Set Sec-Fetch-Site header to same-site where appropriate This also fixes it looking at the request's current URL origin instead of the request's actual origin.
2025-01-28 19:10:32 +00:00
// 4. Set header’s value to same-site.
header_value = "same-site"sv;
LibWeb/Fetch: Implement the "set the Sec-Fetch-Site header" AO
2024-04-29 20:56:30 +01:00
}
}
// 6. Set a structured field value `Sec-Fetch-Site`/header in r’s header list.
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("Sec-Fetch-Site"sv.bytes())),
.value = MUST(ByteBuffer::copy(header_value.bytes())),
};
request.header_list()->append(move(header));
}
LibWeb/Fetch: Implement the "set the Sec-Fetch-User header" AO
2024-04-29 20:56:57 +01:00
// https://w3c.github.io/webappsec-fetch-metadata/#abstract-opdef-set-user
void set_sec_fetch_user_header(Infrastructure::Request& request)
{
// 1. Assert: r’s url is a potentially trustworthy URL.
VERIFY(SecureContexts::is_url_potentially_trustworthy(request.url()) == SecureContexts::Trustworthiness::PotentiallyTrustworthy);
// 2. If r is not a navigation request, or if r’s user-activation is false, return.
if (!request.is_navigation_request() || !request.user_activation())
return;
// 3. Let header be a Structured Header whose value is a token.
// FIXME: This is handled below, as Serenity doesn't have APIs for RFC 8941.
// 4. Set header’s value to true.
// NOTE: See https://datatracker.ietf.org/doc/html/rfc8941#name-booleans for boolean format in RFC 8941.
auto header_value = MUST(ByteBuffer::copy("?1"sv.bytes()));
// 5. Set a structured field value `Sec-Fetch-User`/header in r’s header list.
auto header = Infrastructure::Header {
.name = MUST(ByteBuffer::copy("Sec-Fetch-User"sv.bytes())),
.value = move(header_value),
};
request.header_list()->append(move(header));
}
LibWeb/Fetch: Append the Fetch metadata headers
2024-04-29 20:57:34 +01:00
// https://w3c.github.io/webappsec-fetch-metadata/#abstract-opdef-append-the-fetch-metadata-headers-for-a-request
void append_fetch_metadata_headers_for_request(Infrastructure::Request& request)
{
// 1. If r’s url is not an potentially trustworthy URL, return.
if (SecureContexts::is_url_potentially_trustworthy(request.url()) != SecureContexts::Trustworthiness::PotentiallyTrustworthy)
return;
// 2. Set the Sec-Fetch-Dest header for r.
set_sec_fetch_dest_header(request);
// 3. Set the Sec-Fetch-Mode header for r.
set_sec_fetch_mode_header(request);
// 4. Set the Sec-Fetch-Site header for r.
set_sec_fetch_site_header(request);
// 5. Set the Sec-Fetch-User header for r.
set_sec_fetch_user_header(request);
}
WebContent: Enable in Windows CI
2025-06-28 03:32:38 -07:00
void set_http_cache_enabled(bool const enabled)
{
g_http_cache_enabled = enabled;
}
LibWeb: Implement most of the 'Fetching' AOs This implements the following operations from section 4 of the Fetch spec (https://fetch.spec.whatwg.org/#fetching): - Fetch - Main fetch - Fetch response handover - Scheme fetch - HTTP fetch - HTTP-redirect fetch - HTTP-network-or-cache fetch (without caching) It does *not* implement: - HTTP-network fetch - CORS-preflight fetch Instead, we let ResourceLoader handle the actual networking for now, which isn't ideal, but certainly enough to get enough functionality up and running for most websites to not complain.
2022-10-23 22:16:14 +01:00
}
Reference in a new issue Copy permalink