Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2026-04-19 10:20:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Libraries/LibImageDecoderClient/Client.h

66 lines
2.2 KiB
C
Raw Normal View History

ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
/*
Meta: Update my e-mail address everywhere
2024-10-04 13:19:50 +02:00
* Copyright (c) 2020, Andreas Kling <andreas@ladybird.org>
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 01:24:48 -07:00
* SPDX-License-Identifier: BSD-2-Clause
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
*/
#pragma once
#include <AK/HashMap.h>
#include <ImageDecoder/ImageDecoderClientEndpoint.h>
#include <ImageDecoder/ImageDecoderServerEndpoint.h>
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 14:23:16 -06:00
#include <LibCore/Promise.h>
LibGfx+LibWeb: Do some color management on images with an ICC profile This patch introduces the `Gfx::ColorSpace` class, this is basically a serializable wrapper for skia's SkColorSpace. Creation of the instances of this class (and thus ICC profiles parsing) is performed in the ImageDecoder process. Then the object is serialized and sent through IPC, to finally be handed to skia for rendering. However, to make sure that we're not making all LibGfx's users dependent on Skia as well, we need to ensure the `Gfx::ColorSpace` object has no dependency on objects from Skia. To that end, the only member of the `ColorSpace` class is the opaque `ColorSpaceImpl` struct. Though, there is on issue with that design, the code in `DisplayListPlayer.cpp` needs access to the underlying `sk_sp<SkColorSpace>`. It is provided by a template function, that is only specialized for this type. Doing this work allows us to pass the following WPT tests: - https://wpt.live/css/css-color/tagged-images-001.html - https://wpt.live/css/css-color/tagged-images-003.html - https://wpt.live/css/css-color/tagged-images-004.html - https://wpt.live/css/css-color/untagged-images-001.html Other test cases can also be found here: - https://github.com/svgeesus/PNG-ICC-tests Note that SkColorSpace support quite a limited amount of color spaces, so color profiles like the ones in [1] or the v4 profiles in [2] are not supported yet. In fact, SkColorSpace only accepts skcms_ICCProfile with a linear conversion to XYZ D50. [1] https://www.color.org/browsertest.xalter [2] https://www.color.org/version4html.xalter
2024-12-02 18:55:43 -05:00
#include <LibGfx/ColorSpace.h>
Userland: Rename IPC::ServerConnection=>IPC::ConnectionToServer This was done with CLion's automatic rename feature.
2022-02-25 12:27:37 +02:00
#include <LibIPC/ConnectionToServer.h>
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
namespace ImageDecoderClient {
ImageDecoder+LibImageDecoder+LibWeb: Support animations in ImageDecoder The ImageDecoder service now returns a list of image frames, each with a duration value. The code for in-process image decoding is removed from LibWeb, an all image decode requests are sent out-of-process to ImageDecoder. :^) This won't scale super well to very long and/or large animations, but we can work on improving that separately. The main goal here is simply to stop doing any image decoding inside LibWeb. Fixes #5165.
2021-01-29 22:30:48 +01:00
struct Frame {
Userland: Return empty if ImageDecoder client receives an invalid frame This simplifies error checking for all the users of the ImageDecoder client.
2023-10-01 19:56:19 +01:00
NonnullRefPtr<Gfx::Bitmap> bitmap;
ImageDecoder+LibImageDecoder+LibWeb: Support animations in ImageDecoder The ImageDecoder service now returns a list of image frames, each with a duration value. The code for in-process image decoding is removed from LibWeb, an all image decode requests are sent out-of-process to ImageDecoder. :^) This won't scale super well to very long and/or large animations, but we can work on improving that separately. The main goal here is simply to stop doing any image decoding inside LibWeb. Fixes #5165.
2021-01-29 22:30:48 +01:00
u32 duration { 0 };
};
struct DecodedImage {
bool is_animated { false };
ImageViewer: Transform the image's dimension accordingly to the metadata Exif metadata have two tags to store the pixel density along each axis. If both values are different and no action is taken, the resulting image will appear deformed. This commit scales the displayed bitmap accordingly to these tags in order to show the image in its intended shape. This unfortunately includes a lot of plumbing to get this information through IPC.
2024-02-14 01:15:06 -05:00
Gfx::FloatPoint scale { 1, 1 };
ImageDecoder+LibImageDecoder+LibWeb: Support animations in ImageDecoder The ImageDecoder service now returns a list of image frames, each with a duration value. The code for in-process image decoding is removed from LibWeb, an all image decode requests are sent out-of-process to ImageDecoder. :^) This won't scale super well to very long and/or large animations, but we can work on improving that separately. The main goal here is simply to stop doing any image decoding inside LibWeb. Fixes #5165.
2021-01-29 22:30:48 +01:00
u32 loop_count { 0 };
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
u32 frame_count { 0 };
ImageDecoder+LibImageDecoder+LibWeb: Support animations in ImageDecoder The ImageDecoder service now returns a list of image frames, each with a duration value. The code for in-process image decoding is removed from LibWeb, an all image decode requests are sent out-of-process to ImageDecoder. :^) This won't scale super well to very long and/or large animations, but we can work on improving that separately. The main goal here is simply to stop doing any image decoding inside LibWeb. Fixes #5165.
2021-01-29 22:30:48 +01:00
Vector<Frame> frames;
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
Vector<u32> all_durations;
LibGfx+LibWeb: Do some color management on images with an ICC profile This patch introduces the `Gfx::ColorSpace` class, this is basically a serializable wrapper for skia's SkColorSpace. Creation of the instances of this class (and thus ICC profiles parsing) is performed in the ImageDecoder process. Then the object is serialized and sent through IPC, to finally be handed to skia for rendering. However, to make sure that we're not making all LibGfx's users dependent on Skia as well, we need to ensure the `Gfx::ColorSpace` object has no dependency on objects from Skia. To that end, the only member of the `ColorSpace` class is the opaque `ColorSpaceImpl` struct. Though, there is on issue with that design, the code in `DisplayListPlayer.cpp` needs access to the underlying `sk_sp<SkColorSpace>`. It is provided by a template function, that is only specialized for this type. Doing this work allows us to pass the following WPT tests: - https://wpt.live/css/css-color/tagged-images-001.html - https://wpt.live/css/css-color/tagged-images-003.html - https://wpt.live/css/css-color/tagged-images-004.html - https://wpt.live/css/css-color/untagged-images-001.html Other test cases can also be found here: - https://github.com/svgeesus/PNG-ICC-tests Note that SkColorSpace support quite a limited amount of color spaces, so color profiles like the ones in [1] or the v4 profiles in [2] are not supported yet. In fact, SkColorSpace only accepts skcms_ICCProfile with a linear conversion to XYZ D50. [1] https://www.color.org/browsertest.xalter [2] https://www.color.org/version4html.xalter
2024-12-02 18:55:43 -05:00
Gfx::ColorSpace color_space;
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
i64 session_id { 0 };
ImageDecoder+LibImageDecoder+LibWeb: Support animations in ImageDecoder The ImageDecoder service now returns a list of image frames, each with a duration value. The code for in-process image decoding is removed from LibWeb, an all image decode requests are sent out-of-process to ImageDecoder. :^) This won't scale super well to very long and/or large animations, but we can work on improving that separately. The main goal here is simply to stop doing any image decoding inside LibWeb. Fixes #5165.
2021-01-29 22:30:48 +01:00
};
LibWeb+LibImageDecoderClient: Reuse ImageDecoder service process The overhead from spawning a new ImageDecoder for every decoding job is way too large and causing other problems as well (#5421) Let's keep the same decoder open and reuse it as long as it's working.
2021-02-20 11:35:00 +01:00
class Client final
Userland: Rename IPC::ServerConnection=>IPC::ConnectionToServer This was done with CLion's automatic rename feature.
2022-02-25 12:27:37 +02:00
: public IPC::ConnectionToServer<ImageDecoderClientEndpoint, ImageDecoderServerEndpoint>
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
, public ImageDecoderClientEndpoint {
LibIPC: Remove socket path from IPC Client connections We don't need these for Ladybird, and they are the only users of some LibCore functions we can remove as well.
2024-11-21 12:46:45 -07:00
C_OBJECT_ABSTRACT(Client);
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
public:
LibWebView+ImageDecoder+RequestServer+WebContent: Add init_transport
2025-01-03 21:19:46 +05:00
using InitTransport = Messages::ImageDecoderServer::InitTransport;
Everywhere: Make TransportSocket non-movable Instead of wrapping all non-movable members of TransportSocket in OwnPtr to keep it movable, make TransportSocket itself non-movable and wrap it in OwnPtr.
2025-04-08 22:01:46 +02:00
Client(NonnullOwnPtr<IPC::Transport>);
Ladybird: Decode images out of process This patch brings a service to handle image decompression. With it comes security enhancement due to the process boundary. Indeed, consequences of a potential attack is reduced as only the decoder will crash without perturbing the WebContent process. It also allows us to display pages containing images that we claim to support but still make us crash, like for not-finished-yet decoders. As an example, we can now load https://jpegxl.info/jxl-art.html without crashing the WebContent process.
2023-09-08 06:30:50 -04:00
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 14:23:16 -06:00
NonnullRefPtr<Core::Promise<DecodedImage>> decode_image(ReadonlyBytes, Function<ErrorOr<void>(DecodedImage&)> on_resolved, Function<void(Error&)> on_rejected, Optional<Gfx::IntSize> ideal_size = {}, Optional<ByteString> mime_type = {});
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
void request_animation_frames(i64 session_id, u32 start_frame_index, u32 count);
void stop_animation_decode(i64 session_id);
LibWeb+LibImageDecoderClient: Reuse ImageDecoder service process The overhead from spawning a new ImageDecoder for every decoding job is way too large and causing other problems as well (#5421) Let's keep the same decoder open and reuse it as long as it's working.
2021-02-20 11:35:00 +01:00
Function<void()> on_death;
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
Function<void(i64 session_id, Vector<NonnullRefPtr<Gfx::Bitmap>>)> on_animation_frames_decoded;
Function<void(i64 session_id, String error_message)> on_animation_decode_failed;
LibWeb+LibImageDecoderClient: Reuse ImageDecoder service process The overhead from spawning a new ImageDecoder for every decoding job is way too large and causing other problems as well (#5421) Let's keep the same decoder open and reuse it as long as it's working.
2021-02-20 11:35:00 +01:00
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
private:
LibWeb+LibImageDecoderClient: Reuse ImageDecoder service process The overhead from spawning a new ImageDecoder for every decoding job is way too large and causing other problems as well (#5421) Let's keep the same decoder open and reuse it as long as it's working.
2021-02-20 11:35:00 +01:00
virtual void die() override;
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 14:23:16 -06:00
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
virtual void did_decode_image(i64 image_id, bool is_animated, u32 loop_count, Gfx::BitmapSequence bitmap_sequence, Vector<u32> durations, Gfx::FloatPoint scale, Gfx::ColorSpace color_space, i64 session_id) override;
LibIPC+Everywhere: Always pass ownership of transferred data to clients This has been a longstanding ergonomic issue with our IPC compiler. Non- trivial types were previously passed by const&. So if we wanted to avoid expensive copies, we would have to const_cast and move the data. We now pass ownership of all transferred data to the client subclasses. This allows us to remove const_cast from these methods, and allows us to avoid some trivial expensive copies that we didn't bother to const_cast.
2025-03-08 12:22:39 -05:00
virtual void did_fail_to_decode_image(i64 image_id, String error_message) override;
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 14:23:16 -06:00
LibImageDecoderClient+LibWebView: Add animation session client API Implement the client side of streaming animation decode sessions. The ImageDecoderClient handles new IPC messages for frame delivery and failure reporting, and the ImageCodecPlugin bridges this through the platform abstraction layer used by LibWeb.
2026-02-13 15:03:45 +01:00
virtual void did_decode_animation_frames(i64 session_id, Gfx::BitmapSequence bitmaps) override;
virtual void did_fail_animation_decode(i64 session_id, String error_message) override;
ImageDecoder: Handle decoding of images in an asynchronous manner ImageDecoder now queues up image decoding requests and returns the images back to the caller later. ImageDecoderClient has a new promise-based API that allows callers to attach their own resolve/reject handlers to the responses from ImageDecoder.
2024-04-19 14:23:16 -06:00
HashMap<i64, NonnullRefPtr<Core::Promise<DecodedImage>>> m_pending_decoded_images;
ImageDecoder: Add a new service for out-of-process image decoding :^) The new ImageDecoder service (available for members of "image" via /tmp/portal/image) allows you to decode images in a separate process. This will allow programs to confidently load untrusted images, since the bulk of the security concerns are sandboxed to a separate process. The only API right now is a synchronous IPC DecodeImage() call that takes a shbuf with encoded image data and returns a shared buffer and metadata for the decoded image. It also comes with a very simple library for interfacing with the ImageDecoder service: LibImageDecoderClient. The name is a bit of a mouthful but I guess we can rename it later if we think of something nicer to call it. There's obviously a bit of overhead to spawning a separate process for every image decode, so this is mostly only appropriate for untrusted images (e.g stuff downloaded from the web) and not necessary for trusted local images (e.g stuff in /res)
2020-06-22 21:35:22 +02:00
};
}
Reference in a new issue Copy permalink