A MessagePort can be transferred while it already has local queued
state such as incoming messages drained from its transport,
outgoing messages posted before a transport exists, and a pending
shutdown to apply once the port is enabled.
Serialize and restore that state as part of transfer so it moves with
the port instead of being left behind on the old transport.
Also mark transports that are being transferred so shutdown of the old
endpoint during handoff is not reported as peer EOF. That shutdown is
part of moving the transport to the new owner, not peer disconnected.
Co-Authored-By: Alexander Kalenik <kalenik.aliaksandr@gmail.com>
On macOS, use Mach port messaging instead of Unix domain sockets for
all IPC transport. This makes the transport capable of carrying Mach
port rights as message attachments, which is a prerequisite for sending
IOSurface handles over the main IPC channel (currently sent via a
separate out-of-band path). It also avoids the need for the FD
acknowledgement protocol that TransportSocket requires, since Mach port
right transfers are atomic in the kernel.
Three connection establishment patterns:
- Spawned helper processes (WebContent, RequestServer, etc.) use the
existing MachPortServer: the child sends its task port with a reply
port, and the parent responds with a pre-created port pair.
- Socket-bootstrapped connections (WebDriver, BrowserProcess) exchange
Mach port names over the socket, then drop the socket.
- Pre-created pairs for IPC tests and in-message transport transfer.
Attachment on macOS now wraps a MachPort instead of a file descriptor,
converting between the two via fileport_makeport()/fileport_makefd().
The LibIPC socket transport tests are disabled on macOS since they are
socket-specific.
TransportSocket uses its own pipe-based notification mechanism on the IO
thread, making LocalSocket's built-in Core::Notifier redundant. When the
socket reaches EOF, this notifier is disabled from the IO thread. Since
the QSocketNotifier lives on the main thread, the its destruction is
deferred. If the socket is closed before the deferred destruction runs,
Qt detects the invalid socket on the next poll and prints:
QSocketNotifier: Invalid socket 50 and type 'Read', disabling...
Fix this by disabling the redundant socket-level notifier upfront in the
TransportSocket constructor.
Previously, `create_paired()` returned two full Transport objects, and
callers would immediately call `from_transport()` on the remote side to
extract its underlying fd. This wasted resources: the remote
Transport's IO thread, wakeup pipes, and send queue were initialized
only to be torn down without ever sending or receiving a message.
Now `create_paired()` returns `{Transport, TransportHandle}` — the
remote side is born as a lightweight handle containing just the raw fd,
skipping all unnecessary initialization.
Also replace `release_underlying_transport_for_transfer()` (which
returned a raw int fd) with `release_for_transfer()` (which returns a
TransportHandle directly), hiding the socket implementation detail
from callers including MessagePort.
Replace IPC::File / AutoCloseFileDescriptor / MessageFileType in
the IPC message pipeline with a new IPC::Attachment class. This
wraps a file descriptor transferred alongside IPC messages, and
provides a clean extension point for platform-specific transport
mechanisms (e.g., Mach ports on macOS) that will be introduced later.
Replace clone_from_transport() (which dup()s the FD) with
from_transport() (which releases the FD) in the WebWorkerClient
call site. The UI process never uses the WebWorkerClient connection
after spawning — it only passes the transport to WebContent — so
releasing instead of cloning is safe and simpler.
This removes clone_from_transport() from TransportHandle, and
clone_for_transfer() from TransportSocket/TransportSocketWindows,
as they no longer have any callers.
Consolidate the repeated socketpair + adopt + configure pattern from
4 call sites into a single Transport::create_paired() factory method.
This fixes inconsistent error handling and socket configuration across
call sites, and prepares for future mach port support on macOS.
We were allocating vectors on the heap and copying the message header's
and payload's bytes to it before passing them on to
`::enqueue_message()`.
Remove these allocations and just pass `ReadonlyBytes` views into the
message header and payload directly. On my machine, this reduces the
time spent on the send-side queuing path by 13% to 42%, depending on the
message size.
While we're processing received messages, we can end up with unprocessed
bytes after the last message. Instead of copying the data into a new
ByteBuffer, just move the bytes inside the existing buffer and resize
it.
This does mean that as long as there are unprocessed bytes after reading
incoming messages, the buffer does not shrink. But as soon as there's
nothing left, we clear this buffer again.
Previously, IPC messages were decoded on the main thread:
1. I/O thread received raw bytes and file descriptors
2. I/O thread stored them in a queue and notified main thread
3. Main thread decoded bytes into Message objects
4. Main thread processed the messages
Now, decoding happens on the I/O thread:
1. I/O thread receives raw bytes and file descriptors
2. I/O thread decodes them using a configurable MessageDecoder
3. I/O thread calls MessageHandler which stores decoded messages
4. I/O thread signals condition variable (for sync waiters)
5. I/O thread wakes main event loop via deferred_invoke()
6. Main thread processes already-decoded messages
This is achieved by:
- Adding MessageDecoder and MessageHandler callbacks to TransportSocket
- Connection template sets up the decoder (tries both endpoints)
- ConnectionBase::initialize_messaging() sets up the handler
- Storing a WeakEventLoopReference to wake the main thread
- Using mutex + condition variable for thread-safe queue access
- Sync message waiting now uses the CV directly instead of polling
The raw message API (read_as_many_messages_as_possible_without_blocking)
is preserved for MessagePort which uses its own decoding logic.
This architecture prepares for future multi-thread dispatch where
different message types could be routed to different handler threads
(e.g., scrolling messages to a dedicated scroll thread).
This consolidates the message size and FD count limits into a single
header file that can be used by both the encoding and decoding sides
of the IPC layer.
A malicious or misbehaving peer could send data faster than we process
it, causing unbounded memory growth in the unprocessed bytes buffer
and file descriptor queue.
Add MAX_UNPROCESSED_BUFFER_SIZE (128 MiB) and MAX_UNPROCESSED_FDS (512)
limits. When exceeded, the peer is disconnected gracefully rather than
allowing memory exhaustion.
Reject messages from peers that exceed reasonable limits:
- Maximum payload size: 64 MiB
- Maximum file descriptor count: 128
Also use checked arithmetic for message size calculations to prevent
integer overflow attacks.
These limits prevent malicious peers from causing excessive memory
allocation or resource exhaustion.
The message parsing loop performs arithmetic on payload_size (u32) and
index (size_t). While overflow is unlikely on 64-bit systems, use
Checked<size_t> to explicitly validate:
1. message_size = payload_size + sizeof(MessageHeader)
2. new_index = index + payload_size + sizeof(MessageHeader)
This prevents potential integer overflow attacks from malicious peers.
Replace crash-on-OOM patterns with graceful error handling:
- Use try_append() instead of append() for buffer operations
- Handle ByteBuffer::copy() failure instead of using MUST()
A malicious peer could send messages with large payload sizes to
trigger OOM conditions. Instead of crashing, we now disconnect
the misbehaving peer.
A malicious peer could exploit the file descriptor acknowledgement
protocol in several ways:
1. Send messages with fd_count values that overflow when accumulated
2. Send acknowledgements claiming more FDs were received than we sent
Both attacks would crash the recipient via VERIFY failures in
Queue::dequeue() or integer overflow.
Fix by using Checked<u32> for fd counters and validating the queue
isn't empty before dequeuing acknowledged FDs.
A misbehaving or compromised IPC peer (e.g. a WebContent process running
malicious JavaScript) could previously crash the UI process by sending
malformed messages. This was a problem: untrusted peers should only be
able to kill their own connection, not crash the process handling them.
Replace VERIFY_NOT_REACHED() and VERIFY() calls in error paths with
graceful disconnection using the existing shutdown infrastructure
(m_peer_eof, IOThreadState::Stopped, ShouldShutdown::Yes, etc.)
Affected error paths:
- Malformed IPC messages that fail to parse
- poll() errors and POLLERR/POLLNVAL conditions
- Unexpected send/receive errors on the socket
- Invalid message header types
- Protocol violations (e.g. bad FileDescriptorAcknowledgement)
All error paths now log via dbgln() for debugging before disconnecting.
The wakeup pipe is created with O_NONBLOCK, but MUST() was used for
reads. This could cause crashes on spurious wakeups when EAGAIN is
returned. Since the read is just to drain the pipe and wake the IO
thread, we can safely ignore any errors.
Previously, TransportSocket sent queued messages from a separate thread
but performed all reading on the main thread. With this change, both
reading and writing are handled on the same I/O thread. This would allow
us to read IPC messages even while the main thread is blocked and
process them on a different thread (e.g., a rendering thread).
This limitation of the underlying Unix socket implementation can
cause IPC failures on pages with tons of images and network requests.
Modify the code called from TransportSocket's send thread to limit the
number of fds to MAX_TRANSFER_FDS, and ensure that we will keep sending
as long as we have either bytes or file descriptors to send.
The spec isn't super clear on what disentagling a MessagePort means. But
we are required to send all pending messages before closing the port.
This is a bit tricky because the transport socket performs writes on a
background thread. From the main thread, where the disentanglement will
occur, we don't really know the state of the write thread. So what we do
here is stop the background thread then flush all remaining data from
the main thread.
* We need the full definition of IPC::File in the header.
* We need(ed) Core::System in the header. Move AutoCloseFileDescriptor's
ctor and dtor out-of-line to avoid this.
Memory stream is a more suitable container for the socket send queue,
as using it results in fewer allocations than trying to emulate a stream
using a Vector.
Bring back d6080d1fdc with a missing check
whether underlying socket is closed, before accessing `fd()` that is
optional and empty in case of closed socket.
With this change TransportSocket becomes capable of sending large
messages without relying on workarounds, such as sending the message as
a shared memory file descriptor when it can't fully fit into the socket
buffer.
It's implemented by combining all enqueued messages into two buffers:
one for bytes and another for fds, and repeatedly attempts to write them
in smaller chunks, waiting for the socket to become writable again if
the receiver needs time to consume the data.
Another significant improvement brought by this change is that we no
longer drop messages queued for sending if the socket doesn't become
writable after a 100ms timeout. Instead, we return the message to the
send buffer and continue waiting for the socket to become writable.
Instead of wrapping all non-movable members of TransportSocket in OwnPtr
to keep it movable, make TransportSocket itself non-movable and wrap it
in OwnPtr.
By doing this we also make MessagePort, that relies on IPC transport,
to send messages from separate thread, which solves the problem when
WebWorker and WebContent could deadlock if both were trying to post
messages at the same time.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/4254
Reimplements c3121c9d at the transport layer, allowing us to solve the
same problem once, in a single place, for both the LibIPC connection and
MessagePort. This avoids exposing a workaround for a macOS specific Unix
domain socket issue to higher abstraction layers.
With this change, the responsibility for prepending messages with their
size and ensuring the entire message is received before returning it to
the caller is moved to TransportSocket. This removes the need to
duplicate this logic in both LibIPC and MessagePort.
Another advantage of reducing message granularity at IPC::Transport
layer is that it will make it easier to support alternative transport
implementations (like Mach ports, which unlike Unix domain sockets are
not stream oriented).
It turned out that some web applications want to send fairly large
messages to WebWorker through IPC (for example, MapLibre GL sends
~1200KiB), which led to failures (at least on macOS) because buffer size
of TransportSocket is limited to 128KiB. This change solves the problem
by wrapping messages that exceed socket buffer size into another message
that holds wrapped message content in shared memory.
Co-Authored-By: Luke Wilde <luke@ladybird.org>
