These steps are the best definition we have for how the ready state
should be set, and it seems to be reasonable to apply to plain file
playback as well.
Since our file demuxers are hardcoded to return the entire duration as
buffered, the ready state immediately progresses to HAVE_CURRENT_DATA.
This will probably change once we can check the demuxers for buffered
data.
Removing a display risks triggering callbacks on the playback manager
that may cause a recursive GC. This wasn't having any effect since the
playback manager became an OwnPtr.
...giving tracks a kind attribute, and renaming name to label.
Demuxers will need to determine the kind attribute, since the spec for
sourcing tracks requires us to select based on info we don't expose.
In `::spin_processing_tasks_with_source_until()`, we would first take a
set of tasks based on a filter, and then run them one by one. If there
was more than one task matched and put in that vector, they could
interfere with each other's runnability by making later tasks
permanently unrunnable.
The `::take_tasks_matching()` API is a footgun - remove it in favor of
an API that takes tasks one by one, performing the runnability check
just in time.
Since we know whether we're buffering from the PlaybackManager state,
let's use that to update the ready state. This ensures that when we set
the ready state to HAVE_CURRENT_DATA, we actually have a frame.
Setting the ready state to HAVE_ENOUGH_DATA should probably still be
further conditioned on the buffer size in IncrementallyPopulatedStream,
but this is still an improvement for now.
This workaround avoids a crash that occurs when various fetching
processes (in the media element) are quickly started and canceled.
Although a better solution would be to actually remove the body
callbacks when the fetch is stopped, this works for now.
Rename Document::set_needs_display() to set_needs_repaint() and make it
private. External callers must now go through Node/Paintable which
route the request to the document internally.
Fix one existing misuse in AnimationEffect that was calling
document-level set_needs_display() instead of routing through the
target element's paintable.
This is preparation for per-paintable display list command caching:
repaint requests must go through specific paintables so their cached
command lists can be invalidated.
d146adf made the fetch callbacks use the media element via weak
references. This caused the `error` event not to fire on media elements
that are detached from the document and go out of scope, if the GC got
to them before the fetch completed.
Instead of relying on weak references in the callbacks, we can stop the
ongoing fetch when the document becomes inactive to allow it to be GCed
after that point. By storing the FetchData on the media element, we're
able to resume the fetch where it left off if the document becomes
active again.
We could potentially figure out a way to make elements with no event
handlers and no parent stop their fetches in order to be GCed sooner,
but that is probably a bit fiddly, so may not be worth it for now.
Fixes a rare flake in WPT's `html/semantics/embedded-content/media-
elements/error-codes/error.html` test. A test to force the bug using
`Internals::gc()` has been added.
This fixes a crash that would occasionally happen in media-load-during-
track-setup.html, where the media element would add new tracks to the
AudioTrackList or VideoTrackList after the src attribute changed and
caused the element to clear those lists.
In order to make this safe, the on_unsupported_format_error needs to
queue a task to invoke the failure callback which eventually clears
the playback manager, or ~PlaybackManager() will try to destroy the
enclosing lambda.
We found an HTML spec issue that implies that media element steps that
mutate the DOM should run in tasks rather than awaiting a stable state.
Awaiting a stable state implies running in a microtask, which is
apparently not supposed to be used to mutate.
Add unsafe_layout_node(), unsafe_paintable(), and unsafe_paintable_box()
accessors that skip layout-staleness verification. These are for use in
contexts where accessing layout/paintable data is legitimate despite
layout not being up to date: tree construction, style recalculation,
painting, animation interpolation, DOM mutation, and invalidation
propagation.
Also add wrapper APIs on Node to centralize common patterns:
- set_needs_display() wraps if (unsafe_paintable()) ...set_needs_display
- set_needs_paint_only_properties_update() wraps similar
- set_needs_layout_update() wraps if (unsafe_layout_node()) ...
And add Document::layout_is_up_to_date() which checks whether layout
tree update flags are all clear.
Instead of using a custom paintable to draw the controls for video and
audio elements, we build them out of plain old HTML elements within a
shadow root.
This required a few hacks in the previous commits in order to allow a
replaced element to host children within a shadow root, but it's
fairly self-contained.
A big benefit is that we can drive all the UI updates off of plain old
DOM events (except the play button overlay on videos, which uses the
video element representation), so we can test our media and input event
handling more thoroughly. :^)
The control bar visibility is now more similar to how other browsers
handle it. It will show upon hovering over the element, but if the
cursor is kept still for more than a second, it will hide again. While
dragging, the controls remain visible, and will then hide after the
mouse button is released.
The icons have been redesigned from scratch, and the mute icon now
visualizes the volume level along with indicating the mute state.
These are the same code, so we may as well move them up the chain. This
becomes useful in a later commit, where it will be used to rewrite
inline-flow to inline-block for layout of shadow DOM.
This should help avoid the footgun of forgetting to check for null on
m_fetch_controller. We had missed this check when firing off an error
due to an unsupported format in the PlaybackManager, so we could call
stop_fetch() on a null pointer if the download had completed already.
These tasks' captures aren't clearly safe as written, since raw
references don't make it apparent that we're capturing a GC-aware
reference. Conservative scanning made this safe, but let's make it a
bit clearer.
Publish new video frames to an ExternalContentSource, and switch
VideoPaintable from draw_scaled_immutable_bitmap to
draw_external_content.
Because DrawExternalContent reads the latest bitmap at replay time,
frame-only updates (no timeline or control change) now call
set_needs_display(InvalidateDisplayList::No) — skipping display list
rebuilds entirely. This addresses problem 2 from the previous commit.
The stream's data request callback can't hold a strong reference to
FetchData, as that will create a reference loop:
FetchData -> IncrementallyPopulatedStream -> (lambda) -> FetchData
To prevent a use-after-free on the FetchData& capture, we clear the
data request callback in ~FetchData().
Currently, this just respects the reported value from Accept-Ranges,
but we could also just try sending a range request and see if the
server rejects it, then fall back to a normal request after. For now,
this is fine, and we can make it use a fallback later if needed.
The spec's steps for pausing an HTMLMediaElements prescribe setting the
official playback position to the current playback position, but the
seeking steps are not synchronous, so there's no guarantee that the
current playback position reflects the seek. Therefore, we need to skip
that step if we're in the middle of a seek.
This is included in a pull request to the HTML spec:
https://github.com/whatwg/html/pull/11792
Using the response's header over our own request's byte range is likely
to be a little more reliable for piecing the data together in the
stream, in case the server decides to give us a slightly different
range than we requested. The media element spec requires that the
Content-Range parses correctly anyway, so we should make use of the
values we get out of it.
This makes media playback able to start without having to wait for data
to sequentially download, especially when seeking the media to a
timestamp residing in data that hasn't loaded yet.
Initially, the HTMLMediaElement will request the file without range a
range request. Then, if the IncrementallyPopulatedStream finds that it
needs data that is not yet available, it will decide whether to wait
for that data to be received through the current request, or start a
new request that is closer to the required data.
In this commit, it assumes that the server will support range requests.
Refactor the FFmpeg and Matroska demuxers to consume data through
`IncrementallyPopulatedStream::Cursor` instead of a pointer to fully
buffered.
This change establishes a new rule: each track must be initialized with
its own cursor. Data providers now explicitly create a per-track context
via `Demuxer::create_context_for_track(track, cursor)`, and own pointer
to that cursor. In the upcoming changes, holding the cursor in the
provider would allow to signal "cancel blocking reads" so an
in-flight seek can fail immediately when a newer seek request arrives.
Demuxer creation and track+duration extraction are moved to a separate
thread so that the media data byte buffer is no longer accessed from the
main thread. This will be important once the buffer is populated
incrementally, as having the main thread both populate and read from the
same buffer could easily lead to deadlocks. Aside from that, moving
demuxer creation off the main thread helps to be more responsive.
`VideoDataProvider` and `AudioDataProvider` now accept the main thread
event loop pointer as they are constructed from the thread responsible
for demuxer creation.
Prevents observably calling Trusted Types, which can run arbitrary JS,
cause crashes due to use of MUST and allow arbitrary JS to modify
internal elements.
In cases where a script assigns `x = video.currentTime = y`, we are
expected to have a result of `x === y`, even if the video's duration
is less than y.
According to the spec, this happens because the official playback
position is set to `y` in this case, but since we are following
implementations in making `currentTime` immediately return the position
on the valid media timeline, we have to specifically return the
unchanged value from the setter.
See: https://github.com/whatwg/html/issues/11773
