Add unsafe_layout_node(), unsafe_paintable(), and unsafe_paintable_box()
accessors that skip layout-staleness verification. These are for use in
contexts where accessing layout/paintable data is legitimate despite
layout not being up to date: tree construction, style recalculation,
painting, animation interpolation, DOM mutation, and invalidation
propagation.
Also add wrapper APIs on Node to centralize common patterns:
- set_needs_display() wraps if (unsafe_paintable()) ...set_needs_display
- set_needs_paint_only_properties_update() wraps similar
- set_needs_layout_update() wraps if (unsafe_layout_node()) ...
And add Document::layout_is_up_to_date() which checks whether layout
tree update flags are all clear.
Instead of using a custom paintable to draw the controls for video and
audio elements, we build them out of plain old HTML elements within a
shadow root.
This required a few hacks in the previous commits in order to allow a
replaced element to host children within a shadow root, but it's
fairly self-contained.
A big benefit is that we can drive all the UI updates off of plain old
DOM events (except the play button overlay on videos, which uses the
video element representation), so we can test our media and input event
handling more thoroughly. :^)
The control bar visibility is now more similar to how other browsers
handle it. It will show upon hovering over the element, but if the
cursor is kept still for more than a second, it will hide again. While
dragging, the controls remain visible, and will then hide after the
mouse button is released.
The icons have been redesigned from scratch, and the mute icon now
visualizes the volume level along with indicating the mute state.
These are the same code, so we may as well move them up the chain. This
becomes useful in a later commit, where it will be used to rewrite
inline-flow to inline-block for layout of shadow DOM.
This should help avoid the footgun of forgetting to check for null on
m_fetch_controller. We had missed this check when firing off an error
due to an unsupported format in the PlaybackManager, so we could call
stop_fetch() on a null pointer if the download had completed already.
These tasks' captures aren't clearly safe as written, since raw
references don't make it apparent that we're capturing a GC-aware
reference. Conservative scanning made this safe, but let's make it a
bit clearer.
Publish new video frames to an ExternalContentSource, and switch
VideoPaintable from draw_scaled_immutable_bitmap to
draw_external_content.
Because DrawExternalContent reads the latest bitmap at replay time,
frame-only updates (no timeline or control change) now call
set_needs_display(InvalidateDisplayList::No) — skipping display list
rebuilds entirely. This addresses problem 2 from the previous commit.
The stream's data request callback can't hold a strong reference to
FetchData, as that will create a reference loop:
FetchData -> IncrementallyPopulatedStream -> (lambda) -> FetchData
To prevent a use-after-free on the FetchData& capture, we clear the
data request callback in ~FetchData().
Currently, this just respects the reported value from Accept-Ranges,
but we could also just try sending a range request and see if the
server rejects it, then fall back to a normal request after. For now,
this is fine, and we can make it use a fallback later if needed.
The spec's steps for pausing an HTMLMediaElements prescribe setting the
official playback position to the current playback position, but the
seeking steps are not synchronous, so there's no guarantee that the
current playback position reflects the seek. Therefore, we need to skip
that step if we're in the middle of a seek.
This is included in a pull request to the HTML spec:
https://github.com/whatwg/html/pull/11792
Using the response's header over our own request's byte range is likely
to be a little more reliable for piecing the data together in the
stream, in case the server decides to give us a slightly different
range than we requested. The media element spec requires that the
Content-Range parses correctly anyway, so we should make use of the
values we get out of it.
This makes media playback able to start without having to wait for data
to sequentially download, especially when seeking the media to a
timestamp residing in data that hasn't loaded yet.
Initially, the HTMLMediaElement will request the file without range a
range request. Then, if the IncrementallyPopulatedStream finds that it
needs data that is not yet available, it will decide whether to wait
for that data to be received through the current request, or start a
new request that is closer to the required data.
In this commit, it assumes that the server will support range requests.
Refactor the FFmpeg and Matroska demuxers to consume data through
`IncrementallyPopulatedStream::Cursor` instead of a pointer to fully
buffered.
This change establishes a new rule: each track must be initialized with
its own cursor. Data providers now explicitly create a per-track context
via `Demuxer::create_context_for_track(track, cursor)`, and own pointer
to that cursor. In the upcoming changes, holding the cursor in the
provider would allow to signal "cancel blocking reads" so an
in-flight seek can fail immediately when a newer seek request arrives.
Demuxer creation and track+duration extraction are moved to a separate
thread so that the media data byte buffer is no longer accessed from the
main thread. This will be important once the buffer is populated
incrementally, as having the main thread both populate and read from the
same buffer could easily lead to deadlocks. Aside from that, moving
demuxer creation off the main thread helps to be more responsive.
`VideoDataProvider` and `AudioDataProvider` now accept the main thread
event loop pointer as they are constructed from the thread responsible
for demuxer creation.
Prevents observably calling Trusted Types, which can run arbitrary JS,
cause crashes due to use of MUST and allow arbitrary JS to modify
internal elements.
In cases where a script assigns `x = video.currentTime = y`, we are
expected to have a result of `x === y`, even if the video's duration
is less than y.
According to the spec, this happens because the official playback
position is set to `y` in this case, but since we are following
implementations in making `currentTime` immediately return the position
on the valid media timeline, we have to specifically return the
unchanged value from the setter.
See: https://github.com/whatwg/html/issues/11773
Due to the round trip of Duration -> double -> Duration, seeking to the
end of some media can sometimes result in the seek being resolved close
to the end but not quite there. This is a little bit of a hack to make
that work, but may be necessary depending on how the spec changes with
regard to the value returned by currentTime after a seek begins.
This allows playback to restart when playing is requested after the end
of playback was reached while loop was disabled, regardless of whether
loop is then subsequently enabled.
This matches other browsers' implementations, but differs from the spec
in how the ended attribute is handled.
See: https://github.com/whatwg/html/issues/11775
The spec changed in this regard, and this change ensures that once the
ended attribute is updated only during event loop step 1, ended event
handlers will see the ended attribute set to true.
This fixes a crash when playing video on The Cutting Room Floor.
Without aborting the resource selection algorithm, two resource
selection algorithms could be running at once, resulting in the
element requesting removal of a track from the PlaybackManager
immediately after it had been replaced with a different instance.
PlaybackManager asserts that removal of a track is valid, so this was
causing a WebContent crash.
The two classes now inherit from a common base MediaTrackBase, to
deduplicate the attributes that are shared between the two.
The integer ID from the container is used for each track's id
attribute.
The kind attribute is set to "main" or "translation" according to:
https://dev.w3.org/html5/html-sourcing-inband-tracks/
The label attribute is set to the human-readable name of the track, if
one is present.
The language attribute is set to a BCP 47 language tag, if one can be
parsed successfully.
With this commit, all PlaybackManager can do is autoplay a file from
start to finish, with no pausing or seeking functionality.
All audio playback functionality has been removed from HTMLMediaElement
and HTMLAudioElement in anticipation of PlaybackManager taking that
over, for both audio-only and audio/video.
The `muted` content attribute should only affect the state of the
`muted` IDL property when the media element is first created. The
attribute should have no dynamic effect.
