We could apparently update the ready state within time_marches_on()
called by set_ready_state(), which could result in triggering autoplay
twice for the same state transition.
The case of an unsupported format error wasn't covered for this,
meaning that it could crash if the fetch completed successfully after
the fetch was cancelled due to such an error.
A crash test is included for this issue, using an echo of a large
corrupted WebM file to ensure that the fetch completes after media
init.
Audio output on macOS was consuming Core Audio resources until the
PlaybackStream creation took well over the timeout for some tests.
This was observed in media-source-buffered.html, where it would time
out due to the long-running callback on the main thread to create the
PlaybackStream for AudioMixingSink.
However, the AudioUnit init should definitely not be blocking the main
thread, so I've added a FIXME there.
If we fire the error event synchronously within the on_error callback,
then we'll end up destroying the PlaybackManager inside its own
callback and crash. Instead, queue a task to execute the error steps.
This could happen with or without MSE, but I observed it occurring on
YouTube with MSE when we hit a decoding error, since they immediately
try another source when an error is reported.
The segments are parsed for the SourceBufferProcessor by the
WebMByteStreamParser. It parses the initialization segment to update
its internal set of tracks, then SourceBufferProcessor/SourceBuffer set
them up for playback. When a media segment is received, it also parses
as much of it as is available, returning all the coded frames found so
far. SourceBufferProcessor then tells TrackBufferDemuxer to remove any
overlapping frames and insert the new ones.
TrackBufferDemuxer implements the Demuxer interface in terms of the
coded frame store maintained by the SourceBufferProcessor. It returns
the frames in decode order when requested by a data provider. When a
is needed, it finds the keyframe prior to the target timestamp, and
checks that there are no gaps in data up to the target timestamp. If
there are any gaps, it blocks until the gaps are gone.
These steps are the best definition we have for how the ready state
should be set, and it seems to be reasonable to apply to plain file
playback as well.
Since our file demuxers are hardcoded to return the entire duration as
buffered, the ready state immediately progresses to HAVE_CURRENT_DATA.
This will probably change once we can check the demuxers for buffered
data.
Removing a display risks triggering callbacks on the playback manager
that may cause a recursive GC. This wasn't having any effect since the
playback manager became an OwnPtr.
...giving tracks a kind attribute, and renaming name to label.
Demuxers will need to determine the kind attribute, since the spec for
sourcing tracks requires us to select based on info we don't expose.
In `::spin_processing_tasks_with_source_until()`, we would first take a
set of tasks based on a filter, and then run them one by one. If there
was more than one task matched and put in that vector, they could
interfere with each other's runnability by making later tasks
permanently unrunnable.
The `::take_tasks_matching()` API is a footgun - remove it in favor of
an API that takes tasks one by one, performing the runnability check
just in time.
Since we know whether we're buffering from the PlaybackManager state,
let's use that to update the ready state. This ensures that when we set
the ready state to HAVE_CURRENT_DATA, we actually have a frame.
Setting the ready state to HAVE_ENOUGH_DATA should probably still be
further conditioned on the buffer size in IncrementallyPopulatedStream,
but this is still an improvement for now.
This workaround avoids a crash that occurs when various fetching
processes (in the media element) are quickly started and canceled.
Although a better solution would be to actually remove the body
callbacks when the fetch is stopped, this works for now.
Rename Document::set_needs_display() to set_needs_repaint() and make it
private. External callers must now go through Node/Paintable which
route the request to the document internally.
Fix one existing misuse in AnimationEffect that was calling
document-level set_needs_display() instead of routing through the
target element's paintable.
This is preparation for per-paintable display list command caching:
repaint requests must go through specific paintables so their cached
command lists can be invalidated.
d146adf made the fetch callbacks use the media element via weak
references. This caused the `error` event not to fire on media elements
that are detached from the document and go out of scope, if the GC got
to them before the fetch completed.
Instead of relying on weak references in the callbacks, we can stop the
ongoing fetch when the document becomes inactive to allow it to be GCed
after that point. By storing the FetchData on the media element, we're
able to resume the fetch where it left off if the document becomes
active again.
We could potentially figure out a way to make elements with no event
handlers and no parent stop their fetches in order to be GCed sooner,
but that is probably a bit fiddly, so may not be worth it for now.
Fixes a rare flake in WPT's `html/semantics/embedded-content/media-
elements/error-codes/error.html` test. A test to force the bug using
`Internals::gc()` has been added.
This fixes a crash that would occasionally happen in media-load-during-
track-setup.html, where the media element would add new tracks to the
AudioTrackList or VideoTrackList after the src attribute changed and
caused the element to clear those lists.
In order to make this safe, the on_unsupported_format_error needs to
queue a task to invoke the failure callback which eventually clears
the playback manager, or ~PlaybackManager() will try to destroy the
enclosing lambda.
We found an HTML spec issue that implies that media element steps that
mutate the DOM should run in tasks rather than awaiting a stable state.
Awaiting a stable state implies running in a microtask, which is
apparently not supposed to be used to mutate.
Add unsafe_layout_node(), unsafe_paintable(), and unsafe_paintable_box()
accessors that skip layout-staleness verification. These are for use in
contexts where accessing layout/paintable data is legitimate despite
layout not being up to date: tree construction, style recalculation,
painting, animation interpolation, DOM mutation, and invalidation
propagation.
Also add wrapper APIs on Node to centralize common patterns:
- set_needs_display() wraps if (unsafe_paintable()) ...set_needs_display
- set_needs_paint_only_properties_update() wraps similar
- set_needs_layout_update() wraps if (unsafe_layout_node()) ...
And add Document::layout_is_up_to_date() which checks whether layout
tree update flags are all clear.
Instead of using a custom paintable to draw the controls for video and
audio elements, we build them out of plain old HTML elements within a
shadow root.
This required a few hacks in the previous commits in order to allow a
replaced element to host children within a shadow root, but it's
fairly self-contained.
A big benefit is that we can drive all the UI updates off of plain old
DOM events (except the play button overlay on videos, which uses the
video element representation), so we can test our media and input event
handling more thoroughly. :^)
The control bar visibility is now more similar to how other browsers
handle it. It will show upon hovering over the element, but if the
cursor is kept still for more than a second, it will hide again. While
dragging, the controls remain visible, and will then hide after the
mouse button is released.
The icons have been redesigned from scratch, and the mute icon now
visualizes the volume level along with indicating the mute state.
These are the same code, so we may as well move them up the chain. This
becomes useful in a later commit, where it will be used to rewrite
inline-flow to inline-block for layout of shadow DOM.
This should help avoid the footgun of forgetting to check for null on
m_fetch_controller. We had missed this check when firing off an error
due to an unsupported format in the PlaybackManager, so we could call
stop_fetch() on a null pointer if the download had completed already.
These tasks' captures aren't clearly safe as written, since raw
references don't make it apparent that we're capturing a GC-aware
reference. Conservative scanning made this safe, but let's make it a
bit clearer.
Publish new video frames to an ExternalContentSource, and switch
VideoPaintable from draw_scaled_immutable_bitmap to
draw_external_content.
Because DrawExternalContent reads the latest bitmap at replay time,
frame-only updates (no timeline or control change) now call
set_needs_display(InvalidateDisplayList::No) — skipping display list
rebuilds entirely. This addresses problem 2 from the previous commit.
The stream's data request callback can't hold a strong reference to
FetchData, as that will create a reference loop:
FetchData -> IncrementallyPopulatedStream -> (lambda) -> FetchData
To prevent a use-after-free on the FetchData& capture, we clear the
data request callback in ~FetchData().
