Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-11-08 17:21:02 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Userland/Libraries/LibVideo/PlaybackManager.cpp

346 lines
14 KiB
C++
Raw Normal View History

LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
/*
* Copyright (c) 2022, Gregory Bertilson <zaggy1024@gmail.com>
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#include <AK/Format.h>
#include <LibCore/Timer.h>
LibVideo: Reorganize demuxer file hierarchy and rename Matroska files As new demuxers are added, this will get quite full of files, so it'll be good to have a separate folder for these. To avoid too many chained namespaces, the Containers subdirectory is not also a namespace, but the Matroska folder is for the sake of separating the multiple classes for parsed information entering the Video namespace.
2022-11-09 19:47:56 -06:00
#include <LibVideo/Containers/Matroska/MatroskaDemuxer.h>
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
#include <LibVideo/VP9/Decoder.h>
#include "PlaybackManager.h"
namespace Video {
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
DecoderErrorOr<NonnullOwnPtr<PlaybackManager>> PlaybackManager::from_file(Core::Object& event_handler, StringView filename)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
{
LibVideo: Reorganize demuxer file hierarchy and rename Matroska files As new demuxers are added, this will get quite full of files, so it'll be good to have a separate folder for these. To avoid too many chained namespaces, the Containers subdirectory is not also a namespace, but the Matroska folder is for the sake of separating the multiple classes for parsed information entering the Video namespace.
2022-11-09 19:47:56 -06:00
NonnullOwnPtr<Demuxer> demuxer = TRY(Matroska::MatroskaDemuxer::from_file(filename));
LibVideo: Read Matroska lazily so that large files can start quickly The Demuxer class was changed to return errors for more functions so that all of the underlying reading can be done lazily. Other than that, the demuxer interface is unchanged, and only the underlying reader was modified. The MatroskaDocument class is no more, and MatroskaReader's getter functions replace it. Every MatroskaReader getter beyond the Segment element's position is parsed lazily from the file as needed. This means that all getter functions can return DecoderErrors which must be handled by callers.
2022-11-11 17:14:27 -06:00
auto video_tracks = TRY(demuxer->get_tracks_for_type(TrackType::Video));
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
if (video_tracks.is_empty())
return DecoderError::with_description(DecoderErrorCategory::Invalid, "No video track is present"sv);
auto track = video_tracks[0];
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Selecting video track number {}", track.identifier());
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
return make<PlaybackManager>(event_handler, demuxer, track, make<VP9::Decoder>());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
PlaybackManager::PlaybackManager(Core::Object& event_handler, NonnullOwnPtr<Demuxer>& demuxer, Track video_track, NonnullOwnPtr<VideoDecoder>&& decoder)
: m_event_handler(event_handler)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
, m_main_loop(Core::EventLoop::current())
, m_demuxer(move(demuxer))
, m_selected_video_track(video_track)
, m_decoder(move(decoder))
, m_frame_queue(make<VideoFrameQueue>())
{
Userland: Use Core::Timer::create_foo() factory functions where possible
2023-01-11 20:00:46 +00:00
m_present_timer = Core::Timer::create_single_shot(0, [&] { update_presented_frame(); }).release_value_but_fixme_should_propagate_errors();
m_decode_timer = Core::Timer::create_single_shot(0, [&] { on_decode_timer(); }).release_value_but_fixme_should_propagate_errors();
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
void PlaybackManager::set_playback_status(PlaybackStatus status)
{
if (status != m_status) {
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
bool was_stopped = is_stopped();
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
auto old_status = m_status;
m_status = status;
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Set playback status from {} to {}", playback_status_to_string(old_status), playback_status_to_string(m_status));
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
if (m_status == PlaybackStatus::Playing) {
if (was_stopped)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
restart_playback();
m_last_present_in_real_time = Time::now_monotonic();
LibVideo: Make all PlaybackManager Timer::start calls set an interval Timers keep their previously set interval even for single-shot mode. We want all timers to fire immediately if they don't have a delay set in the start() call.
2022-11-14 05:40:59 -06:00
m_present_timer->start(0);
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
} else if (!is_seeking()) {
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
m_last_present_in_media_time = current_playback_time();
m_last_present_in_real_time = Time::zero();
m_present_timer->stop();
}
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<PlaybackStatusChangeEvent>(status, old_status));
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
}
void PlaybackManager::resume_playback()
{
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (is_seeking()) {
set_playback_status(PlaybackStatus::SeekingPlaying);
return;
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
set_playback_status(PlaybackStatus::Playing);
}
void PlaybackManager::pause_playback()
{
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (is_seeking()) {
set_playback_status(PlaybackStatus::SeekingPaused);
return;
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
set_playback_status(PlaybackStatus::Paused);
}
Time PlaybackManager::current_playback_time()
{
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (is_seeking())
return m_seek_to_media_time;
VERIFY(!m_last_present_in_media_time.is_negative());
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
if (m_status == PlaybackStatus::Playing)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
return m_last_present_in_media_time + (Time::now_monotonic() - m_last_present_in_real_time);
return m_last_present_in_media_time;
}
Time PlaybackManager::duration()
{
LibVideo: Read Matroska lazily so that large files can start quickly The Demuxer class was changed to return errors for more functions so that all of the underlying reading can be done lazily. Other than that, the demuxer interface is unchanged, and only the underlying reader was modified. The MatroskaDocument class is no more, and MatroskaReader's getter functions replace it. Every MatroskaReader getter beyond the Segment element's position is parsed lazily from the file as needed. This means that all getter functions can return DecoderErrors which must be handled by callers.
2022-11-11 17:14:27 -06:00
auto duration_result = m_demuxer->duration();
if (duration_result.is_error())
on_decoder_error(duration_result.release_error());
return duration_result.release_value();
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
void PlaybackManager::on_decoder_error(DecoderError error)
{
LibVideo: Don't crash when a decoder error is encountered while seeking When errors are encountered by PlaybackManager, it attempts to switch states to either Stopped or Corrupted. However, that causes it to set the last presentation media time to the current playback time while the last presentation time is unexpectedly negative because the seek never ended. Ending the seek before the state changes to Stopped or Corrupted prevents this situation from happening.
2022-11-27 00:45:14 -06:00
// If we don't switch to playing/paused before stopping/becoming corrupted, the player will crash
// due to the invalid playback time.
if (is_seeking())
end_seek();
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
switch (error.category()) {
case DecoderErrorCategory::EndOfStream:
LibVideo: Only print non-fatal playback errors when debug mode is on
2022-11-11 22:04:38 -06:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, "{}", error.string_literal());
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
set_playback_status(PlaybackStatus::Stopped);
break;
default:
LibVideo: Only print non-fatal playback errors when debug mode is on
2022-11-11 22:04:38 -06:00
dbgln("Playback error encountered: {}", error.string_literal());
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
set_playback_status(PlaybackStatus::Corrupted);
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<DecoderErrorEvent>(move(error)));
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
break;
}
}
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
void PlaybackManager::end_seek()
{
LibVideo: Implement Matroska Cues for faster keyframe lookup This implements the fastest seeking mode available for tracks with cues using an array of cue points for each track. It approximates the index based on the seeking timestamp and then finds the earliest cue point before the timestamp. The approximation assumes that cues will be on a regular interval, which I don't believe is always the case, but it should at least be faster than iterating the whole set of cue points each time. Cues are stored per track, but most videos will only have cue points for the video track(s) that are present. For now, this assumes that it should only seek based on the cue points for the selected track. To seek audio in a video file, we should copy the seeked iterator over to the audio track's iterator after seeking is complete. The iterator will then skip to the next audio block.
2022-11-13 19:28:56 -06:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, "We've finished seeking, set media time to seek time at {}ms and change status", m_seek_to_media_time.to_milliseconds());
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
VERIFY(!m_seek_to_media_time.is_negative());
m_last_present_in_media_time = m_seek_to_media_time;
m_seek_to_media_time = Time::min();
if (m_status == PlaybackStatus::SeekingPlaying) {
set_playback_status(PlaybackStatus::Playing);
return;
}
VERIFY(m_status == PlaybackStatus::SeekingPaused);
set_playback_status(PlaybackStatus::Paused);
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
void PlaybackManager::update_presented_frame()
{
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
Optional<FrameQueueItem> future_frame_item;
bool should_present_frame = false;
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
// Skip frames until we find a frame past the current playback time, and keep the one that precedes it to display.
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
while ((m_status == PlaybackStatus::Playing || is_seeking()) && !m_frame_queue->is_empty()) {
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
future_frame_item.emplace(m_frame_queue->dequeue());
m_decode_timer->start(0);
if (future_frame_item->is_error() || future_frame_item->timestamp() >= current_playback_time()) {
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Should present frame, future {} is after {}ms", future_frame_item->debug_string(), current_playback_time().to_milliseconds());
should_present_frame = true;
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
break;
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (m_next_frame.has_value() && !is_seeking()) {
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, "At {}ms: Dropped {} in favor of {}", current_playback_time().to_milliseconds(), m_next_frame->debug_string(), future_frame_item->debug_string());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
m_skipped_frames++;
}
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
m_next_frame.emplace(future_frame_item.release_value());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
// If we don't have both of these items, we can't present, since we need to set a timer for
// the next frame. Check if we need to buffer based on the current state.
if (!m_next_frame.has_value() || !future_frame_item.has_value()) {
#if PLAYBACK_MANAGER_DEBUG
StringBuilder debug_string_builder;
debug_string_builder.append("We don't have "sv);
if (!m_next_frame.has_value()) {
debug_string_builder.append("a frame to present"sv);
if (!future_frame_item.has_value())
debug_string_builder.append(" or a future frame"sv);
} else {
debug_string_builder.append("a future frame"sv);
}
debug_string_builder.append(", checking for error and buffering"sv);
Everywhere: Rename to_{string => deprecated_string}() where applicable This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06 01:12:49 +00:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, debug_string_builder.to_deprecated_string());
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
#endif
if (future_frame_item.has_value()) {
if (future_frame_item->is_error()) {
on_decoder_error(future_frame_item.release_value().release_error());
return;
}
m_next_frame.emplace(future_frame_item.release_value());
}
if (m_status == PlaybackStatus::Playing)
set_playback_status(PlaybackStatus::Buffering);
m_decode_timer->start(0);
return;
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
// If we have a frame, send it for presentation.
if (should_present_frame) {
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (is_seeking())
end_seek();
else
m_last_present_in_media_time = current_playback_time();
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
m_last_present_in_real_time = Time::now_monotonic();
m_main_loop.post_event(m_event_handler, make<VideoFramePresentEvent>(m_next_frame.value().bitmap()));
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Sent frame for presentation");
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
// Now that we've presented the current frame, we can throw whatever error is next in queue.
// This way, we always display a frame before the stream ends, and should also show any frames
// we already had when a real error occurs.
if (future_frame_item->is_error()) {
on_decoder_error(future_frame_item.release_value().release_error());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
return;
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
// The future frame item becomes the next one to present.
m_next_frame.emplace(future_frame_item.release_value());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
if (m_status != PlaybackStatus::Playing) {
dbgln_if(PLAYBACK_MANAGER_DEBUG, "We're not playing! Starting the decode timer");
m_decode_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
return;
}
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
auto frame_time_ms = (m_next_frame->timestamp() - current_playback_time()).to_milliseconds();
VERIFY(frame_time_ms <= NumericLimits<int>::max());
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Time until next frame is {}ms", frame_time_ms);
m_present_timer->start(max(static_cast<int>(frame_time_ms), 0));
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
VideoPlayer/LibVideo: Implement the UI functionality for seeking With these changes, the seek bar can be used, but only to seek to the start of the file. Seeking to anywhere else in the file will cause an error in the demuxer. The timestamp label that was previously invisible now has its text set according to either the playback or seek slider's position.
2022-11-11 22:26:19 -06:00
void PlaybackManager::seek_to_timestamp(Time timestamp)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
{
VideoPlayer/LibVideo: Implement the UI functionality for seeking With these changes, the seek bar can be used, but only to seek to the start of the file. Seeking to anywhere else in the file will cause an error in the demuxer. The timestamp label that was previously invisible now has its text set according to either the playback or seek slider's position.
2022-11-11 22:26:19 -06:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Seeking to {}ms", timestamp.to_milliseconds());
// FIXME: When the demuxer is getting samples off the main thread in the future, this needs to
// mutex so that seeking can't happen while that thread is getting a sample.
auto result = m_demuxer->seek_to_most_recent_keyframe(m_selected_video_track, timestamp);
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
if (result.is_error())
on_decoder_error(result.release_error());
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (is_playing())
set_playback_status(PlaybackStatus::SeekingPlaying);
else
set_playback_status(PlaybackStatus::SeekingPaused);
m_frame_queue->clear();
m_next_frame.clear();
m_skipped_frames = 0;
LibVideo: Add a fast seeking mode to seek only to keyframes Now that we're able to find the nearest keyframe, we can have a fast seeking mode that only seeks to keyframes, so that it doesn't have to also decode inter frames until it reaches the timestamp. The default is still accurate seeking, so that the entire seeking implementation can be tested.
2022-11-13 20:33:23 -06:00
if (m_seek_mode == SeekMode::Accurate)
m_seek_to_media_time = timestamp;
else
m_seek_to_media_time = result.release_value();
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
m_last_present_in_media_time = Time::min();
m_last_present_in_real_time = Time::zero();
m_present_timer->stop();
m_decode_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
VideoPlayer/LibVideo: Implement the UI functionality for seeking With these changes, the seek bar can be used, but only to seek to the start of the file. Seeking to anywhere else in the file will cause an error in the demuxer. The timestamp label that was previously invisible now has its text set according to either the playback or seek slider's position.
2022-11-11 22:26:19 -06:00
void PlaybackManager::restart_playback()
{
seek_to_timestamp(Time::zero());
}
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
void PlaybackManager::post_decoder_error(DecoderError error)
{
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<DecoderErrorEvent>(error));
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
bool PlaybackManager::decode_and_queue_one_sample()
{
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
if (m_frame_queue->size() >= FRAME_BUFFER_COUNT) {
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Frame queue is full, stopping");
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
return false;
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
#if PLAYBACK_MANAGER_DEBUG
auto start_time = Time::now_monotonic();
#endif
LibVideo: Only print non-fatal playback errors when debug mode is on
2022-11-11 22:04:38 -06:00
#define TRY_OR_ENQUEUE_ERROR(expression) \
({ \
auto _temporary_result = ((expression)); \
if (_temporary_result.is_error()) { \
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Enqueued decoder error: {}", _temporary_result.error().string_literal()); \
m_frame_queue->enqueue(FrameQueueItem::error_marker(_temporary_result.release_error())); \
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
m_present_timer->start(0); \
LibVideo: Only print non-fatal playback errors when debug mode is on
2022-11-11 22:04:38 -06:00
return false; \
} \
AK+Everywhere: Disallow returning a reference from a fallible expression This will silently make a copy. Rather than masking this behavior, let's explicitly disallow it.
2023-01-13 15:48:09 -05:00
static_assert(!IsLvalueReference<decltype(_temporary_result.release_value())>, \
"Do not return a reference from a fallible expression"); \
LibVideo: Only print non-fatal playback errors when debug mode is on
2022-11-11 22:04:38 -06:00
_temporary_result.release_value(); \
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
})
auto frame_sample = TRY_OR_ENQUEUE_ERROR(m_demuxer->get_next_video_sample_for_track(m_selected_video_track));
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
OwnPtr<VideoFrame> decoded_frame = nullptr;
while (!decoded_frame) {
TRY_OR_ENQUEUE_ERROR(m_decoder->receive_sample(frame_sample->data()));
while (true) {
auto frame_result = m_decoder->get_decoded_frame();
if (frame_result.is_error()) {
if (frame_result.error().category() == DecoderErrorCategory::NeedsMoreInput)
break;
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
post_decoder_error(frame_result.release_error());
return false;
}
decoded_frame = frame_result.release_value();
VERIFY(decoded_frame);
}
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
auto& cicp = decoded_frame->cicp();
cicp.adopt_specified_values(frame_sample->container_cicp());
LibVideo: Treat BT.601/709/2020 input transfer characteristics as sRGB I've realized that it probably makes more sense to change the input transfer characteristics to treat these as sRGB since color conversion in linear converted from BT.709 doesn't really make sense. If content creation applications expect media players to display BT.709 without conversions, this means they expect applications to treat it as sRGB, since that's what most displays use. That most likely also means they process it as sRGB internally, meaning we should do the same for our color primaries conversion.
2022-11-08 03:24:30 -06:00
cicp.default_code_points_if_unspecified({ ColorPrimaries::BT709, TransferCharacteristics::BT709, MatrixCoefficients::BT709, ColorRange::Studio });
// BT.601, BT.709 and BT.2020 have a similar transfer function to sRGB, so other applications
// (Chromium, VLC) forgo transfer characteristics conversion. We will emulate that behavior by
// handling those as sRGB instead, which causes no transfer function change in the output,
// unless display color management is later implemented.
switch (cicp.transfer_characteristics()) {
case TransferCharacteristics::BT601:
case TransferCharacteristics::BT709:
case TransferCharacteristics::BT2020BitDepth10:
case TransferCharacteristics::BT2020BitDepth12:
cicp.set_transfer_characteristics(TransferCharacteristics::SRGB);
break;
default:
break;
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
auto bitmap = TRY_OR_ENQUEUE_ERROR(decoded_frame->to_bitmap());
m_frame_queue->enqueue(FrameQueueItem::frame(bitmap, frame_sample->timestamp()));
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
m_present_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
#if PLAYBACK_MANAGER_DEBUG
auto end_time = Time::now_monotonic();
LibVideo: Rewrite the video frame present function to be more readable The PlaybackManager::update_presented_frame function was getting out of hand and adding seeking was making it illegible. This rewrites it to be (hopefully) quite a bit more readable, and adds a few comments to help future readers of the code. In addition, some helpful debugging prints were added that should help debug any future issues with the player.
2022-11-14 00:48:31 -06:00
dbgln("Decoding took {}ms, queue is {} items", (end_time - start_time).to_milliseconds(), m_frame_queue->size());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
#endif
return true;
}
void PlaybackManager::on_decode_timer()
{
if (!decode_and_queue_one_sample() && is_buffering()) {
set_playback_status(PlaybackStatus::Playing);
return;
}
// Continually decode until buffering is complete
LibVideo: Implement accurate seeking to inter frames in PlaybackManager This implements the PlaybackManager portion of seeking, so that it can seek to any frame in the video by dropping all preceding frames until it reaches the seek point. MatroskaDemuxer currently will only seek to the start of the video. That means that every seek has to drop all the frames until it comes across the target timestamp.
2022-11-14 00:54:21 -06:00
if (is_buffering() || is_seeking())
LibVideo: Make all PlaybackManager Timer::start calls set an interval Timers keep their previously set interval even for single-shot mode. We want all timers to fire immediately if they don't have a delay set in the start() call.
2022-11-14 05:40:59 -06:00
m_decode_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
}
Reference in a new issue Copy permalink