Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-11-07 16:51:01 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Userland/Libraries/LibVideo/PlaybackManager.cpp

273 lines
9.7 KiB
C++
Raw Normal View History

LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
/*
* Copyright (c) 2022, Gregory Bertilson <zaggy1024@gmail.com>
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#include <AK/Format.h>
#include <LibCore/Timer.h>
#include <LibVideo/MatroskaReader.h>
#include <LibVideo/VP9/Decoder.h>
#include "MatroskaDemuxer.h"
#include "PlaybackManager.h"
namespace Video {
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
DecoderErrorOr<NonnullOwnPtr<PlaybackManager>> PlaybackManager::from_file(Core::Object& event_handler, StringView filename)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
{
NonnullOwnPtr<Demuxer> demuxer = TRY(MatroskaDemuxer::from_file(filename));
auto video_tracks = demuxer->get_tracks_for_type(TrackType::Video);
if (video_tracks.is_empty())
return DecoderError::with_description(DecoderErrorCategory::Invalid, "No video track is present"sv);
auto track = video_tracks[0];
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Selecting video track number {}", track.identifier());
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
return make<PlaybackManager>(event_handler, demuxer, track, make<VP9::Decoder>());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
PlaybackManager::PlaybackManager(Core::Object& event_handler, NonnullOwnPtr<Demuxer>& demuxer, Track video_track, NonnullOwnPtr<VideoDecoder>&& decoder)
: m_event_handler(event_handler)
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
, m_main_loop(Core::EventLoop::current())
, m_demuxer(move(demuxer))
, m_selected_video_track(video_track)
, m_decoder(move(decoder))
, m_frame_queue(make<VideoFrameQueue>())
, m_present_timer(Core::Timer::construct())
, m_decode_timer(Core::Timer::construct())
{
m_present_timer->set_single_shot(true);
m_present_timer->set_interval(0);
m_present_timer->on_timeout = [&] { update_presented_frame(); };
m_decode_timer->set_single_shot(true);
m_decode_timer->set_interval(0);
m_decode_timer->on_timeout = [&] { on_decode_timer(); };
}
void PlaybackManager::set_playback_status(PlaybackStatus status)
{
if (status != m_status) {
auto old_status = m_status;
m_status = status;
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Set playback status from {} to {}", playback_status_to_string(old_status), playback_status_to_string(m_status));
if (status == PlaybackStatus::Playing) {
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
if (old_status == PlaybackStatus::Stopped || old_status == PlaybackStatus::Corrupted) {
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
restart_playback();
m_frame_queue->clear();
m_skipped_frames = 0;
}
m_last_present_in_real_time = Time::now_monotonic();
LibVideo: Make all PlaybackManager Timer::start calls set an interval Timers keep their previously set interval even for single-shot mode. We want all timers to fire immediately if they don't have a delay set in the start() call.
2022-11-14 05:40:59 -06:00
m_present_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
} else {
m_last_present_in_media_time = current_playback_time();
m_last_present_in_real_time = Time::zero();
m_present_timer->stop();
}
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<PlaybackStatusChangeEvent>(status, old_status));
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
}
void PlaybackManager::resume_playback()
{
set_playback_status(PlaybackStatus::Playing);
}
void PlaybackManager::pause_playback()
{
set_playback_status(PlaybackStatus::Paused);
}
bool PlaybackManager::prepare_next_frame()
{
if (m_next_frame.has_value())
return true;
if (m_frame_queue->is_empty())
return false;
auto frame_item = m_frame_queue->dequeue();
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
m_next_frame.emplace(move(frame_item));
LibVideo: Make all PlaybackManager Timer::start calls set an interval Timers keep their previously set interval even for single-shot mode. We want all timers to fire immediately if they don't have a delay set in the start() call.
2022-11-14 05:40:59 -06:00
m_decode_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
return true;
}
Time PlaybackManager::current_playback_time()
{
if (is_playing())
return m_last_present_in_media_time + (Time::now_monotonic() - m_last_present_in_real_time);
return m_last_present_in_media_time;
}
Time PlaybackManager::duration()
{
return m_demuxer->duration();
}
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
void PlaybackManager::on_decoder_error(DecoderError error)
{
dbgln("Playback error encountered: {}", error.string_literal());
switch (error.category()) {
case DecoderErrorCategory::EndOfStream:
set_playback_status(PlaybackStatus::Stopped);
break;
default:
set_playback_status(PlaybackStatus::Corrupted);
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<DecoderErrorEvent>(move(error)));
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
break;
}
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
void PlaybackManager::update_presented_frame()
{
bool out_of_queued_frames = false;
Optional<FrameQueueItem> frame_item_to_display;
while (true) {
out_of_queued_frames = out_of_queued_frames || !prepare_next_frame();
if (out_of_queued_frames)
break;
VERIFY(m_next_frame.has_value());
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
if (m_next_frame->is_error() || m_next_frame->timestamp() > current_playback_time())
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
break;
if (frame_item_to_display.has_value()) {
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, "At {}ms: Dropped {} in favor of {}", current_playback_time().to_milliseconds(), frame_item_to_display->debug_string(), m_next_frame->debug_string());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
m_skipped_frames++;
}
frame_item_to_display = m_next_frame.release_value();
}
if (!out_of_queued_frames && frame_item_to_display.has_value()) {
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<VideoFramePresentEvent>(frame_item_to_display->bitmap()));
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
m_last_present_in_media_time = current_playback_time();
m_last_present_in_real_time = Time::now_monotonic();
frame_item_to_display.clear();
}
if (frame_item_to_display.has_value()) {
VERIFY(!m_next_frame.has_value());
m_next_frame = frame_item_to_display;
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Set next frame back to dequeued item {}", m_next_frame->debug_string());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
if (!is_playing())
return;
if (!out_of_queued_frames) {
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
if (m_next_frame->is_error()) {
on_decoder_error(m_next_frame->release_error());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
m_next_frame.clear();
return;
}
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
auto frame_time_ms = (m_next_frame->timestamp() - current_playback_time()).to_milliseconds();
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
VERIFY(frame_time_ms <= NumericLimits<int>::max());
dbgln_if(PLAYBACK_MANAGER_DEBUG, "Time until next frame is {}ms", frame_time_ms);
m_present_timer->start(max(static_cast<int>(frame_time_ms), 0));
return;
}
set_playback_status(PlaybackStatus::Buffering);
LibVideo: Make all PlaybackManager Timer::start calls set an interval Timers keep their previously set interval even for single-shot mode. We want all timers to fire immediately if they don't have a delay set in the start() call.
2022-11-14 05:40:59 -06:00
m_decode_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
void PlaybackManager::restart_playback()
{
m_last_present_in_media_time = Time::zero();
m_last_present_in_real_time = Time::zero();
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
auto result = m_demuxer->seek_to_most_recent_keyframe(m_selected_video_track, 0);
if (result.is_error())
on_decoder_error(result.release_error());
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
void PlaybackManager::post_decoder_error(DecoderError error)
{
VideoPlayer: Make PlaybackManager use OwnPtr VideoPlayerWidget was keeping a reference to PlaybackManager when changing files, so the old and new managers would both send frames to be presented at the same time, causing it to flicker back and forth between the two videos. However, PlaybackManager no longer relies on event bubbling to pass events to its parent. By changing it to send events directly to an Object, it can avoid being ref counted, so that it will get destroyed with its containing object and stop sending events.
2022-11-11 07:55:20 -06:00
m_main_loop.post_event(m_event_handler, make<DecoderErrorEvent>(error));
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
bool PlaybackManager::decode_and_queue_one_sample()
{
if (m_frame_queue->size() >= FRAME_BUFFER_COUNT)
return false;
#if PLAYBACK_MANAGER_DEBUG
auto start_time = Time::now_monotonic();
#endif
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
#define TRY_OR_ENQUEUE_ERROR(expression) \
({ \
auto _temporary_result = ((expression)); \
if (_temporary_result.is_error()) { \
dbgln("Enqueued decoder error: {}", _temporary_result.error().string_literal()); \
m_frame_queue->enqueue(FrameQueueItem::error_marker(_temporary_result.release_error())); \
return false; \
} \
_temporary_result.release_value(); \
})
auto frame_sample = TRY_OR_ENQUEUE_ERROR(m_demuxer->get_next_video_sample_for_track(m_selected_video_track));
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
OwnPtr<VideoFrame> decoded_frame = nullptr;
while (!decoded_frame) {
TRY_OR_ENQUEUE_ERROR(m_decoder->receive_sample(frame_sample->data()));
while (true) {
auto frame_result = m_decoder->get_decoded_frame();
if (frame_result.is_error()) {
if (frame_result.error().category() == DecoderErrorCategory::NeedsMoreInput)
break;
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Allow the VP9 decoder to queue multiple frames Frames will now be queued for retrieval by the user of the decoder. When the end of the current queue is reached, a DecoderError of category NeedsMoreInput will be emitted, allowing the caller to react by displaying what was previously retrieved for sending more samples.
2022-11-03 19:18:38 -05:00
post_decoder_error(frame_result.release_error());
return false;
}
decoded_frame = frame_result.release_value();
VERIFY(decoded_frame);
}
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
auto& cicp = decoded_frame->cicp();
cicp.adopt_specified_values(frame_sample->container_cicp());
LibVideo: Treat BT.601/709/2020 input transfer characteristics as sRGB I've realized that it probably makes more sense to change the input transfer characteristics to treat these as sRGB since color conversion in linear converted from BT.709 doesn't really make sense. If content creation applications expect media players to display BT.709 without conversions, this means they expect applications to treat it as sRGB, since that's what most displays use. That most likely also means they process it as sRGB internally, meaning we should do the same for our color primaries conversion.
2022-11-08 03:24:30 -06:00
cicp.default_code_points_if_unspecified({ ColorPrimaries::BT709, TransferCharacteristics::BT709, MatrixCoefficients::BT709, ColorRange::Studio });
// BT.601, BT.709 and BT.2020 have a similar transfer function to sRGB, so other applications
// (Chromium, VLC) forgo transfer characteristics conversion. We will emulate that behavior by
// handling those as sRGB instead, which causes no transfer function change in the output,
// unless display color management is later implemented.
switch (cicp.transfer_characteristics()) {
case TransferCharacteristics::BT601:
case TransferCharacteristics::BT709:
case TransferCharacteristics::BT2020BitDepth10:
case TransferCharacteristics::BT2020BitDepth12:
cicp.set_transfer_characteristics(TransferCharacteristics::SRGB);
break;
default:
break;
}
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
LibVideo: Handle corrupted video errors without spamming dialogs No longer will the video player explode with error dialogs that then lock the user out of closing them. To avoid issues where the playback state becomes invalid when an error occurs, I've made all decoder errors pass through the frame queue. This way, when a video is corrupted, there should be no chance that the playback state becomes invalid due to setting the state to Corrupted in the event handler while a presentation event is still pending. Or at least I think that was what caused some issues I was seeing :^) This system should be a lot more robust if any future errors need to be handled.
2022-11-09 06:52:37 -06:00
auto bitmap = TRY_OR_ENQUEUE_ERROR(decoded_frame->to_bitmap());
m_frame_queue->enqueue(FrameQueueItem::frame(bitmap, frame_sample->timestamp()));
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
#if PLAYBACK_MANAGER_DEBUG
auto end_time = Time::now_monotonic();
dbgln("Decoding took {}ms", (end_time - start_time).to_milliseconds());
#endif
return true;
}
void PlaybackManager::on_decode_timer()
{
if (!decode_and_queue_one_sample() && is_buffering()) {
set_playback_status(PlaybackStatus::Playing);
return;
}
// Continually decode until buffering is complete
if (is_buffering())
LibVideo: Make all PlaybackManager Timer::start calls set an interval Timers keep their previously set interval even for single-shot mode. We want all timers to fire immediately if they don't have a delay set in the start() call.
2022-11-14 05:40:59 -06:00
m_decode_timer->start(0);
LibVideo: Add PlaybackManager to load and decode videos This file will be the basis for abstracting away the out-of-thread or later out-of-process decoding from applications displaying videos. For now, the demuxer is hardcoded to be MatroskaParser, since that is all we support so far. The demuxer should later be selected based on the file header. The playback and decoding are currently all done on one thread using timers. The design of the code is such that adding threading should be trivial, at least based on an earlier version of the code. For now, though, it's better that this runs in one thread, as the multithreaded approach causes the Video Player to lock up permanently after a few frames are decoded.
2022-10-29 18:16:14 -05:00
}
}
Reference in a new issue Copy permalink