Stowage/ladybird
2
0
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2025-12-08 06:09:58 +00:00
Code Issues Projects Releases Packages Wiki Activity
ladybird/Ladybird/AudioCodecPluginLadybird.cpp

265 lines
7.9 KiB
C++
Raw Normal View History

Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
/*
* Copyright (c) 2023, Tim Flynn <trflynn89@serenityos.org>
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#include "AudioCodecPluginLadybird.h"
#include <AK/Endian.h>
#include <AK/MemoryStream.h>
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
#include <LibAudio/Loader.h>
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
#include <LibAudio/Sample.h>
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
#include <LibCore/SharedCircularQueue.h>
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
#include <QAudioFormat>
#include <QAudioSink>
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
#include <QByteArray>
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
#include <QMediaDevices>
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
#include <QThread>
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
namespace Ladybird {
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
static constexpr u32 UPDATE_RATE_MS = 50;
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
struct AudioTask {
enum class Type {
Stop,
Play,
Pause,
Seek,
Volume,
};
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
Type type;
Optional<double> data {};
};
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
using AudioTaskQueue = Core::SharedSingleProducerCircularQueue<AudioTask>;
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
class AudioThread final : public QThread { // We have to use QThread, otherwise internal Qt media QTimer objects do not work.
Q_OBJECT
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
public:
static ErrorOr<NonnullOwnPtr<AudioThread>> create(NonnullRefPtr<Audio::Loader> loader)
{
auto task_queue = TRY(AudioTaskQueue::create());
return adopt_nonnull_own_or_enomem(new (nothrow) AudioThread(move(loader), move(task_queue)));
}
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
ErrorOr<void> stop()
{
TRY(queue_task({ AudioTask::Type::Stop }));
wait();
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
return {};
}
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
Duration duration() const
{
return m_duration;
}
ErrorOr<void> queue_task(AudioTask task)
{
return m_task_queue.blocking_enqueue(move(task), []() {
usleep(UPDATE_RATE_MS * 1000);
});
}
Q_SIGNALS:
void playback_position_updated(Duration);
private:
AudioThread(NonnullRefPtr<Audio::Loader> loader, AudioTaskQueue task_queue)
: m_loader(move(loader))
, m_task_queue(move(task_queue))
{
auto duration = static_cast<double>(m_loader->total_samples()) / static_cast<double>(m_loader->sample_rate());
m_duration = Duration::from_milliseconds(static_cast<i64>(duration * 1000.0));
m_samples_to_load_per_buffer = static_cast<size_t>(UPDATE_RATE_MS / 1000.0 * static_cast<double>(m_loader->sample_rate()));
}
enum class Paused {
Yes,
No,
};
void run() override
{
auto devices = make<QMediaDevices>();
auto const& device_info = devices->defaultAudioOutput();
auto format = device_info.preferredFormat();
format.setSampleFormat(QAudioFormat::Int16);
format.setChannelCount(2);
auto audio_output = make<QAudioSink>(device_info, format);
auto* io_device = audio_output->start();
auto paused = Paused::Yes;
while (true) {
if (auto result = m_task_queue.dequeue(); result.is_error()) {
VERIFY(result.error() == AudioTaskQueue::QueueStatus::Empty);
} else {
auto task = result.release_value();
switch (task.type) {
case AudioTask::Type::Stop:
return;
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
case AudioTask::Type::Play:
audio_output->resume();
paused = Paused::No;
break;
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
case AudioTask::Type::Pause:
audio_output->suspend();
paused = Paused::Yes;
break;
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
case AudioTask::Type::Seek: {
VERIFY(task.data.has_value());
auto position = *task.data;
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
auto duration = static_cast<double>(this->duration().to_milliseconds()) / 1000.0;
position = position / duration * static_cast<double>(m_loader->total_samples());
m_loader->seek(static_cast<int>(position)).release_value_but_fixme_should_propagate_errors();
if (paused == Paused::Yes) {
m_position = Web::Platform::AudioCodecPlugin::current_loader_position(m_loader, audio_output->format().sampleRate());
Q_EMIT playback_position_updated(m_position);
}
break;
}
case AudioTask::Type::Volume:
VERIFY(task.data.has_value());
audio_output->setVolume(*task.data);
break;
}
}
if (paused == Paused::No) {
if (auto result = play_next_samples(*audio_output, *io_device); result.is_error()) {
// FIXME: Propagate the error to the HTMLMediaElement.
} else {
Q_EMIT playback_position_updated(m_position);
paused = result.value();
}
}
usleep(UPDATE_RATE_MS * 1000);
}
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
ErrorOr<Paused> play_next_samples(QAudioSink& audio_output, QIODevice& io_device)
{
bool all_samples_loaded = m_loader->loaded_samples() >= m_loader->total_samples();
if (all_samples_loaded) {
audio_output.suspend();
(void)m_loader->reset();
m_position = m_duration;
return Paused::Yes;
}
auto samples = TRY(Web::Platform::AudioCodecPlugin::read_samples_from_loader(*m_loader, m_samples_to_load_per_buffer, audio_output.format().sampleRate()));
enqueue_samples(io_device, move(samples));
m_position = Web::Platform::AudioCodecPlugin::current_loader_position(m_loader, audio_output.format().sampleRate());
return Paused::No;
}
void enqueue_samples(QIODevice& io_device, FixedArray<Audio::Sample> samples)
{
auto buffer_size = samples.size() * 2 * sizeof(u16);
if (buffer_size > static_cast<size_t>(m_sample_buffer.size()))
m_sample_buffer.resize(buffer_size);
FixedMemoryStream stream { Bytes { m_sample_buffer.data(), buffer_size } };
for (auto& sample : samples) {
LittleEndian<i16> pcm;
pcm = static_cast<i16>(sample.left * NumericLimits<i16>::max());
MUST(stream.write_value(pcm));
pcm = static_cast<i16>(sample.right * NumericLimits<i16>::max());
MUST(stream.write_value(pcm));
}
io_device.write(m_sample_buffer.data(), buffer_size);
}
NonnullRefPtr<Audio::Loader> m_loader;
AudioTaskQueue m_task_queue;
size_t m_samples_to_load_per_buffer { 0 };
QByteArray m_sample_buffer;
Duration m_duration;
Duration m_position;
};
ErrorOr<NonnullOwnPtr<AudioCodecPluginLadybird>> AudioCodecPluginLadybird::create(NonnullRefPtr<Audio::Loader> loader)
{
auto audio_thread = TRY(AudioThread::create(move(loader)));
audio_thread->start();
return adopt_nonnull_own_or_enomem(new (nothrow) AudioCodecPluginLadybird(move(audio_thread)));
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
AudioCodecPluginLadybird::AudioCodecPluginLadybird(NonnullOwnPtr<AudioThread> audio_thread)
: m_audio_thread(move(audio_thread))
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
{
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
connect(m_audio_thread, &AudioThread::playback_position_updated, this, [this](auto position) {
if (on_playback_position_updated)
on_playback_position_updated(position);
});
}
AudioCodecPluginLadybird::~AudioCodecPluginLadybird()
{
m_audio_thread->stop().release_value_but_fixme_should_propagate_errors();
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
void AudioCodecPluginLadybird::resume_playback()
{
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
m_audio_thread->queue_task({ AudioTask::Type::Play }).release_value_but_fixme_should_propagate_errors();
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
void AudioCodecPluginLadybird::pause_playback()
{
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
m_audio_thread->queue_task({ AudioTask::Type::Pause }).release_value_but_fixme_should_propagate_errors();
}
void AudioCodecPluginLadybird::set_volume(double volume)
{
AudioTask task { AudioTask::Type::Volume };
task.data = volume;
m_audio_thread->queue_task(move(task)).release_value_but_fixme_should_propagate_errors();
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
void AudioCodecPluginLadybird::seek(double position)
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
{
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
AudioTask task { AudioTask::Type::Seek };
task.data = position;
m_audio_thread->queue_task(move(task)).release_value_but_fixme_should_propagate_errors();
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
Duration AudioCodecPluginLadybird::duration()
LibWeb: Expose volume controls through the platform audio plugin
2023-06-14 12:59:22 -04:00
{
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
return m_audio_thread->duration();
LibWeb: Expose volume controls through the platform audio plugin
2023-06-14 12:59:22 -04:00
}
Ladybird+LibWeb+WebContent: Create a platform plugin for playing audio This creates (and installs upon WebContent startup) a platform plugin to play audio data. On Serenity, we use AudioServer to play audio over IPC. Unfortunately, AudioServer is currently coupled with Serenity's audio devices, and thus cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device to play the audio, which requires the Qt multimedia package. While we use Qt to play the audio, note that we can still use LibAudio to decode the audio data and retrieve samples - we simply send Qt the raw PCM signals.
2023-06-12 13:44:10 -04:00
}
Ladybird+LibWeb+WebConent: Drive audio in Ladybird off the main thread The main thread in the WebContent process is often busy with layout and running JavaScript. This can cause audio to sound jittery and crack. To avoid this behavior, we now drive audio on a secondary thread. Note: Browser on Serenity uses AudioServer, the connection for which is already handled on a secondary thread within LibAudio. So this only applies to Lagom. Rather than using LibThreading, our hands are tied to QThread for now. Internally, the Qt media objects use a QTimer, which is forbidden from running on a thread that is not a QThread (the debug console is spammed with messages pointing this out). Ideally, in the future AudioServer will be able to run for non-Serenity platforms, and most of this can be aligned with the Serenity implementation.
2023-06-20 12:43:04 -04:00
#include "AudioCodecPluginLadybird.moc"
Reference in a new issue Copy permalink