The way streams are coded in an IAMF struct follows a scalable model where the
channel layouts for each layer may not match the channel order our API can
represent in a Native order layout.
For example, an audio element may have six coded streams in the form of two
stereo streams, followed by two mono streams, and then by another two stereo
streams, for a total of 10 channels, and define for them four scalable layers
with loudspeaker_layout values "Stereo", "5.1ch", "5.1.2ch", and "5.1.4ch".
The first layer references the first stream, and each following layer will
reference all previous streams plus extra ones.
In this case, the "5.1ch" layer will reference four streams (the first two
stereo and the two mono) to encompass six channels, which does not match out
native layout 5.1(side) given that FC and LFE come after FL+FR but before
SL+SR, and here, they are at the end.
For this reason, we need to build Custom order layouts that properly represent
what we're exporting.
----
Before:
Stream group #0:0[0x12c]: IAMF Audio Element:
Layer 0: stereo
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Layer 1: 5.1(side)
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream #0:1[0x1]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:2[0x2]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:3[0x3]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Layer 2: 5.1.2
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream #0:1[0x1]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:2[0x2]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:3[0x3]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:4[0x4]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Layer 3: 5.1.4
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream #0:1[0x1]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:2[0x2]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:3[0x3]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:4[0x4]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:5[0x5]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
----
AFter:
Stream group #0:0[0x12c]: IAMF Audio Element:
Layer 0: stereo
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Layer 1: 6 channels (FL+FR+SL+SR+FC+LFE)
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream #0:1[0x1]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:2[0x2]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:3[0x3]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Layer 2: 8 channels (FL+FR+SL+SR+FC+LFE+TFL+TFR)
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream #0:1[0x1]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:2[0x2]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:3[0x3]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:4[0x4]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Layer 3: 10 channels (FL+FR+SL+SR+FC+LFE+TFL+TFR+TBL+TBR)
Stream #0:0[0x0]: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream #0:1[0x1]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:2[0x2]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:3[0x3]: Audio: opus, 48000 Hz, mono, fltp (dependent)
Stream #0:4[0x4]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Stream #0:5[0x5]: Audio: opus, 48000 Hz, stereo, fltp (dependent)
Signed-off-by: James Almer <jamrial@gmail.com>
Using audio_substream_id for AVStream ids is not ideal give that in containers
like mp4, the IAMF structure is opaque to the outside and other streams may
share such id values.
Signed-off-by: James Almer <jamrial@gmail.com>
Layouts with both pairs (7.1, 7.1.2, etc) in IAMF that follow the definition in
ITU-R BS.2051-3 for Systems I and J also follow its ordering. This means side
comes before back, which is the inverse of how it's defined in AVChannel.
To workaround this without having to use custom order channel layouts, swap the
stream ids in the input IAMF structure, so packets for one are mapped to the
other.
Signed-off-by: James Almer <jamrial@gmail.com>
and rename it to FF_INFMT_INIT_CLEANUP. This flag is demuxer-only,
so this is the more appropriate place for it.
This does not preclude adding internal flags common to both
demuxer and muxer in the future.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This commit does for AVInputFormat what commit
59c9dc82f4 did for AVOutputFormat:
It adds a new type FFInputFormat, moves all the internals
of AVInputFormat to it and adds a now reduced AVInputFormat
as first member.
This does not affect/improve extensibility of both public
or private fields for demuxers (it is still a mess due to lavd).
This is possible since 50f34172e0
(which removed the last usage of an internal field of AVInputFormat
in fftools).
(Hint: tools/probetest.c accesses the internals of FFInputFormat
as well, but given that it is a testing tool this is not considered
a problem.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
if there's an audio layer with a single stream that can be rendered alone, mark it
as default. Otherwise, mark every stream as dependent.
Signed-off-by: James Almer <jamrial@gmail.com>
IAMFAudioElement and IAMFMixPresentation currently contain
pointers to independently allocated objects that are sometimes
owned by said structures and sometimes not.
More precisely, upon success the demuxer transfers ownership
of these other objects newly created AVStreamGroups, but it
keeps its pointers. iamf_read_close() therefore always resets
these pointers (because the cleanup code always treats them
as ownership pointers). This leads to memory leaks in case
iamf_read_header() without having attached all of these
objects to stream groups.
The muxer has a similar issue: It also clears these pointers
(pointing to objects owned by stream groups created by the user)
in its deinit function.
This commit fixes this memleak by explicitly adding non-ownership
pointers; this also allows to remove the code to reset the
ownership pointers.
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The AVIAMFAudioElement and AVIAMFMixPresentation that are ultimately used
are allocated by ff_iamfdec_read_descriptors().
Fixes some memory leaks reported by Valgrind.
Signed-off-by: James Almer <jamrial@gmail.com>
