Fixes: use of uninitialized memory
Fixes: 439878388/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_UTVIDEO_DEC_fuzzer-5635866203848704
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
in pack mode the array is passed into decode_plane() without being initialized or used
Fixes: use of uninitialized memory
Fixes: 438780119/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_UTVIDEO_DEC_fuzzer-5464037027807232
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: use of uninitialized meory
Fixes: 428034093/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_UTVIDEO_DEC_fuzzer-6195630160805888
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
VLC_MULTI_ELEM contains an uint8_t array that is supposed
to be treated as an array of uint16_t when the used symbols
have a size of two; otherwise it should be treated as just
an array of uint8_t, but it was not always treated that way:
vlc_multi_gen() initialized the first entry of the array
by writing the symbol via AV_WN16; on big endian systems,
the intended value was instead written into the second entry
of the array (where it would likely be overwritten lateron
during initialization).
read_vlc_multi() also treated this case incorrectly: In case
the code is so long that it needs a classical multi-stage lookup,
the symbol has been written to the destination as if via AV_WN16.
On little endian systems, this sets the correct first symbol and
clobbers (zeroes) the next one, but the next one will be overwritten
lateron anyway, so it won't be recognized. But on big-endian systems,
the first symbol will be set to zero and the actually read symbol
will be put into the slot for the next one (where it will be overwritten
lateron).
This commit fixes this; this fixes the magicyuv and utvideo FATE-tests
on big endian arches.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There are lots of files that don't need it: The number of object
files that actually need it went down from 2011 to 884 here.
Keep it for external users in order to not cause breakages.
Also improve the other headers a bit while just at it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Fixes: mem leak
Fixes: 62164/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_UTVIDEO_fuzzer-6666804266926080
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In particular the encoder used only a small part of the context:
The new encoder context is only 128B here. It used to be 32992.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Therefore use a proper prefix for this API, e.g.
ff_init_vlc_sparse -> ff_vlc_init_sparse
ff_free_vlc -> ff_vlc_free
INIT_VLC_LE -> VLC_INIT_LE
INIT_VLC_USE_NEW_STATIC -> VLC_INIT_USE_STATIC
(The ancient INIT_VLC_USE_STATIC has been removed
in 595324e143, so that
the NEW has been dropped.)
Finally, reorder the flags and change their values
accordingly.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It reduces typing: Before this patch, there were 105 codecs
whose long_name-definition exceeded the 80 char line length
limit. Now there are only nine of them.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
and remove FF_CODEC_CAP_INIT_THREADSAFE
All our native codecs are already init-threadsafe
(only wrappers for external libraries and hwaccels
are typically not marked as init-threadsafe yet),
so it is only natural for this to also be the default state.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This is possible, because every given FFCodec has to implement
exactly one of these. Doing so decreases sizeof(FFCodec) and
therefore decreases the size of the binary.
Notice that in case of position-independent code the decrease
is in .data.rel.ro, so that this translates to decreased
memory consumption.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This increases type-safety by avoiding conversions from/through void*.
It also avoids the boilerplate "AVFrame *frame = data;" line
for non-subtitle decoders.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, codec.h contains both public and private parts
of AVCodec. This exposes the internals of AVCodec to users
and leads them into the temptation of actually using them
and forces us to forward-declare structures and types that
users can't use at all.
This commit changes this by adding a new structure FFCodec to
codec_internal.h that extends AVCodec, i.e. contains the public
AVCodec as first member; the private fields of AVCodec are moved
to this structure, leaving codec.h clean.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Also move FF_CODEC_TAGS_END as well as struct AVCodecDefault.
This reduces the amount of files that have to include internal.h
(which comes with quite a lot of indirect inclusions), as e.g.
most encoders don't need it. It is furthemore in preparation
for moving the private part of AVCodec out of the public codec.h.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The majority of frame-threaded decoders (mainly the intra-only)
need exactly one part of ThreadFrame: The AVFrame. They don't
need the owners nor the progress, yet they had to use it because
ff_thread_(get|release)_buffer() requires it.
This commit changes this and makes these functions work with ordinary
AVFrames; the decoders that need the extra fields for progress
use ff_thread_(get|release)_ext_buffer() which work exactly
as ff_thread_(get|release)_buffer() used to do.
This also avoids some unnecessary allocations of progress AVBuffers,
namely for H.264 and HEVC film grain frames: These frames are not
used for synchronization and therefore don't need a ThreadFrame.
Also move the ThreadFrame structure as well as ff_thread_ref_frame()
to threadframe.h, the header for frame-threaded decoders with
inter-frame dependencies.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Given that the AVCodec.next pointer has now been removed, most of the
AVCodecs are not modified at all any more and can therefore be made
const (as this patch does); the only exceptions are the very few codecs
for external libraries that have a init_static_data callback.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
The Huffman trees used by Ut Video have two important characteristics:
(i) Longer codes are on the left of the tree and (ii) for codes of the
same length, the symbol is descending from left to right in the tree.
Therefore all the information that needs to be transmitted is how long
the code corresponding to a given symbol is; and this is also all that
is transmitted.
Before 341914495e, the decoder used qsort
to sort the (length, symbol) pairs by ascending length and for equal
lengths by ascending symbol. Since said commit, the decoder uses
a first pass over the lengths table to count how many symbols of each
length there are; with (i) one can then easily calculate the code of
the left-most code with a given length in the tree and from there one
can calculate the codes for all entries, using one running counter for
each possible length. This eliminated the explicit qsort in
build_huff().
Yet ff_init_vlc_sparse() sorts the table itself as it has to ensure that
all the entries that will be placed in the same subtable are contiguous.
The tables created now are non-contiguous (they are ordered by symbol
and codes of different length aren't ordered at all; only codes of the
same length are ordered according to (ii)).
This commit therefore modifies the algorithm used to automatically create
tables whose codes are sorted from left to right in the tree. The key to
do so is the observation that the counts obtained in the first pass can
be used to contain the range of the codes of each length in the second
pass: If counts[i] is the count of codes with length i, then the first
counts[32] codes are of length 32, the next counts[31] codes are of
length 31 etc. So one knows the index of the lowest symbol whose code
has length 32 (if any): It is counts[32] - 1 due to (ii), whereas the
index of the lowest symbol whose code has length 31 (if any) is
counts[32] + counts[31] - 1; the index of the second-to-lowest symbol of
length 32 (if existing) is counts[32] - 2 etc.
If one follows the algorithm outlined above, one can switch to
ff_init_vlc_from_lengths() which has no implicit qsort; it also means
that one can offload the computation of the codes.
This turned out to be beneficial for performance: For the sample from
ticket #4044 it decreased the decicycles spent on one call to
build_huff() from 508480 to 340688 (GCC 9.3, looping 10 times over the
file to get enough runs and then repeating this ten times); for another
sample (YUV420p, natural content, 5500 frames, also ten iterations)
the time went down from 382346 to 275533 decicycles.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Ut video format uses Huffman trees which are only implicitly coded
in the bitstream: Only the lengths of the codes are coded, the rest has
to be inferred by the decoder according to the rule that the longer
codes are to the left of shorter codes in the tree and on each level the
symbols are descending from left to right.
Because longer codes are to the left of shorter codes, one needs to know
how many non-leaf nodes there are on each level in order to know the
code of the next left-most leaf (which belongs to the highest symbol on
that level). The current code does this by sorting the entries to be
ascending according to length and (for entries with the same length)
ascending according to their symbols. This array is then traversed in
reverse order, so that the lowest level is dealt with first, so that the
number of non-leaf nodes of the next higher level is known when
processing said level.
But this can also be calculated without sorting: Simply count how many
leaf nodes there are on each level. Then one can calculate the number of
non-leaf nodes on each level iteratively from the lowest level upwards:
It is just half the number of nodes of the level below.
This improves performance: For the sample from ticket #4044 the amount
of decicycles for one call to build_huff() decreased from 1055489 to
446310 for Clang 10 and from 1080306 to 535155 for GCC 9.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Ut Video format stores Huffman tables in its bitstream by coding
the length of a given symbol; it does not code the actual code directly,
instead this is to be inferred by the rule that a symbol is to the left
of every shorter symbol in the Huffman tree and that for symbols of the
same length the symbol is descending from left to right. With one
exception, this is also what our de- and encoder did.
The exception only matters when there are codes of length 32, because
in this case the first symbol of this length did not get the code 0,
but 1; this is tantamount to pretending that there is a (nonexistent)
leaf of length 32. This is simply false. The reference software agrees
with this [1].
[1]: 2700a471a7/utv_core/HuffmanCode.cpp (L280)
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: signed integer overflow: 2147483594 + 142 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_UTVIDEO_fuzzer-5658568101724160
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This avoids mixing 8bit variants with pro and 10bit with non pro mode.
Fixes: out of array read
Fixes: poc_03_30.avi
Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '7c25523cc8':
utvideodec: Fix decoding odd sizes with interlaced video with some formats
See 9ef21a897c
Merged-by: James Almer <jamrial@gmail.com>
0.5% faster loop
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Steven Liu <lingjiujianke@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is not needed when the buffer is large enough for the worst case of a line
2% faster vlc reading
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is actually internal utvideo format.
Allows to make use of SIMD for median prediction for rgb(a) formats,
thus speeding up decoding.
Simplifies code, eases further developement and maintenance.
Update FATE because of pixel format switch.
Signed-off-by: Paul B Mahol <onemda@gmail.com>
