Commit graph

41137 commits

Author SHA1 Message Date
Michael Niedermayer
46a1b4e8ac avcodec/vc1_pred: Fix invalid shift in scaleforsame()
Fixes: left shift of negative value -1
Fixes: 15531/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5759556258365440

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6dfda35dd2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
38de5403c8 avcodec/vc1_block: Fix integer overflow in ff_vc1_pred_dc()
Fixes: signed integer overflow: 32796 * 65536 cannot be represented in type 'int'
Fixes: 15430/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5735424087031808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f31ed8f3b0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
9431f39c91 avcodec/truemotion2: Fix several integer overflows in tm2_motion_block()
Fixes: 15524/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5173148372172800
Fixes: signed integer overflow: 13701388 - -2134868270 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9a353ea876)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
7dbc888de6 avcodec/apedec: make left/right unsigned to avoid undefined behavior
Fixes: signed integer overflow: 755176387 + 1515360583 cannot be represented in type 'int'
Fixes: 15506/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5706859232624640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bf778af149)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
102bc56731 avcodec/apedec: Fix multiple integer overflows and undefined behaviorin filter_3800()
Fixes: left shift of negative value -4
Fixes: signed integer overflow: -15091694 * 167 cannot be represented in type 'int'
Fixes: signed integer overflow: 1898547155 + 453967445 cannot be represented in type 'int'
Fixes: 15258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5759095564402688
Fixes: signed integer overflow: 962196438 * 31 cannot be represented in type 'int'
Fixes: 15364/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718799845687296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 267eb2ab7f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
09d321d711 avcodec/eatqi: Check for minimum frame size
The minimum header is 8 bytes, the smallest bitstream that is passed to
the MB decode code is 4 bytes

Fixes: Timeout (35sec -> 18sec)
Fixes: 15800/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EATQI_fuzzer-5684154517159936

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5ffb8e8793)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
9738f204ce avcodec/eatgv: Check remaining size after the keyframe header
The minimal size which unpack() will not fail on is 5 bytes
Fixes: Timeout (14sec -> 77ms) (testcase 15508)
Fixes: 15508/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EATGV_fuzzer-5700053513011200
Fixes: 15996/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EATGV_fuzzer-5751353223151616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 009ec8dc33)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
16f98b9c29 avcodec/assdec: undefined use of memcpy()
Fixes: null pointer passed as argument 2, which is declared to never be null
Fixes: 16008/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SSA_fuzzer-5650582821404672 (this is a separate issue found in this testcase)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47b6ca0b02)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
904a8616a5 avcodec/brenderpix: Check input size before allocating image
An incomplete image is not supported prior to this and will
not produce any output. This commit moves the failure before
time consuming operations.

Fixes: Timeout (81sec -> 76ms)
Fixes: 15723/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BRENDER_PIX_fuzzer-5147265653538816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 38b6c48c43)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
bf28f5f367 avcodec/utils: fix leak of subtitle_header on error path
Fixes: memleak
Fixes: 15528/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_STL_fuzzer-5735993371525120
Fixes: 15792/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SAMI_fuzzer-5737754232619008
Fixes: 16008/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SSA_fuzzer-5650582821404672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 923d5c489f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
f2c01cc90b avcodec/utils: Check close before calling it
Fixes: NULL pointer dereference
Fixes: 15733/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IDF_fuzzer-5658616977162240

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8df6884832)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
c24ff0743f avcodec/vorbisdec: Check vlc for floor0 dec vector offset
Fixes: out of array access
Fixes: 15649/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-5729191309344768

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 99f95f39c6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
19866e4d63 avcodec/vorbisdec: amplitude bits can be more than 25 bits
Fixes: assertion failure, invalid shift
Fixes: 15583/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-5640157484548096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 308771a738)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
f02de2f599 avcodec/apedec: Fix various integer overflows
Fixes: signed integer overflow: -538976267 * 31 cannot be represented in type 'int'
Fixes: left shift of 65312 by 16 places cannot be represented in type 'int'
Fixes: 15255/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718831688843264
Fixes: 15547/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5691384901664768

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 240bf0e596)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
14b5fa9e4e avcodec/apedec: Fix multiple integer overflows in predictor_update_filter()
Fixes: signed integer overflow: -829262115 + -1410750414 cannot be represented in type 'int'
Fixes: 15251/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5651742252859392

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0af08cb803)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
aee5c9494d avcodec/alsdec: fix undefined shift in multiply()
Fixes: left shift of negative value -6
Fixes: 15564/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALS_fuzzer-5701655938465792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b880b3b236)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
882dee37cc avcodec/alsdec: Fix 2 integer overflows
Fixes: signed integer overflow: 1270564968 + 904828220 cannot be represented in type 'int'
Fixes: 15402/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALS_fuzzer-5755426823471104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9cd0d94f59)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
6a7e27f675 avcodec/flicvideo: Make line_packets int
Fixes: signed integer overflow: -32768 * 196032 cannot be represented in type 'int'
Fixes: 15300/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLIC_fuzzer-5733319519502336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 54bd47f861)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
ee2abfe7d4 avcodec/dvbsubdec: Use ff_set_dimensions()
Fixes: signed integer overflow: 65313 * 65313 cannot be represented in type 'int'
Fixes: 15740/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVBSUB_fuzzer-5641749164195840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5941b7f615)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
dda10a2995 avcodec/ffwavesynth: Check if there is enough extradata before allocation
Fixes: OOM
Fixes: 15750/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5702090367696896

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 65bac4a782)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
8e2a5f951f avcodec/ffwavesynth: More correct cast in wavesynth_seek()
Fixes: signed integer overflow: 553590816 - -9223372036315799520 cannot be represented in type 'long'
Fixes: 15743/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5705835377852416

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f4605770af)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
d69822bd9a avcodec/ffwavesynth: Check sample rate before use
Fixes: division by zero
Fixes: 15725/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5641231956180992

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c95857a423)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
cacf998071 avcodec/dnxhd_parser: Fix parser when input does not have nicely sized packets
Fixes: out of array access
Fixes: 15522/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DNXHD_fuzzer-5747756078989312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d900d8fe0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
01f9c1540a avcodec/dnxhd_parser: remove unneeded code
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1707dbdf49)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
6c821c1dee avcodec/h264_refs: Also check reference in ff_h264_build_ref_list()
Fixes: out of array read
Fixes: 15409/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5758846959616000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7d3581e6bb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
6d3c6e8b96 avcodec/atrac9dec: Check conditions before apply_band_extension() to avoid out of array read in initialization of unused variables
Fixes: global-buffer-overflow
Fixes: 15247/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5671602181636096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c5f265bb24)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
a6d1e2c40a avcodec/parser: Check next index validity in ff_combine_frame()
Fixes: out of array access
Fixes: 15522/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DNXHD_fuzzer-5747756078989312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 15008db0fa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
5c2cbf36dd avcodec/ivi: Ask for samples with odd tiles
Fixes: Assertion failure
Fixes: 15422/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO5_fuzzer-5676625481433088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a7e02cf3ad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
fe36a3005b avcodec/pngdec: Check that previous_picture has same w/h/format
Fixes: out of array access
Fixes: 15540/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APNG_fuzzer-5684905029140480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18c808ffbe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
ec806c3da8 avcodec/huffyuv: remove gray8a (the format is listed but not supported by the implementation)
Fixes: null pointer dereference
Fixes: 15464/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HYMT_fuzzer-5681391150301184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6aaa01afe4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
Michael Niedermayer
41c9e6128c avcodec/mpc8: Fixes invalid shift in mpc8_decode_frame()
Fixes: left shift of negative value -456
Fixes: 15561/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPC8_fuzzer-5758130404720640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Suggested-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1dbb67d39b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
James Zern
e8b0f20e51 avcodec/utils, avcodec_open2: close codec on failure
after a successful init if the function fails for another reason close
the codec without requiring FF_CODEC_CAP_INIT_CLEANUP which is meant to
cover init failures themselves. fixes a memory leak in those cases.

BUG=oss-fuzz:15529

Signed-off-by: James Zern <jzern@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b1febda061)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-06 11:30:42 +01:00
James Almer
08d3cc2f1d avcodec/amfnec: allocate packets using av_new_packet()
This ensures they will be reference counted, as required by the AVCodec.receive_packet()
API.

Should fix ticket #8386.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit fdf46b4a6b)
2019-11-19 19:49:09 -03:00
Timo Rothenpieler
4df5dfabcc avcodec/nvenc: make sure newly allocated packets are refcounted
Fixes ticket 8383

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2019-11-17 01:47:49 +01:00
Jun Zhao
4fbeaaa220 lavc/mpeg4audio: add chan_config check to avoid indeterminate channels
add chan_config check to avoid indeterminate channels.

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 333109f469)
2019-09-27 22:25:13 -03:00
Timo Rothenpieler
fe1064f779 avcodec/nvenc: add driver version info for latest SDKs
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2019-09-24 12:07:39 +02:00
James Almer
11cce24e04 avcodec/bsf: check that AVBSFInternal was allocated before dereferencing it
This can happen when av_bsf_free() is called on av_bsf_alloc() failure.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit d889ae3396)
2019-09-23 10:11:02 -03:00
Carl Eugen Hoyos
0253771953 lavc/tableprint_vlc: Remove avpriv_request_sample() from included files.
Fixes compilation with --enable-hardcoded-tables.
Fixes ticket #7962.

(cherry picked from commit c8232e5007)
2019-08-09 10:20:20 +02:00
Stefan Schoenefeld
a12e40e021 avcodec/h263dec: fix hwaccel decoding
Recently we encountered an issue when decoding a h.263 file:

FFmpeg will freeze when decoding h.263 video with NVDEC. Turns out this is not directly related to NVDEC but is a problem that shows with several other HW decoders like VDPAU, though the exact kind of error is different (either error messages or freezing[1]). The root cause is that ff_thread_finish_setup() is called twice per frame from ff_h263_decode_frame(). This is not supported by ff_thread_finish_setup() and specifically checked for and warned against in the functions code. The issue is also specific to hw accelerated decoding only as the second call to ff_thread_finish_setup() is only issued when hw acceleration is on. The fix is simple: add a check that the first call is only send when hw acceleration is off, and the second call only when hw acceleration is on (see attached patch). This works fine as far as I was able to test with vdpau and nvdec/nvcuvid hw decoding. The patch also adds NVDEC to the hw config list if available.

I also noticed a secondary issue when browsing through the code which is that, according to documentation, ff_thread_finish_setup() should only be called if the codec implements update_thread_context(), which h263dec does not. The patch does not address this and I'm not sure any action needs to be taken here at all.

[1] This is depending on whether or not the hw decoder sets the  HWACCEL_CAPS_ASYNC_SAFE flag

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2019-08-04 16:21:11 +02:00
Andreas Rheinhardt
6e0cf9a9df cbs_h2645: Fix infinite loop in more_rbsp_data
cbs_h2645_read_more_rbsp_data does not handle malformed input very well:
1. If there were <= 8 bits left in the bitreader, these bits were read
via show_bits. But show_bits requires the number of bits to be read to
be > 0 (internally it shifts by 32 - number of bits to be read which is
undefined behaviour if said number is zero; there is also an assert for
this, but it is only an av_assert2). Furthermore, in this case a shift
by -1 was performed which is of course undefined behaviour, too.
2. If there were > 0 and <= 8 bits left and all of them were zero
(this can only happen for defective input), it was reported that there
was further RBSP data.

This can lead to an infinite loop in H.265's cbs_h265_read_extension_data
corresponding to the [vsp]ps_extension_data_flag syntax elements. If the
relevant flag indicates the (potential) occurence of these syntax elements,
while all bits after this flag are zero, cbs_h2645_read_more_rbsp_data
always returns 1 on x86. Given that a checked bitstream reader is used,
we are also not "saved" by an overflow in the bitstream reader's index.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d4035ca849)
2019-07-21 11:07:21 -03:00
Andreas Rheinhardt
1b6bcee9fe cbs_mpeg2: Fix storage type for frame_centre_*_offset
The frame_centre_horizontal/vertical_offset values contained in picture
display extensions are actually signed values (i.e. it is possible to
indicate that the display device should add black bars/pillars).

The files sony-ct3.bs and tcela-6.bits (which are both used in fate
tests for mpeg2_metadata) contain picture display extensions; the former
even contains a negative frame_centre_vertical_offset. Fortunately, the
old code did not damage the picture display extensions when one did a
cycle of reading and writing. For the same reason the fate tests needn't
be updated either.

Furthermore these fields now use the trace output for matrices.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit de58803839)
2019-07-21 01:05:39 -03:00
Andreas Rheinhardt
b010caa6c9 cbs_mpeg2: Improve checks for invalid values
MPEG-2 contains several elements that mustn't be zero according to the
specifications: horizontal/vertical_size_value, aspect_ratio_information,
frame_rate_code, the quantiser matrices, the colour_description
elements, picture_coding_type, the f_code[r][s] values and
quantiser_scale_code. It is now checked that the invalid values don't
occur.

The colour_description elements are treated specially in this regard:
Given that there are files in the wild which use illegal values for the
colour_description elements (some of them created by mpeg2_metadata),
they will be corrected to the value meaning "unknown" (namely 2) during
reading. This has been done in such a way that trace_headers will
nevertheless report the original value, together with a message about
the fixup.

Furthermore, the trace_headers output of user_data has been beautified.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9c3f2a8894)
2019-07-21 01:05:24 -03:00
James Almer
ae5c80b9ca avcodec/cbs_mpeg2: fix leak of extra_information_slice buffer in cbs_mpeg2_read_slice_header()
cbs_mpeg2_free_slice() calls av_buffer_unref() on extra_information_ref,
meaning allocating with av_malloc() was not the intention.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit d903c09d9a)
2019-07-21 01:04:42 -03:00
Carl Eugen Hoyos
b3b5941ec7 lavc/cbs: Do not use format specifier "z" on Windows.
(cherry picked from commit 0b7269e62d)
2019-07-21 01:04:34 -03:00
Carl Eugen Hoyos
84b94fdd05 lavc/cbs_vp9: Make variable prob unsigned.
Silences a warning with clang:
libavcodec/cbs_vp9_syntax_template.c:220:17: warning: implicit conversion from 'int' to 'int8_t' (aka 'signed char')
      changes value from 255 to -1

(cherry picked from commit de441ad52a)
2019-07-21 01:04:28 -03:00
James Almer
a213213985 avcodec/cbs_h264: fix storage type for time_offset in Pic Timing SEI
The spec defines it as a signed value.

Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 9bf520d04d)
2019-07-21 01:04:23 -03:00
James Almer
94b1630b7c avcodec/cbs_h2645: add helper macros for signed values
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 3dc6adf326)
2019-07-21 01:04:15 -03:00
James Almer
7dc2366533 avcodec/cbs: add helper functions and macros to read and write signed values
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 5006dcdf9a)
2019-07-21 01:04:10 -03:00
Andreas Rheinhardt
2ac6315c7c cbs_h264: Fix handling of auxiliary pictures
The earlier code used the most recent non-auxiliary slice to determine
whether an auxiliary slice has the syntax of an IDR slice, even when
the most recent slice was from a slice of a redundant frame. Now only
slices of the primary coded picture are used, as the specifications
mandate.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com>
(cherry picked from commit 8d1cf2d894)
2019-07-21 01:04:04 -03:00
Michael Niedermayer
7654e5aa3b avcodec/ilbcdec: Simplify use of unsigned and fix more undefined overflows
Fixes: signed integer overflow: 2147475672 + 8192 cannot be represented in type 'int'
Fixes: 15415/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ILBC_fuzzer-5712074128228352

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 019d729039)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-08 11:52:25 +02:00