Commit graph

41875 commits

Author SHA1 Message Date
Michael Niedermayer
02346292a3 avcodec/alsdec: fix mantisse shift
Fixes: shift exponent -1 is negative
Fixes: 16039/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALS_fuzzer-5656825657032704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-11 19:13:21 +02:00
Michael Niedermayer
70432eac0b avcodec/pngdec: consider chunk size in minimal size check
assuming each block contains an empty chunk there has to be at least 8 bytes extra.

Fixes: 15327/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LSCR_fuzzer-5676669303521280
Fixes: Timeout (11->5sec)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-11 19:13:21 +02:00
Michael Niedermayer
c9415e815a avcodec/vc1_block: Fix invalid shifts in vc1_decode_i_blocks()
Fixes: left shift of negative value -9
Fixes: 15299/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MSS2_fuzzer-5660922678345728
Fixes: 15557/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5673351911047168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-11 19:13:21 +02:00
Michael Niedermayer
b153ba1c2e avcodec/vc1_block: fix invalid shift in vc1_decode_p_mb()
Fixes: left shift of negative value -5
Fixes: 15294/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1_fuzzer-5733921754447872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-11 19:13:21 +02:00
Michael Niedermayer
da93e2b142 avcodec/aacdec_template: fix integer overflow in imdct_and_windowing()
Fixes: signed integer overflow: 2147483645 + 4 cannot be represented in type 'int'
Fixes: 15418/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-5685269069561856

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-11 19:13:21 +02:00
Limin Wang
1ee863a7b0 lavc/videotoolboxenc: make transfer_fnc initialized for unsupport function
The current function will report one error message, but the caller func
haven't check it, so change the default to process as AVCOL_TRC_UNSPECIFIED.

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Rick Kern <kernrj@gmail.com>
2019-08-11 10:52:15 -04:00
Limin Wang
391b67fcb5 lavc/videotoolboxenc: add hdr10, linear, hlg color transfer function for videotoolboxenc
Below is the testing ffmpeg command for the setting:
./ffmpeg -i input.ts -c:v hevc_videotoolbox -color_primaries bt2020 -colorspace bt2020_ncl -color_trc smpte2084 smpte2048.ts
./ffmpeg -i input.ts -c:v hevc_videotoolbox -color_primaries bt2020 -colorspace bt2020_ncl -color_trc linear linear.ts
./ffmpeg -i input.ts -c:v hevc_videotoolbox -color_primaries bt2020 -colorspace bt2020_ncl -color_trc arib-std-b67 hlg.ts

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Rick Kern <kernrj@gmail.com>
2019-08-11 10:52:15 -04:00
Carl Eugen Hoyos
2828f5b0d8 lavc/r210enc: Fix undefined behaviour encoding r10k.
Fixes the following ubsan error:
libavcodec/r210enc.c:69:28: runtime error: left shift of 522 by 22 places cannot be represented in type 'int'

Fixes ticket #7982.
2019-08-11 02:06:44 +02:00
Carl Eugen Hoyos
ac457a3bc5 lavc/vc2enc_dwt: Avoid left-shifting a negative value.
Fixes ticket #7985.
2019-08-11 00:06:12 +02:00
Carl Eugen Hoyos
e82a619c2a lavc/frame_thread_encoder: Do not memcpy() from NULL.
Fixes ticket #7981.
2019-08-11 00:03:35 +02:00
Carl Eugen Hoyos
690cab3232 lavc/libx264: Cast cpb bit_rates to int64_t to avoid an integer overflow.
Fixes remaining part of ticket #8071 on next version bump.
2019-08-10 23:52:59 +02:00
Carl Eugen Hoyos
4b1687f23c lavc/libx264: Cast bit_rate to int64_t to avoid an integer overflow.
Fixes ticket #8071.
2019-08-10 23:29:58 +02:00
Carl Eugen Hoyos
57987deefc lavc/libx264: bit_rates > INT_MAX are not supported. 2019-08-10 23:27:58 +02:00
Michael Niedermayer
be4fb282f9 avcodec/alsdec: Check for block_length <= 0 in read_var_block_data()
Fixes: left shift of negative value -1
Fixes: 15719/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALS_fuzzer-5685731105701888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 18:44:34 +02:00
Michael Niedermayer
02f909dc24 avcodec/vqavideo: Set video size
Fixes: out of array access
Fixes: 15919/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VQA_fuzzer-5657368257363968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 18:43:45 +02:00
Michael Niedermayer
172a43ce36 avcodec/sanm: Check extradata_size before allocations
Fixes: Leaks
Fixes: 15349/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SANM_fuzzer-5102530557640704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 18:42:51 +02:00
Andreas Rheinhardt
f12e662a3d libavcodec/iff: Use unsigned to avoid undefined behaviour
The initialization of the uint32_t plane32_lut matrix uses left shifts
of the form 1 << plane; plane can be as big as 31 which means that this
is undefined behaviour as 1 will be simply an int. So make it unsigned
to avoid this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
43015afd7c avcodec/mss1: check for overread and forward errors
Fixes: Timeout (106sec -> 14ms)
Fixes: 15576/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MSS1_fuzzer-5688080461201408

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
8305a4509a avcodec/loco: Check for end of input in pixel decode
Fixes: Timeout (100sec -> 5sec)
Fixes: 15509/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5724297261219840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
549fcba8fc avcodec/dirac_parser: Fix overflow in dts
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int'
Fixes: 15568/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5634719611355136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
3c06ba1716 avcodec/ralf: Fix undefined pointer in decode_channel()
Fixes: 16203/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5086088934195200

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
ccca484324 avcodec/ralf: Fix integer overflow in apply_lpc()
Fixes: signed integer overflow: 1603085316 + 1238786562 cannot be represented in type 'int'
Fixes: 16203/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5086088934195200

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
5a5f12e3b3 avcodec/vorbisdec: Implement vr->classifications = 1
It appears no valid file uses this, so this is not testable with
a valid file.

Fixes: assertion failure
Fixes: 16187/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-5638880618872832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
aecc9b96d6 avcodec/vorbisdec: Check parameters in vorbis_floor0_decode() before divide
Fixes: division by zero
Fixes: 16183/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-5688966782648320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
58e084bd2c avcodec/cfhd: Check destination space for bayer before writing
Fixes: out of array write
Fixes: 16105/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5690817309573120
Fixes: 16119/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5099050675732480
Fixes: 16135/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5705501601431552

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
fe2e1cfc19 avcodec/cfhd: Fix linesize type
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:54:24 +02:00
Michael Niedermayer
8174e5c77d avcodec/cbs_av1_syntax_template: Check ref_frame_idx before use
Fixes: index -1 out of bounds for type 'AV1ReferenceFrameState [8]'
Fixes: 16079/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5758807440883712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
See: [FFmpeg-devel] [PATCH 05/13] avcodec/cbs_av1_syntax_template: Check ref_frame_idx before use
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:53:14 +02:00
Michael Niedermayer
392c028cd2 avcodec/apedec: Fix 2 signed overflows
Fixes: left shift of 1073741824 by 1 places cannot be represented in type 'int'
Fixes: signed integer overflow: 2049431315 + 262759074 cannot be represented in type 'int'
Fixes: 16012/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5719016003338240

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:50:27 +02:00
Michael Niedermayer
99a172f3f4 avcodec/mss3: Check for the rac stream being invalid in rac_normalize()
Fixes: out of array read
Fixes: 15982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MSA1_fuzzer-5630676251967488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:50:27 +02:00
Michael Niedermayer
2cb1f79735 avcodec/vc1_block: Check get_vlc2() return before use
Fixes: index -1 out of bounds for type 'const uint8_t [185][2]'
Fixes: 15720/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MSS2_fuzzer-5666071933091840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:50:27 +02:00
Michael Niedermayer
8e4b522c91 avcodec/apedec: Do not partially clear data array
Fixes: Assertion failure and memleak
Fixes: 15709/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5182435093905408

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:50:27 +02:00
Michael Niedermayer
208225bd78 avcodec/atrac9dec: Check grad_range[1] more tightly
Alternatively the array could be made bigger but the extra values
would not be read without other changes.

Fixes: Out of array access
Fixes: 15658/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5738260074070016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:50:27 +02:00
Michael Niedermayer
d555a220b4 avcodec/atrac9dec: Remove impossible condition
Suggested-by: Lynne <dev@lynne.ee>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 17:50:23 +02:00
Michael Niedermayer
005de57d84 avcodec/bink: Add many end of input checks
Fixes: Timeout (83sec -> 15sec)
Fixes: 15595/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINK_fuzzer-5689153263501312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-05 16:58:40 +02:00
Stefan Schoenefeld
ad97be9fa3 avcodec/h263dec: enable nvdec hwaccel
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2019-08-04 16:12:02 +02:00
Stefan Schoenefeld
39a5c0ac06 avcodec/h263dec: fix hwaccel decoding
Recently we encountered an issue when decoding a h.263 file:

FFmpeg will freeze when decoding h.263 video with NVDEC. Turns out this is not directly related to NVDEC but is a problem that shows with several other HW decoders like VDPAU, though the exact kind of error is different (either error messages or freezing[1]). The root cause is that ff_thread_finish_setup() is called twice per frame from ff_h263_decode_frame(). This is not supported by ff_thread_finish_setup() and specifically checked for and warned against in the functions code. The issue is also specific to hw accelerated decoding only as the second call to ff_thread_finish_setup() is only issued when hw acceleration is on. The fix is simple: add a check that the first call is only send when hw acceleration is off, and the second call only when hw acceleration is on (see attached patch). This works fine as far as I was able to test with vdpau and nvdec/nvcuvid hw decoding. The patch also adds NVDEC to the hw config list if available.

I also noticed a secondary issue when browsing through the code which is that, according to documentation, ff_thread_finish_setup() should only be called if the codec implements update_thread_context(), which h263dec does not. The patch does not address this and I'm not sure any action needs to be taken here at all.

[1] This is depending on whether or not the hw decoder sets the  HWACCEL_CAPS_ASYNC_SAFE flag

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2019-08-04 16:11:53 +02:00
Michael Niedermayer
9af8ce754b avcodec/hnm4video: Forward errors of decode_interframe_v4()
Fixes: Timeout (108sec -> 160ms)
Fixes: 15570/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HNM4_VIDEO_fuzzer-5085482213441536

Reviewed-by: Tomas Härdin <tjoppen@acc.umu.se>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-03 20:32:20 +02:00
Michael Niedermayer
a3adc3b6a0 avcodec/interplayvideo: Avoid ff_get_buffer() during init
This is unneeded for interplay video

Fixes: memleak
Fixes: 15562/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INTERPLAY_VIDEO_fuzzer-5162268645392384

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-03 17:44:23 +02:00
Michael Niedermayer
5dc94924d0 avcodec/clearvideo: fix invalid shift in tile size check
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 15631/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CLEARVIDEO_fuzzer-5690110605000704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-03 17:44:23 +02:00
Michael Niedermayer
db6ed2b828 avcodec/vp8: do vp7_fade_frame() later
Fixes: Timeout (100sec -> 5sec)
Fixes: 15073/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5649257362620416

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-03 17:10:24 +02:00
Michael Niedermayer
58c7f419ce avcodec/vp3: Check for end of input in vp4_unpack_vlcs()
Fixes: Timeout (too long -> 1sec)
Fixes: 15232/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP3_fuzzer-5769583086010368

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-03 12:05:37 +02:00
Michael Niedermayer
b4bf7226af avcodec/vp3: Check that theora is theora
Theora is forced to be non zero if it is zero and a sample
is asked for, as suggested by reimar

Fixes: Timeout (2min -> 600ms)
Fixes: 15366/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THEORA_fuzzer-5737849938247680

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-02 21:13:41 +02:00
Shiyou Yin
11f99a9a45 avutil/mips: Avoid instruction exception caused by gssqc1/gslqc1.
Ensure the address accesed by gssqc1/gslqc1 are 16-byte aligned.
2019-08-02 19:01:51 +02:00
Paul B Mahol
9a2dbfde2e avcodec/msrle: remove unused items 2019-08-02 16:44:34 +01:00
Michael Niedermayer
6dfda35dd2 avcodec/vc1_pred: Fix invalid shift in scaleforsame()
Fixes: left shift of negative value -1
Fixes: 15531/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5759556258365440

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-31 20:35:07 +02:00
Michael Niedermayer
f31ed8f3b0 avcodec/vc1_block: Fix integer overflow in ff_vc1_pred_dc()
Fixes: signed integer overflow: 32796 * 65536 cannot be represented in type 'int'
Fixes: 15430/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5735424087031808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-31 20:35:07 +02:00
Michael Niedermayer
9a353ea876 avcodec/truemotion2: Fix several integer overflows in tm2_motion_block()
Fixes: 15524/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5173148372172800
Fixes: signed integer overflow: 13701388 - -2134868270 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-31 20:35:07 +02:00
Michael Niedermayer
1aad8937f7 avcodec/apedec: Make coeffsA/B uint32_t, this avoids several cases of undefined behavior
Changing the type to an unsigned one to avoid many casts was suggested
This may be inadequate for fixing the UB on ILP64

Fixes: signed integer overflow: -1418162611 * 383 cannot be represented in type 'int'
Fixes: 15547/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5691384901664768

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-31 20:35:07 +02:00
Michael Niedermayer
bf778af149 avcodec/apedec: make left/right unsigned to avoid undefined behavior
Fixes: signed integer overflow: 755176387 + 1515360583 cannot be represented in type 'int'
Fixes: 15506/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5706859232624640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-31 20:35:07 +02:00
Michael Niedermayer
267eb2ab7f avcodec/apedec: Fix multiple integer overflows and undefined behaviorin filter_3800()
Fixes: left shift of negative value -4
Fixes: signed integer overflow: -15091694 * 167 cannot be represented in type 'int'
Fixes: signed integer overflow: 1898547155 + 453967445 cannot be represented in type 'int'
Fixes: 15258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5759095564402688
Fixes: signed integer overflow: 962196438 * 31 cannot be represented in type 'int'
Fixes: 15364/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718799845687296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-07-31 20:35:07 +02:00