Commit graph

38443 commits

Author SHA1 Message Date
Michael Niedermayer
136ce8baa4 avcodec/ac3dec_fixed: Fix runtime error: left shift of 419 by 23 places cannot be represented in type 'int'
Fixes: 1352/clusterfuzz-testcase-minimized-5757565017260032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 13:04:46 +02:00
Michael Niedermayer
98256595fa avcodec/tiff: Clear deinvert_buf_size on deallocation
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 00:25:09 +02:00
Michael Niedermayer
9221445fa0 avcodec/tiff: Use av_fast_padded_malloc() in tiff_unpack_fax()
Fixes: Timeout
Fixes: 1213/clusterfuzz-testcase-minimized-6022987469815808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 23:58:46 +02:00
Michael Niedermayer
361e0310d9 avcodec/mlpdec: Check quant_step_size against huff_lsbs
This reorders the operations so as to avoid computations with the above arguments
before they have been initialized.
Fixes part of 1708/clusterfuzz-testcase-minimized-5035111957397504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 23:30:31 +02:00
James Almer
be3809a521 x86/aacpsdsp: optimize ff_ps_stereo_interpolate_sse3
Move the unpacking outside of the loop. 5% to 10% faster.

Suggested-by: ubitux
Signed-off-by: James Almer <jamrial@gmail.com>
2017-06-03 12:39:43 -03:00
Michael Niedermayer
9faf098163 avcodec/aacps: Fix runtime error: left shift of 1073741824 by 1 places cannot be represented in type 'INTFLOAT' (aka 'int')
Fixes: 2005/clusterfuzz-testcase-minimized-5744226438479872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 00:09:58 +02:00
Michael Niedermayer
14b6adfd46 avcodec/snowdec: Fix runtime error: signed integer overflow: 1404 * 8388608 cannot be represented in type 'int'
Fixes: 2004/clusterfuzz-testcase-minimized-5533262866808832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 00:09:58 +02:00
Ganapathy Kasi
43c417ac1a avcodec/nvenc: fix hw accelerated transcode with bframes
hw accelerated transcode (h264_cuvid -> h264_nvenc with -hwaccel cuvid) was
broken after the filtergraph initialization was changed to intialize decoder
first followed by encoder (commit af1761f7b5).
During initialzing encoder with bframes, local buffers are allocated
internally in encoder which fails since no cuda context is available. Now
pushing the correct cuda context before encoder initialization fixes the issue.
Also adding push/pop cuda ctx during create/destroy/map/unmap resources and
destroy encoder session.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2017-06-02 21:32:35 +02:00
James Almer
b5a0971ff0 x86/aacps: add ff_ps_stereo_interpolate_ipdopd_sse3()
About 2x faster than the c version.

Signed-off-by: James Almer <jamrial@gmail.com>
2017-06-02 11:06:24 -03:00
Michael Niedermayer
adb4854aac avcodec/asvdec: Use rounded up dimenensions in input size check
Fixes: Timeout
Fixes: 2001/clusterfuzz-testcase-minimized-6187599389523968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-01 22:20:16 +02:00
Michael Niedermayer
8b3e580b7f avcodec/wavpack: Fix runtime error: shift exponent 32 is too large for 32-bit type 'int'
Fixes: 1967/clusterfuzz-testcase-minimized-5757031199801344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-01 22:20:16 +02:00
Michael Niedermayer
cd6f319a74 avcodec/cfhd: Fix runtime error: signed integer overflow: 65280 * 65288 cannot be represented in type 'int'
Fixes: 1925/clusterfuzz-testcase-minimized-5564569688735744

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-01 22:20:16 +02:00
Timo Rothenpieler
ff3084606c avcodec/cuvid: make capability check optional 2017-06-01 12:39:06 +02:00
Timo Rothenpieler
cb3358b68f avcodec/nvenc: print minimum driver version on error 2017-06-01 11:55:25 +02:00
Srinath K R
d8da329cc3 avcodec/nvenc: Add default value for AVCodecContext::refs
AVCodecContext::refs is used to control the DPB size to be used by the
encoder. The default value for AVCodecContext::refs as set in
libavcodec/options_table.h is 1.

This patch sets AVCodecContext::refs to 0 for h264_nvenc and hevc_nvenc in
order to let the driver take the decision of the correct DPB size to use in
all cases.

Signed-off-by: Srinath K R <skr@nvidia.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2017-06-01 11:28:30 +02:00
Michael Niedermayer
a47273c803 avcodec/wavpack: Fix runtime error: signed integer overflow: 2013265955 - -134217694 cannot be represented in type 'int'
Fixes: 1922/clusterfuzz-testcase-minimized-5561194112876544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-31 22:53:51 +02:00
Michael Niedermayer
e47057e932 avcodec/cinepak: Check input packet size before frame reallocation
Reduces time spend decoding 1917/clusterfuzz-testcase-minimized-5023221273329664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-31 22:24:16 +02:00
Michael Niedermayer
6726328f79 avcodec/hevc_ps: Fix runtime error: signed integer overflow: 2147483628 + 256 cannot be represented in type 'int'
Fixes: 1909/clusterfuzz-testcase-minimized-6732072662073344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-31 22:05:32 +02:00
Michael Niedermayer
08cb69e870 avcodec/ra144: Fixes runtime error: signed integer overflow: 7160 * 327138 cannot be represented in type 'int'
Fixes: 1908/clusterfuzz-testcase-minimized-5392712477966336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-31 16:08:46 +02:00
Michael Niedermayer
a1c0d1d906 avcodec/pnm: Use ff_set_dimensions()
Fixes: OOM
Fixes: 1906/clusterfuzz-testcase-minimized-4599315114754048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-31 16:08:46 +02:00
Michael Niedermayer
58f8cd4ac5 avcodec/cavsdec: Fix runtime error: signed integer overflow: 59 + 2147483600 cannot be represented in type 'int'
Fixes: 1903/clusterfuzz-testcase-minimized-5359318167715840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-31 16:08:46 +02:00
wm4
0160230382 videotoolbox: log errors
With the new decode API, you can't handle errors directly in the API
user - you only know that the hwaccel did not initialize at all.

Add some approximate logging.
2017-05-31 12:06:51 +02:00
James Darnley
0dea0114fb avcodec/x86/idctdsp_init: reindent 2017-05-30 13:20:44 +02:00
James Darnley
8e89f6fd37 avcodec/x86: move simple_idct to external assembly 2017-05-30 13:20:42 +02:00
Michael Niedermayer
87bddba43b avcodec/acelp_pitch_delay: Fix runtime error: value 4.83233e+39 is outside the range of representable values of type 'float'
Fixes: 1902/clusterfuzz-testcase-minimized-4762451407011840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-30 04:45:27 +02:00
Michael Niedermayer
4020b009d1 avcodec/wavpack: Check float_shift
Fixes: runtime error: shift exponent 40 is too large for 32-bit type 'unsigned int'
Fixes: 1898/clusterfuzz-testcase-minimized-5970744880136192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-30 04:45:27 +02:00
Michael Niedermayer
d90c5bf105 avcodec/wavpack: Fix runtime error: signed integer overflow: 24 * -2147483648 cannot be represented in type 'int'
Fixes: 1894/clusterfuzz-testcase-minimized-4716739789062144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-30 04:45:27 +02:00
Rostislav Pehlivanov
220b24c7c9 lavc: remove libschroedinger encoding and decoding wrappers
The library has stopped being developed and Debian has removed it
from its repositories citing security issues.
The native Dirac decoder supports everything the library has and basic
encoding support is still provided via the native vc2 (Dirac Pro, intra
only version of Dirac) encoder. Hence, there's no reason to still support
linking to the library and potentially leading users into security issues.
2017-05-29 20:15:58 +01:00
Michael Niedermayer
e091b9b3c7 avcodec/ansi: Fix frame memleak
Fixes: 1892/clusterfuzz-testcase-minimized-4519341733183488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-29 14:08:07 +02:00
Michael Niedermayer
c49fa2a514 avcodec/dds: Fix runtime error: left shift of 145 by 24 places cannot be represented in type 'int'
Fixes: 1891/clusterfuzz-testcase-minimized-6274417925554176

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-29 13:51:42 +02:00
Michael Niedermayer
f3da6fbff8 avcodec/jpeg2000dec: Use ff_set_dimensions()
Fixes: OOM
Fixes: 1890/clusterfuzz-testcase-minimized-6329019509243904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-29 13:46:09 +02:00
Michael Niedermayer
f6ba58d193 avcodec/aacsbr: Fix libavcodec/aacsbr.c:257:59: runtime error: division by zero
Fixes: 1882/clusterfuzz-testcase-minimized-5539735650959360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-29 02:45:17 +02:00
Michael Niedermayer
c901627918 avcodec/truemotion2: Fix passing null pointer to memset()
Fixes part of: 1888/clusterfuzz-testcase-minimized-5237704826552320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 21:56:02 +02:00
Michael Niedermayer
c9e884f3d9 avcodec/truemotion2: Fix runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes part of: 1888/clusterfuzz-testcase-minimized-5237704826552320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 21:55:06 +02:00
Michael Niedermayer
7c845450d2 avcodec/ra144: Fix runtime error: signed integer overflow: -2449 * 1398101 cannot be represented in type 'int'
Fixes: 1885/clusterfuzz-testcase-minimized-5336328549957632

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 21:55:02 +02:00
Michael Niedermayer
4c472c5252 avcodec/ra144: Fix runtime error: signed integer overflow: 11184810 * 404 cannot be represented in type 'int'
Fixes: 1884/clusterfuzz-testcase-minimized-4637425835966464

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 21:38:55 +02:00
Michael Niedermayer
872bac8159 avcodec/aac_defines: Add missing () to AAC_HALF_SUM() macro
Fixes: runtime error: shift exponent 1073741848 is too large for 32-bit type 'INTFLOAT' (aka 'int')
Fixes: 1880/clusterfuzz-testcase-minimized-4900645322620928

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 21:20:28 +02:00
Michael Niedermayer
67020711b7 avcodec/webp: Fixes null pointer dereference
Fixes: 1470/clusterfuzz-testcase-minimized-5404421666111488
Fixes: 1472/clusterfuzz-testcase-minimized-5677426430443520
Fixes: 1875/clusterfuzz-testcase-minimized-5536474562822144

Approved-by: BBB

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 21:20:09 +02:00
Michael Niedermayer
6b9cb5d26a avcodec/aacdec_fixed: Fix runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 1878/clusterfuzz-testcase-minimized-6441918630199296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 18:27:43 +02:00
Michael Niedermayer
67b30decf7 avcodec/ylc: Check count in build_vlc()
Fixes: runtime error: signed integer overflow: 211633430 + 2147483647 cannot be represented in type 'int'
Fixes: 1874/clusterfuzz-testcase-minimized-5037763613163520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 17:21:41 +02:00
Michael Niedermayer
b9c032ebc0 avcodec/snow: Fix runtime error: signed integer overflow: 1086573993 + 1086573994 cannot be represented in type 'int'
Fixes: 1871/clusterfuzz-testcase-minimized-5719950331215872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 17:13:07 +02:00
Michael Niedermayer
781f88bb26 avcodec/jpeg2000: Fix runtime error: signed integer overflow: 4185 + 2147483394 cannot be represented in type 'int'
Fixes: 1870/clusterfuzz-testcase-minimized-4686788029317120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 14:01:12 +02:00
Michael Niedermayer
9c1812491f avcodec/jpeg2000dec: Check tile offsets more completely
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 13:52:13 +02:00
Michael Niedermayer
d8030c14bd avcodec/sheervideo: Check input buffer size before allocating and decoding
Fixes: Timeout
Fixes: 1858/clusterfuzz-testcase-minimized-6450473802399744

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 13:32:12 +02:00
Michael Niedermayer
c51357d206 avcodec/wavpack: Fix runtime error: signed integer overflow: -1386217472 * 4 cannot be represented in type 'int'
Fixes: 1853/clusterfuzz-testcase-minimized-5471155626442752

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 12:57:14 +02:00
Michael Niedermayer
6c3a63fc3d avcodec/aacdec_fixed: Fix multiple runtime error: shift exponent 127 is too large for 32-bit type 'int'
Fixes: 1851/clusterfuzz-testcase-minimized-5692607495667712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 03:34:43 +02:00
Michael Niedermayer
7f50c25124 avcodec/wnv1: More strict buffer size check
This requires at least 25% of a picture to allocate and decode it

Fixes: Timeout
Fixes: 1845/clusterfuzz-testcase-minimized-5075974343360512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 03:20:09 +02:00
Michael Niedermayer
ca6776a993 avcodec/libfdk-aacdec: Correct buffer_size parameter
the timeDataSize argument to aacDecoder_DecodeFrame() seems undocumented and until
2016 04 (203e3f28fbebec7011342017fafc2a0bda0ce530) unused.
after that commit libfdk-aacdec interprets it as size in sample units and memsets that on error.
FFmpeg as well as others (like GStreamer) did interpret it as size in bytes

Fixes: 1442/clusterfuzz-testcase-minimized-4540199973421056 (This requires recent libfdk to reproduce)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 03:08:33 +02:00
Michael Niedermayer
7c36ee216f avcodec/sbrdsp_template: Fix: runtime error: signed integer overflow: 849815297 + 1315389781 cannot be represented in type 'int'
Fixes: 1770/clusterfuzz-testcase-minimized-5285511235108864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 03:07:45 +02:00
Michael Niedermayer
fe8c9420dd avcodec/aacps: Check border_position to be monotone
Fixes: runtime error: left shift of negative value -67108864
Fixes: 1738/clusterfuzz-testcase-minimized-6734814327603200

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-28 03:07:02 +02:00