Commit graph

38480 commits

Author SHA1 Message Date
Michael Niedermayer
6d499ecef9 avcodec/aacdec_fixed: Fix runtime error: left shift of negative value -1297616
Fixes: 2195/clusterfuzz-testcase-minimized-4736721533009920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-11 15:22:07 +02:00
Paul B Mahol
9a949cdf8f avcodec/gdv: fix compiler warnings
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2017-06-11 10:26:18 +02:00
Paul B Mahol
d1c08027d8 avcodec: add Gremlin Digital Video decoder
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2017-06-11 10:00:39 +02:00
Paul B Mahol
29bdcf588f avcodec: add Gremlin DPCM decoder
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2017-06-11 10:00:16 +02:00
Michael Niedermayer
22a25ab389 avcodec/tiff: Fix leak of geotags[].val
Fixes: 2176/clusterfuzz-testcase-minimized-5908197216878592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-11 01:06:08 +02:00
Michael Niedermayer
1197c04896 avcodec/fft_template: Fix multiple runtime error: signed integer overflow: -1943918714 - 1935113003 cannot be represented in type 'int'
Fixes: 1735/clusterfuzz-testcase-minimized-5350472347025408

This uses unsigned instead of SUINT because the later was rejected by multiple developers
See: [FFmpeg-devel] [PATCH] avcodec/fft_template: Fix multiple runtime error: signed integer overflow: -1943918714 - 1935113003 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-11 00:56:33 +02:00
Michael Niedermayer
71da0a5c97 avcodec/ra144: Fix runtime error: signed integer overflow: -2200 * 1033073 cannot be represented in type 'int'
Fixes: 2175/clusterfuzz-testcase-minimized-5809657849315328

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-11 00:47:29 +02:00
Michael Niedermayer
90e8317b3b avcodec/flicvideo: Fix runtime error: signed integer overflow: 4864 * 459296 cannot be represented in type 'int'
Fixes: 2174/clusterfuzz-testcase-minimized-5739234533048320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-10 19:46:04 +02:00
Michael Niedermayer
54aaadf648 avcodec/cfhd: Check band parameters before storing them
Fixes out of array read
Fixes: 2169/clusterfuzz-testcase-minimized-5688641642823680

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-10 18:46:27 +02:00
Michael Niedermayer
09096fb687 avcodec/h264_parse: Check picture structure when initializing weight table
Fixes: runtime error: index 49 out of bounds for type 'int [48][2][2]'
Fixes: 2159/clusterfuzz-testcase-minimized-5267945972301824

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-10 00:48:23 +02:00
Michael Niedermayer
e1b0044c23 avcodec/dvbsubdec: Check pixel buffer size constraint from ETSI EN 300 743 V1.3.1
Fixes: OOM
Fixes: 2143/clusterfuzz-testcase-minimized-5482288060039168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-10 00:48:07 +02:00
Michael Niedermayer
4bcde26172 avcodec/dvbsubdec: Use av_image_check_size2()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-10 00:48:07 +02:00
Aman Gupta
a32a6b4201 lavc: add mpeg2 mediacodec decoder
Android TV and FireOS hardware supports mpeg2 hardware decoding via
MediaCodec.

Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
2017-06-08 19:16:09 +02:00
Michael Niedermayer
a3b5b60bdf avcodec/indeo4: Check remaining data in Pic hdr extension parsing code
Fixes: Timeout
Fixes: 2115/clusterfuzz-testcase-minimized-6594111748440064

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-08 17:37:16 +02:00
Michael Niedermayer
4e3ab1a5c1 avcodec/ac3dec_fixed: Fix multiple runtime error: signed integer overflow: -39271008 * 59 cannot be represented in type 'int'
Fixes: 2113/clusterfuzz-testcase-minimized-6510704959946752

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-08 17:37:16 +02:00
James Almer
497a4b554c x86/aacpsdsp: fix output of ff_ps_stereo_interpolate_ipdopd_sse3
The fate-aac-al_sbr_ps_04_ur test did not detect this mistake.
2017-06-07 13:53:51 -03:00
Tyler Jones
34c5200560 vorbisenc: Fix memory leak on errors
Switches temporary samples for processing to be stored in the encoder's
context, avoids memory leaks if any errors occur while encoding a frame.

Fixes CID1412026

Signed-off-by: Tyler Jones <tdjones879@gmail.com>
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2017-06-06 17:57:49 +01:00
Michael Niedermayer
18bca25adb avcodec/mpeg4videodec: Fix runtime error: signed integer overflow: 53098 * 40448 cannot be represented in type 'int'
Fixes: 2106/clusterfuzz-testcase-minimized-6136503639998464

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-06 16:54:50 +02:00
Michael Niedermayer
c4360559ee avcodec/pafvideo: Fix assertion failure
Fixes: 2100/clusterfuzz-testcase-minimized-4522961547558912

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-06 16:54:50 +02:00
Michael Niedermayer
e4efd41b83 avcodec/takdec: Fix multiple runtime error: signed integer overflow: 637072 * 4096 cannot be represented in type 'int'
Fixes: 2079/clusterfuzz-testcase-minimized-5345861779324928

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-06 16:54:50 +02:00
Ilia Valiakhmetov
73d9a9a6af libavcodec/vp9: ipred_dl_32x32_16 avx2 implementation
vp9_diag_downleft_32x32_8bpp_c: 580.2
vp9_diag_downleft_32x32_8bpp_sse2: 75.6
vp9_diag_downleft_32x32_8bpp_ssse3: 73.7
vp9_diag_downleft_32x32_8bpp_avx: 72.7
vp9_diag_downleft_32x32_10bpp_c: 1101.2
vp9_diag_downleft_32x32_10bpp_sse2: 145.4
vp9_diag_downleft_32x32_10bpp_ssse3: 137.5
vp9_diag_downleft_32x32_10bpp_avx: 134.8
vp9_diag_downleft_32x32_10bpp_avx2: 94.0
vp9_diag_downleft_32x32_12bpp_c: 1108.5
vp9_diag_downleft_32x32_12bpp_sse2: 145.5
vp9_diag_downleft_32x32_12bpp_ssse3: 137.3
vp9_diag_downleft_32x32_12bpp_avx: 135.2
vp9_diag_downleft_32x32_12bpp_avx2: 94.0

~30% faster than avx implementation

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2017-06-06 08:05:03 -04:00
Stefano Sabatini
dc70ea8c19 lavc/mpegvideo_enc: allow low_delay for non MPEG2 codecs depending on strict_std_compliance
Forcing low_delay can be useful, even if not officially supported.
2017-06-06 09:56:49 +02:00
Michael Niedermayer
4705edbbb9 avcodec/mjpegdec: Check that reference frame matches the current frame
Fixes: out of array read
Fixes: 2097/clusterfuzz-testcase-minimized-5036861833609216

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-05 22:43:23 +02:00
Michael Niedermayer
d7cbeab4c1 avcodec/tiff: Avoid loosing allocated geotag values
Fixes memleak
Fixes: 2076/clusterfuzz-testcase-minimized-6542640243802112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-05 22:43:23 +02:00
Michael Niedermayer
1e6ee86d92 avcodec/cavs: Fix runtime error: signed integer overflow: -12648062 * 256 cannot be represented in type 'int'
Fixes: 2067/clusterfuzz-testcase-minimized-5578430902960128

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-05 22:43:23 +02:00
Rostislav Pehlivanov
7fc1be9a01 vorbisenc: signal samples to skip
The encoder never actually signalled how many samples to skip.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2017-06-05 16:51:08 +01:00
Tyler Jones
29c13fed68 avcodec/vorbisenc: Use a bufqueue in encoding with smaller lengths
Switching the vorbis encoder to use a buffer queue for input frames allows
saving lookahead samples more easily and safely for psychoacoustic systems,
requiring less pointer arithmetic in the case of transient windows.
2017-06-05 16:35:07 +01:00
Tyler Jones
25260b5161 avcodec/vorbisenc: Include bufqueue and afqueue 2017-06-05 16:35:05 +01:00
Tyler Jones
79941602a3 avcodec/vorbisenc: Use fdsp for applying windows
Using fdsp improves readability and allows using architecture-specific
optimizations.

Signed-off-by: Tyler Jones <tdjones879@gmail.com>
2017-06-05 16:35:03 +01:00
Tyler Jones
610864dc36 avcodec/vorbisenc: Include fdsp
Signed-off-by: Tyler Jones <tdjones879@gmail.com>
2017-06-05 16:35:01 +01:00
James Almer
933dd62288 x86/aacpsdsp: optimize ff_ps_mul_pair_single_sse
~2% faster.
2017-06-04 23:29:56 -03:00
Michael Niedermayer
46b865ea9f avcodec/qdrw: Fix null pointer dereference
The RGB555 PACKBITSRGN case tries to read a palette, if such
palette is actually stored then it accesses a null pointer.
All 16bit samples i could find use DIRECTBITSRGN.

Fixes: 2065/clusterfuzz-testcase-minimized-6298930457346048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 23:41:35 +02:00
Michael Niedermayer
b315a3cf42 avcodec/sbrdsp_fixed: Fix assertion failure in sbr_sum_square_c()
This also increases the range of input values supported as well as
decreasing the operation dependencies in the main loop, improving
speed on modern CPUs.

Fixes part of: 2045/clusterfuzz-testcase-minimized-6751255865065472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 23:41:35 +02:00
Michael Niedermayer
e2bbb95d58 avcodec/wavpack: Fix runtime error: signed integer overflow: 2081021665 - -130689706 cannot be represented in type 'int'
Fixes: 2038/clusterfuzz-testcase-minimized-4521466148159488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 17:51:02 +02:00
Michael Niedermayer
29808fff33 avcodec/hevc_ps: Fix runtime error: index 32 out of bounds for type 'uint8_t [32]'
Fixes: 2010/clusterfuzz-testcase-minimized-6209288450080768

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 17:51:02 +02:00
Michael Niedermayer
eb50492270 avcodec/dxv: Check remaining bytes in dxv_decompress_raw()
Fixes: Timeout
Fixes: 2006/clusterfuzz-testcase-minimized-5766515037044736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 17:51:02 +02:00
Michael Niedermayer
faa5a2181d avcodec/pafvideo: Check packet size and frame code before ff_reget_buffer()
Fixes 1745/clusterfuzz-testcase-minimized-6160693365571584
Fixes: Timeout

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 13:40:54 +02:00
Michael Niedermayer
136ce8baa4 avcodec/ac3dec_fixed: Fix runtime error: left shift of 419 by 23 places cannot be represented in type 'int'
Fixes: 1352/clusterfuzz-testcase-minimized-5757565017260032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 13:04:46 +02:00
Michael Niedermayer
98256595fa avcodec/tiff: Clear deinvert_buf_size on deallocation
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-04 00:25:09 +02:00
Michael Niedermayer
9221445fa0 avcodec/tiff: Use av_fast_padded_malloc() in tiff_unpack_fax()
Fixes: Timeout
Fixes: 1213/clusterfuzz-testcase-minimized-6022987469815808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 23:58:46 +02:00
Michael Niedermayer
361e0310d9 avcodec/mlpdec: Check quant_step_size against huff_lsbs
This reorders the operations so as to avoid computations with the above arguments
before they have been initialized.
Fixes part of 1708/clusterfuzz-testcase-minimized-5035111957397504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 23:30:31 +02:00
James Almer
be3809a521 x86/aacpsdsp: optimize ff_ps_stereo_interpolate_sse3
Move the unpacking outside of the loop. 5% to 10% faster.

Suggested-by: ubitux
Signed-off-by: James Almer <jamrial@gmail.com>
2017-06-03 12:39:43 -03:00
Michael Niedermayer
9faf098163 avcodec/aacps: Fix runtime error: left shift of 1073741824 by 1 places cannot be represented in type 'INTFLOAT' (aka 'int')
Fixes: 2005/clusterfuzz-testcase-minimized-5744226438479872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 00:09:58 +02:00
Michael Niedermayer
14b6adfd46 avcodec/snowdec: Fix runtime error: signed integer overflow: 1404 * 8388608 cannot be represented in type 'int'
Fixes: 2004/clusterfuzz-testcase-minimized-5533262866808832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-03 00:09:58 +02:00
Ganapathy Kasi
43c417ac1a avcodec/nvenc: fix hw accelerated transcode with bframes
hw accelerated transcode (h264_cuvid -> h264_nvenc with -hwaccel cuvid) was
broken after the filtergraph initialization was changed to intialize decoder
first followed by encoder (commit af1761f7b5).
During initialzing encoder with bframes, local buffers are allocated
internally in encoder which fails since no cuda context is available. Now
pushing the correct cuda context before encoder initialization fixes the issue.
Also adding push/pop cuda ctx during create/destroy/map/unmap resources and
destroy encoder session.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2017-06-02 21:32:35 +02:00
James Almer
b5a0971ff0 x86/aacps: add ff_ps_stereo_interpolate_ipdopd_sse3()
About 2x faster than the c version.

Signed-off-by: James Almer <jamrial@gmail.com>
2017-06-02 11:06:24 -03:00
Michael Niedermayer
adb4854aac avcodec/asvdec: Use rounded up dimenensions in input size check
Fixes: Timeout
Fixes: 2001/clusterfuzz-testcase-minimized-6187599389523968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-01 22:20:16 +02:00
Michael Niedermayer
8b3e580b7f avcodec/wavpack: Fix runtime error: shift exponent 32 is too large for 32-bit type 'int'
Fixes: 1967/clusterfuzz-testcase-minimized-5757031199801344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-01 22:20:16 +02:00
Michael Niedermayer
cd6f319a74 avcodec/cfhd: Fix runtime error: signed integer overflow: 65280 * 65288 cannot be represented in type 'int'
Fixes: 1925/clusterfuzz-testcase-minimized-5564569688735744

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-01 22:20:16 +02:00
Timo Rothenpieler
ff3084606c avcodec/cuvid: make capability check optional 2017-06-01 12:39:06 +02:00