Commit graph

40528 commits

Author SHA1 Message Date
Michael Niedermayer
2e6ea39386 avcodec/vp8_parser: Remove redundant output initialization
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-15 19:42:25 +02:00
Michael Niedermayer
9e6c843776 avcodec/dvdsub_parser: Init output buf/size
No testcase

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-15 19:42:25 +02:00
Michael Niedermayer
8d21ab4d12 avcodec/lagarith: Check that the range coded data stream is consistent when the probabilities indicate no data could have been coded.
Fixes: Timeout
Fixes: 8638/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LAGARITH_fuzzer-5132046098759680
Fixes: 8943/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LAGARITH_fuzzer-4883030219948032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-15 19:42:25 +02:00
Michael Niedermayer
7334985ffa libavcodec/cfhd: Fix signed overflow in shift
Fixes: 8695/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-4906172426485760

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-15 19:42:25 +02:00
Michael Niedermayer
181435a4de avcodec/dirac_dwt_template: Fix signedness regression in interleave()
Found-by: <jdarnley>
Tested-by: James Darnley <james.darnley@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-14 18:02:26 +02:00
Shiyou Yin
c0b42987a2 avcodec/mips: fix conflicting types error of ff_vc1_h_s_overlap_mmi.
In commit 975a1a8,function ff_vc1_h_s_overlap_mmi was refactored,
but the declaration in libavcodec/mips/vc1dsp_mips.h was unchanged.

Change-Id: I90beae683511622a0cc1130ab1660ac8669ec3ef
Signed-off-by: Shiyou Yin <yinshiyou-hf@loongson.cn>
Reviewed-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-14 18:02:26 +02:00
Marton Balint
3e0f3be2b0 avcodec/internal: increase FF_SANE_NB_CHANNELS to 256
This was reduced from 128 in libav commit
192f1984b1, but since we support unknown channel
layouts, we can increase this limit.

Fixes ticket #6332.

Signed-off-by: Marton Balint <cus@passwd.hu>
2018-07-13 22:24:33 +02:00
Marton Balint
5982078e8d avcodec/utils: report insane channel count errors
More than 64 is not *that* insane, so let's report the error at least.

Signed-off-by: Marton Balint <cus@passwd.hu>
2018-07-13 22:24:33 +02:00
Carl Eugen Hoyos
0576ef466d lavc/aarch64/h264dsp_init_aarch64: Fix weight function prototypes.
Fixes the following warnings:
libavcodec/aarch64/h264dsp_init_aarch64.c: In function ‘ff_h264dsp_init_aarch64’:
libavcodec/aarch64/h264dsp_init_aarch64.c:84:38: warning: assignment from incompatible pointer type [enabled by default]
         c->weight_h264_pixels_tab[0] = ff_weight_h264_pixels_16_neon;
                                      ^
libavcodec/aarch64/h264dsp_init_aarch64.c:85:38: warning: assignment from incompatible pointer type [enabled by default]
         c->weight_h264_pixels_tab[1] = ff_weight_h264_pixels_8_neon;
                                      ^
libavcodec/aarch64/h264dsp_init_aarch64.c:86:38: warning: assignment from incompatible pointer type [enabled by default]
         c->weight_h264_pixels_tab[2] = ff_weight_h264_pixels_4_neon;
                                      ^
libavcodec/aarch64/h264dsp_init_aarch64.c:88:40: warning: assignment from incompatible pointer type [enabled by default]
         c->biweight_h264_pixels_tab[0] = ff_biweight_h264_pixels_16_neon;
                                        ^
libavcodec/aarch64/h264dsp_init_aarch64.c:89:40: warning: assignment from incompatible pointer type [enabled by default]
         c->biweight_h264_pixels_tab[1] = ff_biweight_h264_pixels_8_neon;
                                        ^
libavcodec/aarch64/h264dsp_init_aarch64.c:90:40: warning: assignment from incompatible pointer type [enabled by default]
         c->biweight_h264_pixels_tab[2] = ff_biweight_h264_pixels_4_neon;
                                        ^
2018-07-13 21:28:04 +02:00
Jun Zhao
d746a2e3cf lavc/hevc_ps: use skip_bits instead of get_bits when skip bits.
use skip_bits when want to skip some bits.

Signed-off-by: Jun Zhao <mypopydev@gmail.com>
2018-07-13 20:18:40 +08:00
Jun Zhao
896854b233 lavc/hevc_ps: Refine sps_range_extension parse.
Signed-off-by: Jun Zhao <mypopydev@gmail.com>
2018-07-13 20:15:15 +08:00
Karsten Otto
ce372bc278 libavcodec/mpegaudiodecheader.h: fix version check pattern
This fixes the check for the reserved MPEG audio version ID,
used to detect an invalid frame header.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-13 02:19:50 +02:00
Carl Eugen Hoyos
5545a6df87 lavc/h263dec: Remove a variable declaration that can lead to a warning. 2018-07-13 01:01:06 +02:00
Zhong Li
3c26ce4644 lavc/qsv: handle MFX_FRAMETYPE_UNKNOWN case
Signed-off-by: Zhong Li <zhong.li@intel.com>
2018-07-12 15:29:55 +08:00
Karsten Otto
3bf39f2aef libavcodec/mpegaudiodecheader.h : detect reserved mpeg id
Check the MPEG version ID for the reserved bit pattern 01, and abort the
header check in that case. This reduces the chance of misinterpreting
arbitrary data as a valid header, and prevents resulting audio artifacts.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-09 03:25:53 +02:00
Michael Niedermayer
ee1e3ca5eb avcodec/mjpegdec: Check for odd progressive RGB
Fixes: out of array access
Fixes: 9225/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEGLS_fuzzer-5684770334834688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-08 00:55:49 +02:00
Michael Niedermayer
115b7c96e9 libavcodec/vp8: Do not compute line pointers per pixel in fade()
72->60 seconds
Testcase: 8680/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5861504418054144

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-07 12:09:34 +02:00
Michael Niedermayer
284dde24da avcodec/vp8_parser: Do not leave data/size uninitialized
This is identical to what the VP9 parser does

Fixes: 9215/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBVPX_VP8_fuzzer-5768227253649408
Fixes: out of memory access

This may also fix oss fuzz issue 9212

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-07 12:09:34 +02:00
Michael Niedermayer
5aba5b89d0 avcodec/mpeg4videodec: Check for bitstream end in read_quant_matrix_ext()
Fixes: out of array read
Fixes: asff-crash-0e53d0dc491dfdd507530b66562812fbd4c36678

Found-by: Paul Ch <paulcher@icloud.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 23:14:24 +02:00
James Almer
a61b56624b avcodec/atrac9tab: add missing header include
Fixes make checkheaders

Signed-off-by: James Almer <jamrial@gmail.com>
2018-07-04 15:56:44 -03:00
Carl Eugen Hoyos
e25c25ebd8 lavc/atrac9tab: Add inclusion guards.
Fixes fate-source.
2018-07-04 18:54:44 +02:00
Michael Niedermayer
267ba2aa96 avcodec/indeo4: Check for end of bitstream in decode_mb_info()
Fixes: Timeout
Fixes: 8776/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO4_fuzzer-5361788798369792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 12:15:10 +02:00
Michael Niedermayer
4423085ca5 avcodec/truemotion2: Check len in tm2_read_stream()
Fixes: Timeout
Fixes: 8774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5942199639343104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
2018-07-04 12:14:00 +02:00
Michael Niedermayer
00f98d23b1 avcodec/ac3dec: Check channel_map index
Fixes: out of array read
Fixes: 8924/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EAC3_fuzzer-5851861780267008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 12:12:22 +02:00
Michael Niedermayer
bd27a9364c avcodec/mpeg4videodec: Remove use of FF_PROFILE_MPEG4_SIMPLE_STUDIO as indicator of studio profile
The profile field is changed by code inside and outside the decoder,
its not a reliable indicator of the internal codec state.
Maintaining it consistency with studio_profile is messy.
Its easier to just avoid it and use only studio_profile

Fixes: assertion failure
Fixes: ffmpeg_crash_9.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 12:11:36 +02:00
Michael Niedermayer
3b10bb8772 avcodec/shorten: Fix undefined addition in shorten_decode_frame()
Fixes: signed integer overflow: 1139785606 + 1454196085 cannot be represented in type 'int'
Fixes: 8937/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6202943597445120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Michael Niedermayer
70832333bb avcodec/shorten: Fix undefined integer overflow
Fixes: signed integer overflow: 8454144 * 256 cannot be represented in type 'int'
Fixes: 8788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-5728205041303552

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Michael Niedermayer
652d7c6348 avcodec/jpeg2000dec: Fixes invalid shifts in jpeg2000_decode_packets_po_iteration()
Fixes: shift exponent 47 is too large for 32-bit type 'int'
Fixes: 9163/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5661750182543360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Michael Niedermayer
0898a3d990 avcodec/jpeg2000dec: Check that there are enough bytes for all tiles
Fixes: OOM
Fixes: 8781/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5810709081358336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Rostislav Pehlivanov
755e618399 lavc: implement an ATRAC9 decoder
This commit implements a full ATRAC9 decoder, a simple low-delay codec
developed by Sony and used in most PSVita games, some PS3 games and some
PS4 games. Its similar to AAC in that it uses Huffman coded scalefactors
but instead of vector quantization it just Huffman codes the spectral
coefficients (in a way similar to how Opus splits band energy coding
into coarse and fine precision). It opts to write rather large Huffman
codes by packing several small coefficients into one Huffman coded
symbol, though I don't believe this increases efficiency at all.
Band extension implements SBC in a simple way, first it mirrors the
lower spectrum onto the higher frequencies and then it uses one of 5
filters to shape it. Noise substitution is implemented via 2 of them.
Unlike previous ATRAC codecs, there's no QMF, this is a standard MDCT
codec.

Based off of the reverse engineering work of Alex Barney.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2018-07-03 20:15:01 +01:00
Michael Niedermayer
d24c9e55f6 avcodec/dvdsubdec: Check for fully transparent rectangles earlier
Testcase with large transparent rectangles changes from 67 sec to 3 sec decode time

Fixes: Timeout
Fixes: 8728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVDSUB_fuzzer-5190088756559872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-02 13:55:28 +02:00
Michael Niedermayer
d08d4a8c73 avcodec/vc1_block: Fix mqaunt check for negative values
Fixes: out of array access
Fixes: ffmpeg_bof_4.avi
Fixes: ffmpeg_bof_5.avi
Fixes: ffmpeg_bof_6.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Reviewed-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-30 00:41:35 +02:00
Jerome Borsboom
efde2a70ff avcodec/vc1: fix check for missing CBPTAB
CBPTAB must be present in (non skipped) P and B pictures.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:19:10 +02:00
Jerome Borsboom
975a1a81b2 avcodec/vc1: fix overlap filter for frame interlaced pictures
The overlap filter is not correct for vertical edges in frame interlaced
I and P pictures. When filtering macroblocks with different FIELDTX values,
we have to match the lines at both sides of the vertical border. In addition,
we have to use the correct rounding values, depending on the line we are
filtering.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:18:44 +02:00
Jerome Borsboom
f92e95e9b5 avcodec/vc1: change the internal ordering of blocks within a macroblock
The overlap filter needs to cover a full macroblock vertical edge when the
FIELDTX value for two neighbouring macroblocks is not equal. By changing
the internal ordering of the blocks from row major to column major, we do
not need to reinterlace a FIELDTX coded macroblock before running the overlap
filter.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:17:23 +02:00
Jerome Borsboom
1cf6129d13 avcodec/vc1: fix condition guarding overlap filter on I picture
The overlap filter needs to run when PQUANT is 9 or higher, irrespective
of CONDOVER.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:16:51 +02:00
John Stebbins
08b1d1d812 h264_slice: Fix return of incomplete frames from decoder
When not using libavformat for demuxing, AVCodecContext.has_b_frames
gets set too late causing the recovery frame heuristic in h264_refs to
incorrectly flag an early frame as recovered.

This patch sets has_b_frames earlier to prevent improperly flagging the
frame as recovered.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Michael Niedermayer
2aa9047486 avcodec/mpeg4videodec: Check read profile before setting it
Fixes: null pointer dereference
Fixes: ffmpeg_crash_7.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Michael Niedermayer
00e8181bd9 avcodec/ac3_parser: Check init_get_bits8() for failure
Fixes: null pointer dereference
Fixes: ffmpeg_crash_6.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Michael Niedermayer
75a2db5524 avcodec/dpx: Check elements in 12bps planar path
Fixes: null pointer dereference
Fixes: 8946/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DPX_fuzzer-5078915222601728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Timo Rothenpieler
3f953379e1 avcodec/qsvenc: fix version detection on cygwin 2018-06-27 16:04:08 +02:00
Michael Niedermayer
f59c4e4391 avcodec/escape124: Fix spelling errors in comment
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-27 13:50:14 +02:00
Michael Niedermayer
71ba79702d avcodec/dvbsubdec: Compute scoretab without iterating over pixels multiple times in compute_default_clut()
Improves speed 102->2 sec
Testcase: 8655/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVBSUB_fuzzer-6277869285146624
Fixes: Timeout

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-27 13:43:39 +02:00
Michael Niedermayer
b31189881a avcodec/ra144: Fix integer overflow in ff_eval_refl()
Fixes: signed integer overflow: -4096 * -524288 cannot be represented in type 'int'
Fixes: 8650/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RA_144_fuzzer-5734816036159488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-27 13:43:39 +02:00
Lukas Rusak
7395f13df9 v4l2_m2m: decoder: fix memory leak 2018-06-27 08:38:50 +02:00
Carl Eugen Hoyos
52e8a0d96d lavc/libopenjpeg: Support GRAY10, GRAY12 and GRAY14. 2018-06-26 00:20:04 +02:00
Michael Niedermayer
78167b498f avcodec/cscd: Check output buffer size for lzo.
Fixes: Timeout
Fixes: 8665/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CSCD_fuzzer-5768442610188288

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
2018-06-25 22:58:45 +02:00
Michael Niedermayer
84bbdc255a avcodec/ffv1dec: Check state transition table
Fixes: Timeout
Fixes: 8646/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFV1_fuzzer-5649968353247232

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 22:58:45 +02:00
Michael Niedermayer
6677c98626 avcodec/escape124: Check buf_size against num_superblocks
Fixes: Timeout
Fixes: 8722/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-4843268402577408

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 22:58:45 +02:00
Michael Niedermayer
d25c945247 avcodec/h264_parser: Reduce needed history for parsing mb index
This fixes a bug/regression with very small packets
Fixes: output_file

Regression since: 0782fb6bcb

Reported-by: Thierry Foucu <tfoucu@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 02:11:13 +02:00