Commit graph

46846 commits

Author SHA1 Message Date
Michael Niedermayer
515c7b21f4
avcodec/utils: fix 2 integer overflows in get_audio_frame_duration()
Fixes: signed integer overflow: 256 * 668003712 cannot be represented in type 'int'
Fixes: 59819/clusterfuzz-testcase-minimized-ffmpeg_dem_MATROSKA_fuzzer-4674636538052608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a4bf559683)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:58 +01:00
Michael Niedermayer
9b10b9b8cb
avcodec/hevcdec: Avoid null pointer dereferences in MC
Fixes: runtime error: pointer index expression with base 0x000000000000 overflowed to 0xfffffffffffffff8
Fixes: 58440/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5956015530311680

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a0f4d4e650)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:58 +01:00
Michael Niedermayer
df4170b2bc
avcodec/takdsp: Fix integer overflows
Fixes: avcodec/takdsp.c:44:23: runtime error: signed integer overflow: -2097158 - 2147012608 cannot be represented in type 'int'
Fixes: 58417/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TAK_fuzzer-5268919664640000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ff8a496d41)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:57 +01:00
Michael Niedermayer
74d3c9261a
avcodec: Ignoring errors is only possible before the input end
Fixes: out of array read
Fixes: Ticket 10308

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fead656a7b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:57 +01:00
Michael Niedermayer
2cb93e863f
avcodec/noise_bsf: Check for wrapped frames
Wrapped frames contain pointers so they need specific code to
noise them, the generic code would lead to segfaults

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0889ebc577)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:56 +01:00
Michael Niedermayer
7337f2e8e3
avcodec/tiff: add a zero DNG_LINEARIZATION_TABLE check
Fixes: index 4294967295 out of bounds for type 'uint16_t [65536]'
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5950405086674944
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-6666195176914944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6e98cf0280)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:55 +01:00
Michael Niedermayer
f174aa7134
avcodec/tak: Check remaining bits in ff_tak_decode_frame_header()
Fixes: out of array access
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TAK_fuzzer-6682195323650048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 19b66b89da)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:54 +01:00
Michael Niedermayer
2878299b73
avcodec/sonic: Fix two undefined integer overflows
Fixes: signed integer overflow: 2147483372 - -148624 cannot be represented in type 'int'
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-5477177805373440
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-6681622236233728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2632e90493)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:54 +01:00
Michael Niedermayer
0c2b08227e
avcodec/utils: the IFF_ILBM implementation assumes that there are a multiple of 16 allocated
Fixes: out of array access
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5124452659888128
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-6362836707442688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 34056a94ea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:54 +01:00
Michael Niedermayer
b9c79be726
avcodec/exr: Cleanup befor return
Fixes: leaks
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-6703454090559488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 885ff3b879)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:53 +01:00
Michael Niedermayer
13644a75d8
avcodec/pngdec: Do not pass AVFrame into global header decode
The global header should not contain a frame, and decoding it
would result in leaks

Fixes: memleak
Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APNG_fuzzer-6603443149340672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d31d4f3228)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:53 +01:00
Michael Niedermayer
ac015347a8
avcodec/pngdec: remove AVFrame argument from decode_iccp_chunk()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7117b380a7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:53 +01:00
Michael Niedermayer
efcb821157
avcodec/vorbisdec: Check codebook float values to be finite
Fixes: Timeout
Fixes: 55116/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-4572159970508800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cadd7e7a75)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:52 +01:00
Michael Niedermayer
c2b46db3f4
avcodec/g2meet: Replace fake allocation avoidance for framebuf
framebuf is only allocated when the new width/height are larger than the old
but nothing sets the old so its always allocated.
Use av_fast_mallocz() instead.

Fixes: Timeout
Fixes: 55094/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G2M_fuzzer-5116909932904448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 38adbc6eeb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:52 +01:00
Michael Niedermayer
1480cc1d9c
avcodec/lcldec: More space for rgb24
Fixes: Ticket 10239
Fixes: zlib_306_306_rgb24.av

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e2c3aa8e2b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:51 +01:00
Michael Niedermayer
002471255d
avcodec/lcldec: Support 4:1:1 and 4:2:2 with odd width
Fixes: Ticket10240
Fixes: zlib_306_306_yuv422.avi
Fixes: zlib_306_306_yuv411.avi

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0cf1ac905d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:51 +01:00
Michael Niedermayer
8abcd8b124
libavcodec/lcldec: width and height should not be unsigned
Computations like col < width - 3 will not work with unsigned width=1

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3eb4e28c26)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:51 +01:00
Michael Niedermayer
d1553cad73
avcodec/escape124: Check that blocks are allocated before use
Fixes: NULL pointer dereference
Fixes: 57819/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-5077280228769792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5366ae12b9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:50 +01:00
Michael Niedermayer
7158ad82da
avcodec/huffyuvdec: Fix undefined behavior with shift
Fixes: left shift of negative value -1
Fixes: 57554/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFVHUFF_fuzzer-4853603839115264

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 27e7857bd1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:50 +01:00
Michael Niedermayer
7c5d1725aa
avcodec/j2kenc: Replace RGB24 special case by generic test
This fixes RGB48 with libavcodec as decoder

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ad4d647591)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:49 +01:00
Michael Niedermayer
4c07c4d29a
avcodec/j2kenc: Replace BGR48 / GRAY16 test by test for number of bits
BGR48 is not supported and this was probably meant to be RGB48 so this fixes
RGB48 a bit

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7fb70d27a2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:49 +01:00
Michael Niedermayer
cf6ae79a71
avcodec/j2kenc: simplify pixel format setup
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 644d15716d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:49 +01:00
Michael Niedermayer
0ab5965b69
avcodec/j2kenc: Fix funky bpno errors on decoding
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 816676085e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:48 +01:00
Michael Niedermayer
1990527edb
avcodec/j2kenc: remove misleading pred value
This field is only checked for being 0 or not and not zero means 5/3

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0adb375377)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:48 +01:00
Michael Niedermayer
f38e5efbf2
avcodec/j2kenc: fix 5/3 DWT identifer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f6955b6df4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:48 +01:00
Michael Niedermayer
f34fe6b4a4
avcodec/vp3: Check width to avoid assertion failure
Fixes: Assertion failure on x86-32
       av_assert2(block_w * sizeof(pixel) <= FFABS(buf_linesize)); in ff_emulated_edge_mc()
Fixes: 39641/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THEORA_fuzzer-5925660741206016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dab1cd2dc0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:47 +01:00
Michael Niedermayer
538f355b9a
avcodec/g729postfilter: Limit shift in long term filter
Fixes: shift exponent 34 is too large for 32-bit type 'int'
Fixes: 57389/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-6229522659016704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6d1d8609ac)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-30 00:38:47 +01:00
Rémi Denis-Courmont
25cd95a9dc avcodec/x86/mathops: clip constants used with shift instructions within inline assembly
Fixes assembling with binutil as >= 2.41

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit effadce6c7)
2023-07-22 19:29:08 -03:00
Jan Ekström
e5b5dd6653 avcodec/libsvtav1: use larger of bit rate and max rate for buffer size
Generally if maxrate is set, the calculation should be maxrate over
bufsize. This additionally enables CRF + maxrate & bufsize usage.

In order to keep negative values from enabling zero to be treated
as larger and causing a division by zero, check that one of the
variables is larger than zero.
2023-06-06 23:54:50 +02:00
Christopher Degawa
8fe2fec453 avcodec/libsvtav1: only set max_buf_sz if both bitrate and rc_buf_sz is set
maximum_buffer_size_ms should only be set if both are specified or if
the user sets it through -svtav1-params buf-sz=val

Signed-off-by: Christopher Degawa <ccom@randomderp.com>
2023-06-06 23:50:18 +02:00
Christopher Degawa
3344d47a88 avcodec/libsvtav1: remove compressed_ten_bit_format and simplify alloc_buffer
compressed_ten_bit_format has been deprecated upstream and has no effect
and can be removed. Plus, technically it was never used in the first place
since it would require the app (ffmpeg) to set it and do additional
processing of the input frames.

Also simplify alloc_buffer by removing calculations relating to the
non-existant processing.

Signed-off-by: Christopher Degawa <christopher.degawa@intel.com>
2023-05-30 00:28:14 +02:00
Christopher Degawa
bea695d543 avcodec/libsvtav1: replace vbv_bufsize with maximum_buffer_size_ms
svt-av1 v1.2.0 has deprecated vbv_bufsize in favor of using
- maximum_buffer_size_ms (--buf-sz)
- starting_buffer_level_ms (--buf-initial-sz)
- optimal_buffer_level_ms (--buf-optimal-sz)

and vbv_bufsize has not been in use since svt-av1 v0.8.6

Signed-off-by: Christopher Degawa <christopher.degawa@intel.com>
2023-05-29 23:12:13 +02:00
Hendrik Leppkes
33ed503e59
avcodec/vdpau_mpeg4: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:42:37 +02:00
Hendrik Leppkes
4015609fcd
avcodec/vdpau_mpeg12: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:42:37 +02:00
Hendrik Leppkes
a4f995fa84
avcodec/nvdec_mpeg4: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:42:37 +02:00
Hendrik Leppkes
e6f7183eb6
avcodec/nvdec_mpeg2: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:42:37 +02:00
James Almer
c36f69d7e3 avcodec/mp_cmp: reject invalid comparison function values
Fixes tickets #10306 and #10318.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 7c6e26a184)
2023-04-16 11:39:13 -03:00
James Almer
58912f665b avcodec/aacpsy: clip global_quality within the psy_vbr_map array boundaries
Fixes ticket #10317.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 5cda6b94f4)
2023-04-16 11:39:13 -03:00
Kyle Manning
0028275216 avcodec/nvenc: fix b-frame DTS behavior with fractional framerates
When using fractional framerates (or any fraction with a numerator != 1),
DTS values for packets would be calculated incorrectly.

Signed-off-by: Kyle Manning <tt2468@irltoolkit.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2023-04-03 12:05:15 +02:00
Michael Niedermayer
037c7a2eac
avcodec/tests/snowenc: Fix 2nd test
(cherry picked from commit 163013c724)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-26 15:30:31 +02:00
Michael Niedermayer
06dfb4fef2
avcodec/tests/snowenc: return a failure if DWT/IDWT mismatches
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 771c266c0b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-26 15:30:31 +02:00
Michael Niedermayer
28b03ac192
avcodec/snowenc: Fix visual weight calculation
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b5fcadea0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-26 15:30:30 +02:00
Michael Niedermayer
29412c75eb
avcodec/tests/snowenc: unbreak DWT tests
the IDWT data type mismatched current code

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b3351bbea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-26 15:30:30 +02:00
Michael Niedermayer
43ea18703c
avcodec/mpeg12dec: Check input size
Fixes: Timeout
Fixes: 53599/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IPU_fuzzer-4950102511058944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c130d6911)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:41 +01:00
Michael Niedermayer
405bfbd873
avcodec/escape124: Fix some return codes
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 98df605f7a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:41 +01:00
Michael Niedermayer
0dca540766
avcodec/escape124: fix signdness of end of input check
Fixes: Timeout
Fixes: 56561/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-5560363635834880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 87ad0a5dd7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:41 +01:00
Paul B Mahol
7c234248f8
avcodec/rpzaenc: stop accessing out of bounds frame
(cherry picked from commit 92f9b28ed8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:40 +01:00
Paul B Mahol
9886e4c3b0
avcodec/smcenc: stop accessing out of bounds frame
(cherry picked from commit 13c1310975)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:40 +01:00
Michael Niedermayer
0b6c4936dd
avcodec/motionpixels: Mask pixels to valid values
Fixes: out of array access
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOTIONPIXELS_fuzzer-6724203352555520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ac6eec1fc2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:39 +01:00
Michael Niedermayer
fbe44d7a82
avcodec/xpmdec: Check size before allocation to avoid truncation
Fixes:OOM
Fixes:out of array access (no testcase)
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XPM_fuzzer-6573323838685184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 95f0f84dae)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-03-16 14:48:39 +01:00