Commit graph

32452 commits

Author SHA1 Message Date
Michael Niedermayer
6e53134f98 avcodec/pthread_frame: check avctx on deallocation
Fixes null pointer dereferences
Fixes: af1a5a33e67e479f439239097bd0d4fd_signal_sigsegv_7ffff713351a_152_Dolby_Rain_Logo.pmp with memlimit of 8388608

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5d346feafa)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-18 20:23:46 +02:00
Michael Niedermayer
237751eb25 avcodec/sanm: Reset sizes in destroy_buffers()
Fixes crash in 1288a2fe8e9ae6b00ca40e089d08ca65_signal_sigsegv_7ffff71426a7_354_accident.san with allocation limit 65536

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 39bbdebb1e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-18 20:23:46 +02:00
Michael Niedermayer
264eb0074f avcodec/alac: Clear pointers in allocate_buffers()
Fixes: 06a4edb39ad8a9883175f9bd428334a2_signal_sigsegv_7ffff713351a_706_mov__alac__ALAC_6ch.mov

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f7068bf277)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-18 20:23:46 +02:00
Anton Khirnov
7db809a373 bytestream2: set the reader to the end when reading more than available
This prevents possible infinite loops with the calling code along the
lines of while (bytestream2_get_bytes_left()) { ... }, where the reader
does not advance.

CC: libav-stable@libav.org
(cherry picked from commit 86eee85dad)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
0df814cf97 avcodec/utils: use a minimum 32pixel width in avcodec_align_dimensions2() for H.264
Fixes Assertion failure
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7ef6656b1e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
88fa3243dd avcodec/mpegvideo: Clear pointers in ff_mpv_common_init()
This ensures that no stale pointers leak through on any path

Fixes: signal_sigsegv_c3097a_991_xtrem_e2_m64q15_a32sxx.3gp

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b160fc290c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Andreas Cadhalpun
c001472226 wmalosslessdec: reset frame->nb_samples on packet loss
Otherwise a frame with non-zero nb_samples but without any data can be
returned.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 42e7a5b3c7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Andreas Cadhalpun
1ec0541ae0 wmalosslessdec: avoid reading 0 bits with get_bits
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f9020d514e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
ac91bfe086 avcodec/rawenc: Use ff_alloc_packet() instead of ff_alloc_packet2()
the later is not optimal when the buffer size is well known at allocation time

This avoids a memcpy()
Overall 2.5% speedup with a random 1920x1080 video

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 47496eb97c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
bbcf6f5c62 avcodec/aacsbr: Assert that bs_num_env is positive
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2e13a45b1a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
e740506d31 avcodec/aacsbr: check that the element type matches before applying SBR
Fixes out of array access
Fixes: signal_sigsegv_3670fc0_2818_cov_2307326154_moon.mux

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 79a98294da)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
65aac419e5 avcodec/h264_slice: Use w/h from the AVFrame instead of mb_w/h
Fixes out of array access
Fixes: asan_heap-oob_4d5bb0_682_cov_3124593265_Fraunhofer__a_driving_force_in_innovation__small.mp4

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 330863c9f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
James Zern
662714abbe vp9/update_prob: prevent out of bounds table read
the max value of the lookup in expanded form is:
(((1 << 7) - 1) << 1) - 65 + 1 + 64 = 254

add one entry of padding to inv_map_table[] to prevent out of bounds
access with non-conforming / fuzzed bitstreams

Signed-off-by: James Zern <jzern@google.com>
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91f860ea7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
0afb004d3c avcodec/h264_refs: discard mismatching references
Fixes inconsistency and out of array access
Fixes: asan_heap-oob_17301a3_2100_cov_3226131691_ff_add_pixels_clamped_mmx.m2ts

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4aa0de644a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
a9c3b588af avcodec/mjpegdec: Fix small picture upscale
Fixes out of array access

Fixes: asan_heap-oob_1dd60fd_267_cov_2954683513_5baad44ca4702949724234e35c5bb341.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 84afc6b70d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
f775a92054 avcodec/pngdec: Check values before updating context in decode_fctl_chunk()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b54ac8403b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
cccb06b095 avcodec/pngdec: Copy IHDR & plte state from last thread
Previously these chunks where parsed again for each frame with threads
but not without leading to a different path and the potential for
inconsistencies

This also removes a related special case from decode_ihdr_chunk()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1ffa01dd3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
be54d1f104 avcodec/pngdec: Require a IHDR chunk before fctl
This is required by the APNG spec

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a1736926e9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Michael Niedermayer
e84d17c7c9 avcodec/pngdec: Only allow one IHDR chunk
Multiple IHDR chunks are forbidden in PNG
Fixes inconsistency and out of array accesses

Fixes: asan_heap-oob_4d5c5a_1738_cov_2638287726_c-m2-8f2b481b7fd9bd745e620b7c01a18df2.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 47f4e2d896)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Andreas Cadhalpun
254fabe758 wmavoice: limit wmavoice_decode_packet return value to packet size
Claiming to have decoded more bytes than the packet size is wrong.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2a4700a4f0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-18 20:23:45 +02:00
Gwenole Beauchesne
071d7f4e17 vaapi_h264: fix RefPicList[] field flags.
Use new H264Ref.reference field to track field picture flags. The
H264Picture.reference flag in DPB is now irrelevant here.

This is a regression from git commit d8151a7, and that affected
multiple interlaced video streams.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
(cherry picked from commit 88325c2e0b)
2015-06-23 13:52:30 +02:00
Andreas Cadhalpun
620197d1ff doc: avoid incorrect phrase 'allows to'
Also fix typo found by Lou Logan:
Sacrifying -> Sacrificing

Reviewed-by: Lou Logan <lou@lrcd.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 28efeb6502)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-06-19 20:31:44 +02:00
Michael Niedermayer
157dd52700 avcodec/dpxenc: implement write16/32 as functions
Fixes undefined behavior and segfault

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8edc17b639)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-19 18:40:49 +02:00
Andreas Cadhalpun
250cf2d4da vp9: don't retain NULL as segmentation_map
This fixes segmentation faults, which were introduced in commit
4ba8f327.

Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d216b9debd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-19 18:40:49 +02:00
Michael Niedermayer
8749b83e0b avcodec/libx264: Avoid reconfig on equivalent aspect ratios
Workaround for ticket #4287.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7b1c03aa74)

Conflicts:
	libavcodec/libx264.c
2015-06-19 13:27:42 +02:00
George Boyle
a9b600cf39 avcodec/flacenc: Fix Invalid Rice order
Fixes ticket #4628.

The problem arose, in the sample file at least, in the last block where the
minimum and maximum Rice partition orders were both 0. In that case, and any
other where pmax == pmin, the original UINT32_MAX placeholder value for
bits[opt_porder] was getting overwritten before the comparison to check if the
current partition order is a new optimal, so the correct partition order and
RiceContext params were not being set.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2469ed32c8)
2015-06-19 10:10:43 +02:00
Michael Niedermayer
6432f8826d avcodec/hevc_ps: Only discard overread VPS if a previous is available
Fixes Ticket4621

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57078e4d25)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-18 22:09:01 +02:00
Michael Niedermayer
08fadda68a avcodec/ffv1enc: fix bps for >8bit yuv when not explicitly set
Fixes Ticket4636

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3a6a8f6ee1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-18 22:09:01 +02:00
Andreas Cadhalpun
3ba55ea4ae h264: er: Copy from the previous reference only if compatible
Also use the frame pixel format instead of the one from the codec
context, which is more robust.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdc64a1044)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-18 22:09:01 +02:00
Andreas Cadhalpun
265db41540 doc: fix spelling errors
Neccessary -> Necessary
formated   -> formatted
thee       -> the
eventhough -> even though
seperately -> separately

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ed0b1db640)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-18 22:09:01 +02:00
Andreas Cadhalpun
1051c152f9 takdec: ensure chan2 is a valid channel index
If chan2 is not smaller than the number of channels, it can cause
segmentation faults due to dereferencing a NULL pointer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 05c57ba2f4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 12:23:21 +02:00
Michael Niedermayer
5a0862af55 avcodec/h264_slice: Use AVFrame diemensions for grayscale handling
The AVFrame values are closer to the AVFrame bitmap changed instead of
the AVCodecContext values, so this should be more robust

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit aef0e0f009)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 12:05:42 +02:00
Andreas Cadhalpun
58995f647b sonic: set avctx->channels in sonic_decode_init
Otherwise it can be 0 in sonic_decode_frame, causing SIGFPE crashes.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-06-09 23:21:51 +02:00
Michael Niedermayer
0916938a8d avcodec/jpeg2000dec: Fix gain regression with gray16
Found-by: carl
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 20:26:38 +02:00
Michael Niedermayer
adbe1d7de4 avcodec/libopenjpegdec: Mark as experimental if <= 1.3
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 18:59:30 +02:00
Michael Niedermayer
dbbb31e5bb avcodec/jpeg2000: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 18:48:39 +02:00
Michael Niedermayer
551813a963 Merge commit '210921722b'
* commit '210921722b':
  imc: add required padding for GetBitContext buffer

Conflicts:
	libavcodec/imc.c

See: 7444cf9a9c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 17:44:13 +02:00
Michael Niedermayer
44327cbc9a avcodec/jpeg2000: Disable special case for JPEG2000_QSTY_SI
The code gave apparently completely wrong values

Fixes Ticket2872

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 16:44:32 +02:00
Michael Niedermayer
45db921806 avcodec/jpeg2000dec: Add placeholder for PLT parsing
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 15:16:46 +02:00
Michael Niedermayer
b6ee1912f9 avcodec/jpeg2000dec: Handle Psot = 0
Fixes Ticket2869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 14:43:03 +02:00
Janne Grunau
210921722b imc: add required padding for GetBitContext buffer
Fixes stack buffer overflow errors detected by address sanitizer in
fate-imc.

CC: libav-stable@libav.org
2015-06-09 10:07:04 +02:00
Janne Grunau
09447f2b0f ac3_parser: add required padding for GetBitContext buffer
Fixes stack buffer overflow errors detected by address sanitizer in
various fate tests.

CC: libav-stable@libav.org
2015-06-09 10:06:59 +02:00
Janne Grunau
fb14730802 aac_parser: add required padding for GetBitContext buffer
Fixes stack buffer overflow errors detected by address sanitizer in
various fate tests.

CC: libav-stable@libav.org
2015-06-09 10:05:54 +02:00
Michael Niedermayer
e100966575 avcodec/x86/h264_weight: handle weight1=128
Fix ticket4596

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 05:11:09 +02:00
Michael Niedermayer
7c9fcdfabd avcodec/jpeg2000dec: Fix some 5/3 bitexactness issues
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 02:17:53 +02:00
Michael Niedermayer
074159ed70 avcodec/jpeg2000dec: Fix subsampled decoding
Fixes part of Ticket3619

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 01:02:05 +02:00
Michael Niedermayer
12ba1b2b4d avcodec/jpeg2000dec: Check that coords match before applying ICT
This avoid potential out of array accesses

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 01:02:05 +02:00
Michael Niedermayer
028c59c17b avcodec/jpeg2000dec: Fix high bit depth branch sample shift
Fix part of Ticket3619

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-09 01:01:01 +02:00
Andreas Cadhalpun
6fdbaa2b7f vp8: change mv_{min,max}.{x,y} type to int
If one of the dimensions is larger than 8176, s->mb_width or
s->mb_height is larger than 511, leading to an int16_t overflow of
s->mv_max.{x,y}. This then causes av_clip to be called with amin > amax.

Changing the type to int avoids the overflow and has no negative
effect, because s->mv_max is only used in clamp_mv for clipping.
Since mv_max.{x,y} is positive and mv_min.{x,y} negative, av_clip can't
increase the absolute value. The input to av_clip is an int16_t, and
thus the output fits into int16_t as well.

For additional safety, s->mv_{min,max}.{x,y} are clipped to int16_t range
before use.

Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-06-08 23:29:14 +02:00
Michael Niedermayer
83e3516e64 Merge commit '925b80d640'
* commit '925b80d640':
  mpegvideo: Move OutFormat enum to mpegutils.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-08 22:05:25 +02:00