Commit graph

370 commits

Author SHA1 Message Date
Michael Niedermayer
b66aa37834 avcodec/h264_slice: Fix overflow in slice offset
Fixes: runtime error: signed integer overflow: 1610612736 * 2 cannot be represented in type 'int'
Fixes: 2817/clusterfuzz-testcase-minimized-5289691240726528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1f53bde6d8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-09-17 12:21:32 +02:00
Michael Niedermayer
aaecd2ef68 avcodec/h264_slice: Fix signed integer overflow
Fixes: runtime error: signed integer overflow: 26 + 2147483644 cannot be represented in type 'int'
Fixes: 2456/clusterfuzz-testcase-minimized-4822695051001856

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7592d97f10)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-07-19 02:50:18 +02:00
Michael Niedermayer
e5714e4ccb avcodec/h264_parse: Check picture structure when initializig weight table
Fixes: runtime error: index 49 out of bounds for type 'int [48][2][2]'
Fixes: 2159/clusterfuzz-testcase-minimized-5267945972301824

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3a1ad368a78b153b63ccc07af864b3611e2a4ac3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-06-16 01:05:29 +02:00
Michael Niedermayer
7655f73c45 avcodec/h264: Check weight values to be within the specs limits.
Fixes: integer overflows
Fixes: 911/clusterfuzz-testcase-5415105606975488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 08117a4015)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-14 12:20:15 +02:00
Michael Niedermayer
63637e457c avcodec/h264_slice: Clear ref_counts on redundant slices
Fixes reading freed memory
Fixes: 568/clusterfuzz-testcase-6107186067406848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c03029a835)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-08 19:22:05 +01:00
Timothy Gu
7f7cedd713 Merge commit '58640fe89e'
* commit '58640fe89e':
  h264: handle frame recovery in h264_field_start()

Conflicts:
	libavcodec/h264dec.c

Merged-by: Timothy Gu <timothygu99@gmail.com>
2016-08-03 20:07:21 -07:00
Timothy Gu
de62531286 Merge commit 'e26c64148b'
* commit 'e26c64148b':
  h264: discard slices of redundant pictures right after parsing the slice header

Conflicts:
	libavcodec/h264_slice.c
	libavcodec/h264dec.c

TODO: fix indentation

Merged-by: Timothy Gu <timothygu99@gmail.com>
2016-08-03 19:50:06 -07:00
Timothy Gu
e0c8250a59 Merge commit '6efc463858'
* commit '6efc463858':
  h264: always set redundant_pic_count during slice header parsing

Merged-by: Timothy Gu <timothygu99@gmail.com>
2016-08-03 19:09:19 -07:00
Timothy Gu
826c780c12 Revert "Revert "Merge commit 'd1d7678040cd60148f97b372cb4291bcc45b2e22'""
This reverts commit e4af9be0f4 and redoes
796027f221.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-03 17:45:52 -07:00
Timothy Gu
e4af9be0f4 Revert "Merge commit 'd1d7678040'"
This reverts commit 796027f221, reversing
changes made to bca30ed2b6.

Preemptive revert before further testing has been done.
2016-08-03 17:06:55 -07:00
Timothy Gu
796027f221 Merge commit 'd1d7678040'
* commit 'd1d7678040':
  h264: fix the check for mixed IDR/non-IDR slices

Conflicts:
	libavcodec/h264_slice.c
	libavcodec/h264dec.c

Merged-by: Timothy Gu <timothygu99@gmail.com>
2016-08-03 15:59:51 -07:00
Timothy Gu
bca30ed2b6 Merge commit 'b13fc1e344'
* commit 'b13fc1e344':
  h264: do not pass H264Context to h264_slice_header_parse()

Conflicts:
	libavcodec/h264dec.h

Did not merge the h264_slice_header_parse() part. We use a few other
members of H264Context for error checking in that function.

Merged-by: Timothy Gu <timothygu99@gmail.com>
2016-08-03 15:41:59 -07:00
James Almer
01f0598b18 Merge commit '996f13413b'
* commit '996f13413b':
  h264: fix warnings in fill_filter_caches_inter()

Conflicts:
libavcodec/h264_slice.c

See ef8f6464a5

Merged-by: James Almer <jamrial@gmail.com>
2016-08-01 19:11:08 -03:00
James Almer
517dd04f6d Merge commit 'bc7f426851'
* commit 'bc7f426851':
  h264: drop tests whether the codec id is AV_CODEC_ID_H264

Conflicts:
libavcodec/h264dec.c

Merged-by: James Almer <jamrial@gmail.com>
2016-08-01 18:15:07 -03:00
James Almer
f41048f6ec Merge commit '5c2fb561d9'
* commit '5c2fb561d9':
  h264: add H264_ prefix to the NAL unit types

Conflicts:
libavcodec/h264_parse.c
libavcodec/h264_parser.c
libavcodec/h264_slice.c
libavcodec/h264dec.c

Merged-by: James Almer <jamrial@gmail.com>
2016-08-01 15:11:05 -03:00
James Almer
8c7932884d Merge commit 'f638b67e57'
* commit 'f638b67e57':
  h264: move the parameter set definitions to a new header file

Conflicts:
libavcodec/h264_parse.h
libavcodec/h264_ps.c
libavcodec/h264dec.h

Merged-by: James Almer <jamrial@gmail.com>
2016-08-01 12:58:09 -03:00
Clément Bœsch
ecf65c30cf Merge commit '251cbb4400'
* commit '251cbb4400':
  h264: create a new header for common h264 definitions

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-29 11:13:44 +02:00
Clément Bœsch
15b26e88cb Merge commit '9df889a5f1'
* commit '9df889a5f1':
  h264: rename h264.[ch] to h264dec.[ch]

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-29 11:01:36 +02:00
Clément Bœsch
7dda97fb86 Merge commit 'f651c6a259'
* commit 'f651c6a259':
  h264: factor out setting frame properties / side data

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-29 10:39:57 +02:00
Clément Bœsch
13872d2aa1 Merge commit 'bcd91f1644'
* commit 'bcd91f1644':
  h264: move a per-field block from decode_slice_header() to field_start()

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-29 10:24:06 +02:00
Michael Niedermayer
8385e1718e avcodec/h264_slice: Make setup_finished check cover more cases 2016-07-27 19:29:55 +02:00
Clément Bœsch
6c41eda18e Merge commit 'debca90863'
* commit 'debca90863':
  h264: store {curr,max}_pic_num in the per-slice context

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-27 18:07:30 +02:00
Clément Bœsch
87d1f82059 Merge commit 'f966498e43'
* commit 'f966498e43':
  h264: decode the poc values from the slice header into the per-slice context

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-27 17:31:18 +02:00
Clément Bœsch
f8ce1e8282 lavc/h264_slice: adjust a few line breaks to reduce diff with Libav 2016-07-27 17:24:48 +02:00
Clément Bœsch
78c7197ea0 Merge commit '54dd9b1cdd'
* commit '54dd9b1cdd':
  h264: set mb_aff_frame in frame_start()
  h264: move the block starting a new field out of slice_header_parse()

Both commits are merged at the same time in order to prevent a
regression with Ticket #4440 (see 38660128).

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-27 17:00:14 +02:00
Michael Niedermayer
9bf3fdcd3c avcodec/h264_slice: Move h264_init_ps() call into h264_field_start()
This was missing from f512e4a390

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-07-20 00:07:18 +02:00
Clément Bœsch
2c138b2c9b Merge commit 'b25cd7540e'
* commit 'b25cd7540e':
  h264: pass a H2645NAL to slice header decoding

Merged-by: Clément Bœsch <u@pkh.me>
2016-07-14 19:41:37 +02:00
Clément Bœsch
f512e4a390 Merge commit '61f168ae34'
* commit '61f168ae34':
  h264: factor out setting the parameter sets for a frame

Michael's changes on top of the merge undo parts of the original diff
that are not factorization:

"The call point is left where it was before. Such a change should be in
a separate commit and has multiple issues, one being null pointer
dereferences the other is that some safety checks would become
conditional.

I tried to split the PPS init between the new and old functions
similarly to the SPS code."

Merged-by: Clément Bœsch <u@pkh.me>
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2016-07-09 13:57:43 +02:00
Clément Bœsch
64c619369b lavc/h264_slice: use sps directly when checking for invalid 8x8 inference 2016-07-03 15:11:40 +02:00
James Almer
77eb05a2f1 avcodec/h264_slice: Only call ff_h264_flush_change() on initialized contexts
Changed by committer to ensure context_initialized is reset
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-07-01 21:39:09 +02:00
Clément Bœsch
5565e27111 Merge commit '6e92181bf8'
* commit '6e92181bf8':
  h264: pass just the PPS to get_chroma_qp()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-07-01 14:23:38 +02:00
Clément Bœsch
2021326f99 Merge commit 'd1f539c97e'
* commit 'd1f539c97e':
  h264: merge the two reinit blocks in slice_header_parse()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-07-01 14:19:09 +02:00
Clément Bœsch
99b37f53a1 Merge commit '3fba16ecd9'
* commit '3fba16ecd9':
  h264: factor starting a new field out of parsing the slice header

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-07-01 14:02:00 +02:00
Clément Bœsch
f48aea66dd Merge commit 'bec993381c'
* commit 'bec993381c':
  h264: postpone generating the implicit MMCOs

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-30 10:24:03 +02:00
Clément Bœsch
d407e76c42 Merge commit '2d410ebbaa'
* commit '2d410ebbaa':
  h264: decode the MMCOs into per-slice contexts

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-30 10:15:51 +02:00
Benoit Fouet
879330c561 h264: make H264ParamSets sps const 2016-06-30 09:24:39 +02:00
Benoit Fouet
4cc1ce4a91 h264: straighten dimensions check ff_h264_decode_seq_parameter_set
The MBS only flag was not taken into account when checking macroblock dimensions.
Also removes the unneeded check in init_dimensions for slices.
2016-06-30 09:24:39 +02:00
Clément Bœsch
57d30fde9e Merge commit '39ab2ea531'
* commit '39ab2ea531':
  h264: rename mmco_index to nb_mmco

Merged-by: Clément Bœsch <u@pkh.me>
2016-06-29 20:07:52 +02:00
Clément Bœsch
c28aecc56a Merge commit '7ab5d577a9'
* commit '7ab5d577a9':
  h264: move initializing the slice start out of h264_slice_header_parse()

Merged-by: Clément Bœsch <u@pkh.me>
2016-06-29 20:04:23 +02:00
Clément Bœsch
4890b0851c Merge commit '4cec43a9ee'
* commit '4cec43a9ee':
  h264: move calculating the POC out of h264_slice_header_parse()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 17:59:57 +02:00
Clément Bœsch
9ab6f01839 Merge commit '6dd996c7c8'
* commit '6dd996c7c8':
  h264: move building the reference list out of h264_slice_header_parse()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 17:42:06 +02:00
Clément Bœsch
3b95452ca8 Merge commit '0bad254300'
* commit '0bad254300':
  h264: move initing the implicit pred weight table out of h264_slice_header_parse()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 17:39:33 +02:00
Clément Bœsch
2aff557c6a Merge commit 'ed9a20ebe4' into merge-libav-new
* commit 'ed9a20ebe4':
  h264: split reading the ref list modifications and actually building the ref list

ref_modifications.val are read as u32 instead of u8 in FFmpeg.

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 17:32:05 +02:00
Clément Bœsch
a2901472fe lavc/h264_slice: properly forward positive "error" code
Fixes ./ffmpeg -skip_frame nokey -i h264/h264_intra_first-small.ts

Regression since c54e2740e1
2016-06-29 16:16:26 +02:00
Clément Bœsch
cdecb39fca Merge commit 'fa57059079'
* commit 'fa57059079':
  h264: move initialising the implicit pred weight table for MBAFF

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 15:29:38 +02:00
Clément Bœsch
527a579327 Merge commit '7b50d60442'
* commit '7b50d60442':
  h264: call ff_h264_fill_mbaff_ref_list() when constructing the normal ref list

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 15:17:16 +02:00
Clément Bœsch
3c5a3882f9 Merge commit '77a1e2c5f8'
* commit '77a1e2c5f8':
  h264: move direct mode inits out of h264_slice_header_parse()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 14:58:31 +02:00
Clément Bœsch
c54e2740e1 Merge commit 'd06e4d8aab'
* commit 'd06e4d8aab':
  h264: start splitting decode_slice_header()

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 14:55:02 +02:00
Michael Niedermayer
0c50f6905f avcodec/h264: Remove au_pps_id
This should not be needed anymore and simplifies the next merge

Requested-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-06-29 14:49:15 +02:00
Michael Niedermayer
e6e8750e94 avcodec/h264: Remove current_sps_id
This should not be needed anymore and simplifies the next merge

Requested-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-06-29 14:49:15 +02:00