Commit graph

114 commits

Author SHA1 Message Date
Frank Plowman
d866f49791 lavc/vvc: Reallocate pixel buffers if pixel shift changes
Allocations in the following lines depend on the pixel shift, and so
these buffers must be reallocated if the pixel shift changes.  Patch
fixes segmentation faults in fuzzed bitstreams.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-06-04 20:13:47 +08:00
Michael Niedermayer
2e5433dc12
avcodec/vvc/mvs: Initialize mvf
This might not be needed for correctness but it could
help general reproducability of issues

Related to: CID1560037 Uninitialized scalar variable
Related to: CID1560044 Uninitialized scalar variable

Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-06-02 22:29:22 +02:00
Frank Plowman
49c3918c1a lavc/vvc: Validate temporal MVP references
Per VVCv3 p. 157, the collocated reference picture used in temporal
motion vector prediction must have RprConstraintsActiveFlag equal to
zero and the same CTU size as the current picture.  Add these checks,
fixing crashes decoding some fuzzed bitstreams.

Additionally, only set up the collocated reference picture if it is
actually going to be used (i.e. if ph_temporal_mvp_enabled_flag is 1),
else legal RPR bitstreams will fail the new checks.

Co-authored-by: Nuo Mi <nuomi2021@gmail.com>
Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-05-27 20:24:21 +08:00
Nuo Mi
1b33c9a50a avcodec/vvcdec: support Reference Picture Resampling
passed clips:
    RPR_A_Alibaba_4.bit
    RPR_B_Alibaba_3.bit
    RPR_C_Alibaba_3.bit
    RPR_D_Qualcomm_1.bit
    VVC_HDR_UHDTV1_OpenGOP_Max3840x2160_50fps_HLG10_res_change_with_RPR.ts
2024-05-21 20:20:25 +08:00
Nuo Mi
cae0b01282 avcodec/vvcdec: increase edge_emu_buffer for RPR 2024-05-21 20:20:25 +08:00
Nuo Mi
7904ec2d34 avcodec/vvcdec: refact, remove hf_idx and vf_idx from mc_xxx's param list 2024-05-21 20:20:25 +08:00
Nuo Mi
77d971c348 avcodec/vvcdec: refact out luma_prof from luma_prof_bi 2024-05-21 20:20:25 +08:00
Nuo Mi
ac4575594f avcodec/vvcdec: fix dmvr, bdof, cb_prof for RPR 2024-05-21 20:20:25 +08:00
Nuo Mi
77acd0a0dd avcodec/vvcdec: inter, wait reference with a different resolution
For RPR, the current frame may reference a frame with a different resolution.
Therefore, we need to consider frame scaling when we wait for reference pixels.
2024-05-21 20:20:25 +08:00
Nuo Mi
deda59a996 avcodec/vvcdec: add RPR dsp 2024-05-21 20:20:25 +08:00
Nuo Mi
e70225e0a8 avcodec/vvcdec: emulated_edge, use reference frame's sps and pps
a preparation for Reference Picture Resampling
2024-05-21 20:20:25 +08:00
Nuo Mi
aa8d5c6e7e avcodec/vvcdec: add vvc inter filters for RPR 2024-05-21 20:20:25 +08:00
Nuo Mi
08ad51ece6 avcodec/vvcdec: refact, pred_get_refs return VVCRefPic instead of VVCFrame 2024-05-21 20:20:25 +08:00
Nuo Mi
66c6bee061 avcodec/vvcdec: refact out VVCRefPic from RefPicList 2024-05-21 20:20:25 +08:00
Nuo Mi
44bbafb69f avcodec/vvcdec: refact, unify pred_regular_{luma, chroma} to pred_regular 2024-05-21 20:20:25 +08:00
Nuo Mi
875fa9692c avcodec/vvcdec: misc, remove unused EMULATED_EDGE_{LUMA, CHROMA}, EMULATED_EDGE_DMVR_{LUAM, CHROMA} 2024-05-21 20:20:25 +08:00
Nuo Mi
84a93d91d1 avcodec/vvcdec: refact, unify {luma, chroma}_mc_bi to mc_bi 2024-05-21 20:20:25 +08:00
Nuo Mi
6769fe1614 avcodec/vvcdec: refact, unify {luma, chroma}_mc_uni to mc_uni 2024-05-21 20:20:25 +08:00
Nuo Mi
bc099afc8d avcodec/vvcdec: refact, unify {luma, chroma}_mc to mc 2024-05-21 20:20:25 +08:00
Nuo Mi
1289da9244 avcodec/vvcdec: misc, inter, use is_chroma instead of is_luma 2024-05-21 20:20:25 +08:00
Michael Niedermayer
4a8506c794
avcodec/vvc/dec: Check init_get_bits8() for failure
Fixes: CID1560042 Unchecked return value

Sponsored-by: Sovereign Tech Fund
Reviewed-by: Nuo Mi <nuomi2021@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-05-19 22:12:55 +02:00
Nuo Mi
a9586a00df avcodec/vvcdec: ff_vvc_frame_submit, avoid initializing task twice.
For some error bitstreams, a CTU belongs to two slices/entry points.
If the decoder initializes and submmits the CTU task twice, it may crash the program
or cause it to enter an infinite loop.

Reported-by: Frank Plowman <post@frankplowman.com>
2024-05-06 20:22:42 +08:00
Jun Zhao
822e2843ca lavc/vvc/refs: Use dpb_max_num_reorder_pics to control output
Use dpb_max_num_reorder_pics to control output instead of
dpb_max_dec_pic_buffering, when dpb_max_dec_pic_buffering
is much larger than dpb_max_num_reorder_pics, it may cause
dpb overflow error.

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Signed-off-by: elinyhuang <elinyhuang@tencent.com>
2024-04-21 21:06:50 +08:00
Frank Plowman
bb9e4ff355 lavc/vvc: Skip enhancement layer NAL units
The native VVC decoder does not yet support quality/spatial/multiview
scalability.  Bitstreams requiring this feature could cause crashes.
Patch fixes this by skipping NAL units which are not in the base layer,
warning the user while doing so.

Signed-off-by: Frank Plowman <post@frankplowman.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2024-04-18 17:38:15 -03:00
Frank Plowman
993c231de0 lavc/vvc: Increase size of ctb_size_y
sps_log2_ctu_size_minus5 is between 0 and 2, with 3 reserved for future
use.  The VVC decoder allows sps_log2_ctu_size_minus5 to be 3, and so
ctb_size_y should be at least 16 bits to prevent overflows.  An
alternative patch would leave sps_log2_ctu_size_minus5 as 8 bits and
disallow sps_log2_ctu_size_minus5 = 3.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-04-15 21:06:13 +08:00
Frank Plowman
1c0149359a lavc/vvc: Fix out-of-bounds array access
The 2 which has been changed to an 8 in the array length expression is
the maximum value of sps_bitdepth_minus8.  This was missed when updating
to VVCv2, which increased this maximum from 2 to 8.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-04-09 22:35:48 +08:00
Frank Plowman
2b7e79e890 lavc/vvc: Avoid overflow in coeff scale intermediate
Make intermediate result 64-bits to avoid an overflow before the right
shift.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-04-09 22:30:35 +08:00
Frank Plowman
fcf74c5ebc lavc/vvc: Fix buffer overread in CABAC
The size variable here is taken as gospel for the bounds of the input
buffer in later logic.  Clamp it to ensure that the returned region
does not extend past that allocated in the underlying GetBitContext,
even in the case entry point offsets are signalled in the bitstream.
Also assert this for good measure.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-04-09 22:30:18 +08:00
Frank Plowman
b4baa4c0b3 lavc/vvc: Fix left shifts of negative values
All these variables lie in the range [-12..12]

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-04-09 22:13:49 +08:00
James Almer
1f049d5627 avcodec/vvc/dec: fix typo
Signed-off-by: James Almer <jamrial@gmail.com>
2024-04-08 09:10:04 -03:00
James Almer
9eca5b8369 avcodec/vvc/ps: reset sps_id_used on flush
Signed-off-by: James Almer <jamrial@gmail.com>
2024-04-08 08:56:53 -03:00
Frank Plowman
2d79ae3f8a lavc/vvc: Error if SPS ID is duplicated within CVS
Key line from the spec is:

"All SPS NAL units with a particular value of sps_seq_parameter_set_id
in a CVS shall have the same content."

Prior to this patch, the VVC decoder's behaviour on encountering a
duplicated SPS ID (within the entire bitstream, not restricted to
a CVS) was simply to replace the entry in the SPS lookup table with the
new data.  Illegal bitstreams with multiple SPSs in the same CVS sharing
an ID but differing elsewhere could cause all manner of issues.

The patch tracks which SPS IDs have been used in the given CVS using the
new sps_id_used field of VVCParamSets.  If it encounters an SPS with an
ID already in use and whose content differs from the previous SPS, it
throws an AVERROR_INVALIDDATA.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-04-07 10:13:45 +08:00
Andreas Rheinhardt
db063212c8 avcodec/vvc: Rename vvc_?foo->foo
A namespace is unnecessary here given that all these files
are already in the vvc subfolder.

Reviewed-by: Nuo Mi <nuomi2021@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2024-04-04 16:45:00 +02:00
Andreas Rheinhardt
9309b5bc64 avcodec/vvc/vvc_inter_template: Fix left shift of negative number
Affected the vvc-conformance-WP_A_3 FATE test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2024-04-04 11:36:41 +08:00
James Almer
45b56455ad avcodec/vvc_refs: don't ask for a "Inter layer ref" sample
The FATE suite has two already.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-04-02 11:48:32 -03:00
Nuo Mi
238bb653e7 avcodec/vvcdec: inter prediction, support subpicture
passed files:
    LMCS_B_Dolby_2.bit
    CodingToolsSets_E_Tencent_1.bit
    SUBPIC_A_HUAWEI_3.bit
    SUBPIC_B_HUAWEI_3.bit
    SUBPIC_C_ERICSSON_1.bit
    SUBPIC_D_ERICSSON_1.bit
    SUBPIC_E_MediaTek_1.bit

passed dvb conformance files (https://dvb.org/specifications/verification-validation/vvc-test-content):
    VVC_HDR_UHDTV1_OpenGOP_3840x2160_50fps_HLG10_mosaic.bit
    VVC_HDR_UHDTV1_OpenGOP_3840x2160_50fps_HLG10_PiP.bit
2024-04-02 20:56:22 +08:00
Nuo Mi
9bc3f3e5fc avcodec/vvcdec: mvs, support subpicture 2024-04-02 20:56:22 +08:00
Nuo Mi
adeb51c30f avcodec/vvcdec: alf, support subpicture 2024-04-02 20:56:22 +08:00
Nuo Mi
bbf0ccb8e7 avcodec/vvcdec: sao, support subpicture 2024-04-02 20:56:22 +08:00
Nuo Mi
0c3018b30a avcodec/vvcdec: sao, refact out tile_edge arrays 2024-04-02 20:56:22 +08:00
Nuo Mi
c9e75393ed avcodec/vvcdec: refact, movie the lc->sc assignment to task_run_stage to simplify the code
This change also make the lc->sc assigned for run_sao
2024-04-02 20:56:22 +08:00
Nuo Mi
8b7304247a avcodec/vvcdec: deblock, support subpicture 2024-04-02 20:56:22 +08:00
Nuo Mi
0d12e9c3c8 avcodec/vvcdec: refact out deblock_is_boundary 2024-04-02 20:56:22 +08:00
Nuo Mi
4020e68d73 avcodec/vvcdec: misc, rename x_ctb, y_ctb, ctu_x, ctu_y to rx, ry to avoid misleading 2024-04-02 20:56:22 +08:00
Nuo Mi
260130aae8 avcodec/vvcdec: ff_vvc_decode_neighbour, support subpicture 2024-04-02 20:56:22 +08:00
Nuo Mi
4e47847119 avcodec/vvcdec: derive subpic position for PPS 2024-04-02 20:56:03 +08:00
Frank Plowman
8078a0b0fa avcodec/vvcdec: support rectangular single-slice subpics
Co-authored-by: Nuo Mi <nuomi2021@gmail.com>
2024-04-02 20:24:31 +08:00
Frank Plowman
9c845e9087 avcodec/vvcdec: fix uninitialized last element of xxx_bd and ctb_to_xxx_bd arrays
see "6.5.1 CTB raster scanning, tile scanning, and subpicture scanning processes"

Co-authored-by: Nuo Mi <nuomi2021@gmail.com>
2024-04-02 20:24:31 +08:00
Nuo Mi
64d5cc67cd avcodec/vvcdec: misc, add specification name for pps members 2024-04-02 20:24:31 +08:00
Nuo Mi
cd540a333e avcodec/vvcdec: NoBackwardPredFlag, only check active pictures
see "8.3.6 Decoding process for collocated picture and no backward prediction"
2024-04-02 20:24:31 +08:00