Commit graph

167 commits

Author SHA1 Message Date
Wu Jianhua
ca5c9e810a avcodec/vvc/dsp: prefix TxType and TxSize with VVC
See https://patchwork.ffmpeg.org/project/ffmpeg/patch/TYSPR06MB64337C4A9ADF5312E6648543AA62A@TYSPR06MB6433.apcprd06.prod.outlook.com/#81892

Signed-off-by: Wu Jianhua <toqsxw@outlook.com>
2024-08-15 20:52:14 +08:00
Wu Jianhua
ae1a9cfd52 avcodec/vvc_parser: move avctx->has_b_frames initialization to dec
From Jun Zhao <mypopydev@gmail.com>:
> Should we relocate this to the decoder? Other codecs typically set this
> parameter in the decoder.

Signed-off-by: Wu Jianhua <toqsxw@outlook.com>
2024-08-15 20:50:24 +08:00
Nuo Mi
80af195804 avcodec/vvcdec: move frame tab memset from the main thread to worker threads
memset tables in the main thread can become a bottleneck for the decoder.
For example, if it takes 1% of the processing time for one core, the maximum achievable FPS will be 100.
Move the memeset to worker threads will fix the issue.
2024-08-15 20:33:57 +08:00
Nuo Mi
daf6fcd816 avcodec/vvcdec: do not zero frame qp table
For luma, qp can only change at the CU level, so the qp tab size is related to the CU.
For chroma, considering the joint CbCr, the QP tab size is related to the TU.
2024-08-15 20:33:57 +08:00
Nuo Mi
ca2caeb21d avcodec/vvcdec: do not zero frame msf mmi table 2024-08-15 20:33:57 +08:00
Nuo Mi
0c7106f618 avcodec/vvcdec: do not zero frame cpm table 2024-08-15 20:33:57 +08:00
Nuo Mi
3e497249eb avcodec/vvcdec: check_available, use && instead of &= for shortcut evaluation 2024-08-15 20:33:57 +08:00
Nuo Mi
4f882c453e avcodec/vvcdec: do not zero frame mvf table 2024-08-15 20:33:57 +08:00
Nuo Mi
93b0d5f9ce avcodec/vvcdec: refact out is_available from is_a0_available 2024-08-15 20:33:57 +08:00
Nuo Mi
e8ac761293 avcodec/vvcdec: split ctu table to zero init and no zero init parts
cus need to init to zero, other parts are not
2024-08-15 20:19:45 +08:00
Nuo Mi
4f60961498 avcodec/vvcdec: remove unnecessary perframe initializations
deblock, sao, alf
skip, imtf, ipm, cqt_depth, cb_pos_x, cb_pos_y, cb_height, cp_mv,
tb_pos_x0, tb_pos_y0, tb_width, tb_height
2024-08-15 20:19:45 +08:00
Nuo Mi
210bf1b154 avcodec/vvcdec: refact, combine bs tab with tu tab 2024-08-15 20:19:45 +08:00
Nuo Mi
bdb79fe60a avcodec/vvcdec: thread, ensure the parse stage gets the highest priority
The parser stage is not parallelizable.
We need to schedule it as soon as possible to create later stages, which are more parallelizable

clips                                       | before | after | delta
--------------------------------------------|--------|-------|------
RitualDance_1920x1080_60_10_420_37_RA.266   | 342.7  | 365.3 |  6.59%
NovosobornayaSquare_1920x1080.bin           | 321.7  | 400   | 24.34%
Tango2_3840x2160_60_10_420_27_LD.266        |  82.3  |  91.7 | 11.42%
RitualDance_1920x1080_60_10_420_32_LD.266   | 323.7  | 319.3 | -1.36%
Chimera_8bit_1080P_1000_frames.vvc          | 364    | 411.3 | 12.99%
BQTerrace_1920x1080_60_10_420_22_RA.vvc     | 162.7  | 185.7 | 14.14%
2024-08-15 20:19:45 +08:00
Nuo Mi
1c17520e79 avcodec/vvcdec: Use av_image_copy_plane for DMVR 10-bit integer pixels
It's no need to shift and interpolate for 10-bit integer pixels,
av_image_copy_plane is enough
2024-08-15 20:19:45 +08:00
Zhao Zhili
2d4ef304c9 avcodec/vvc: Add aarch64 neon optimization for ALF
vvc_alf_filter_chroma_4x4_8_c: 3.0
vvc_alf_filter_chroma_4x4_8_neon: 1.0
vvc_alf_filter_chroma_4x4_10_c: 2.7
vvc_alf_filter_chroma_4x4_10_neon: 1.0
vvc_alf_filter_chroma_4x4_12_c: 2.7
vvc_alf_filter_chroma_4x4_12_neon: 1.0
vvc_alf_filter_chroma_8x8_8_c: 10.2
vvc_alf_filter_chroma_8x8_8_neon: 3.0
vvc_alf_filter_chroma_8x8_10_c: 10.0
vvc_alf_filter_chroma_8x8_10_neon: 2.5
vvc_alf_filter_chroma_8x8_12_c: 10.0
vvc_alf_filter_chroma_8x8_12_neon: 2.5
vvc_alf_filter_chroma_16x16_8_c: 41.7
vvc_alf_filter_chroma_16x16_8_neon: 11.2
vvc_alf_filter_chroma_16x16_10_c: 39.0
vvc_alf_filter_chroma_16x16_10_neon: 10.0
vvc_alf_filter_chroma_16x16_12_c: 40.2
vvc_alf_filter_chroma_16x16_12_neon: 10.2
vvc_alf_filter_chroma_32x32_8_c: 162.0
vvc_alf_filter_chroma_32x32_8_neon: 45.0
vvc_alf_filter_chroma_32x32_10_c: 155.5
vvc_alf_filter_chroma_32x32_10_neon: 39.5
vvc_alf_filter_chroma_32x32_12_c: 155.5
vvc_alf_filter_chroma_32x32_12_neon: 40.0
vvc_alf_filter_chroma_64x64_8_c: 646.0
vvc_alf_filter_chroma_64x64_8_neon: 175.5
vvc_alf_filter_chroma_64x64_10_c: 708.2
vvc_alf_filter_chroma_64x64_10_neon: 166.7
vvc_alf_filter_chroma_64x64_12_c: 619.2
vvc_alf_filter_chroma_64x64_12_neon: 157.2
vvc_alf_filter_chroma_128x128_8_c: 2611.5
vvc_alf_filter_chroma_128x128_8_neon: 698.2
vvc_alf_filter_chroma_128x128_10_c: 2470.0
vvc_alf_filter_chroma_128x128_10_neon: 616.0
vvc_alf_filter_chroma_128x128_12_c: 2531.5
vvc_alf_filter_chroma_128x128_12_neon: 620.2
vvc_alf_filter_luma_8x8_8_c: 25.2
vvc_alf_filter_luma_8x8_8_neon: 4.2
vvc_alf_filter_luma_8x8_10_c: 18.5
vvc_alf_filter_luma_8x8_10_neon: 4.0
vvc_alf_filter_luma_8x8_12_c: 19.0
vvc_alf_filter_luma_8x8_12_neon: 4.0
vvc_alf_filter_luma_16x16_8_c: 106.5
vvc_alf_filter_luma_16x16_8_neon: 16.2
vvc_alf_filter_luma_16x16_10_c: 75.2
vvc_alf_filter_luma_16x16_10_neon: 14.7
vvc_alf_filter_luma_16x16_12_c: 79.7
vvc_alf_filter_luma_16x16_12_neon: 14.7
vvc_alf_filter_luma_32x32_8_c: 400.5
vvc_alf_filter_luma_32x32_8_neon: 63.2
vvc_alf_filter_luma_32x32_10_c: 299.2
vvc_alf_filter_luma_32x32_10_neon: 57.7
vvc_alf_filter_luma_32x32_12_c: 299.2
vvc_alf_filter_luma_32x32_12_neon: 57.7
vvc_alf_filter_luma_64x64_8_c: 1602.5
vvc_alf_filter_luma_64x64_8_neon: 251.7
vvc_alf_filter_luma_64x64_10_c: 1197.0
vvc_alf_filter_luma_64x64_10_neon: 235.5
vvc_alf_filter_luma_64x64_12_c: 1220.2
vvc_alf_filter_luma_64x64_12_neon: 235.7
vvc_alf_filter_luma_128x128_8_c: 6570.2
vvc_alf_filter_luma_128x128_8_neon: 1007.7
vvc_alf_filter_luma_128x128_10_c: 4822.7
vvc_alf_filter_luma_128x128_10_neon: 936.2
vvc_alf_filter_luma_128x128_12_c: 4791.2
vvc_alf_filter_luma_128x128_12_neon: 938.5

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-07-22 21:09:56 +08:00
Zhao Zhili
b3aeef3bf9 avcodec/vvc: Remove write-only assignments in alf_filter_chroma 2024-07-17 21:23:41 +08:00
Zhao Zhili
8bac9d4a21 avcodec/vvc: Remove NOP condition check in alf_filter_luma
If (y + i == vb_above) or (y + i == vb_below), the if body has no
operation.
2024-07-17 21:23:41 +08:00
Michael Niedermayer
8f74c313f1
avcodec/vvc/ctu: Simplify code at the end of pred_mode_decode()
This simplification assumes that the code is correct

Fixes: CID1560036 Logically dead code

Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-15 01:59:37 +02:00
Zhao Zhili
0e5f8ddc1d avcodec/vvc: Use static const for function table 2024-07-11 20:26:47 +08:00
Michael Niedermayer
eb552ecd54
avcodec/vvc/refs: Use unsigned mask
Not a bugfix, but might fix CID1604361 Overflowed constant

Sponsored-by: Sovereign Tech Fund
Reviewed-by: Nuo Mi <nuomi2021@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-10 18:10:08 +02:00
Michael Niedermayer
97ae47f9e9
avcodec/vvc/ctu: Remove dead ret check
Fixes: CID1560040 Logically dead code

Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-02 21:57:21 +02:00
Michael Niedermayer
d22a33710a
avcodec/vvc/dec: Remove constant eos_at_start
Fixes: CID1560041 'Constant' variable guards dead code

Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-02 21:57:21 +02:00
Michael Niedermayer
6398242bb2
avcodec/vvc/dec: Check ff_init_cabac_decoder() for failure
Fixes: signed integer overflow: 1107820800 + 1107820800 cannot be represented in type 'int'
Fixes: left shift of 1091059712 by 6 places cannot be represented in type 'int'
Fixes: 69910/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VVC_fuzzer-5162839971528704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Nuo Mi <nuomi2021@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-02 21:57:17 +02:00
Frank Plowman
83b77990c6 lavc/vvc: Always set flags for the current picture
ff_vvc_frame_rpl uses the flags to detect whether a frame is in use.
Therefore, in the case of a CVSS AU (RASL/GDR with
NoOutputBeforeRecoveryFlag) with ph_non_ref_pic_flag = 1, the frame
would be freed before it is used.  Fix this by always marking the
current frame with VVC_FRAME_FLAG_SHORT_REF, as is done by the HEVC
decoder.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-06-28 21:37:40 +08:00
Zhao Zhili
6aeb084c39 avcodec/vvc: Don't create new thread when thread_count is 1
Make its behavior consistent with other decoders, e.g., H.264/H.265.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-06-27 20:54:42 +08:00
Frank Plowman
d79c926ab6 lavc/vvc: Validate IBC block vector
From H.266 (V3) (09/2023) p. 321:

It is a requirement of bitstream conformance that the luma block
vector bvL shall obey the following constraints:
- CtbSizeY is greater than or equal to
((yCb + (bvL[ 1 ] >> 4)) & (CtbSizeY − 1)) + cbHeight

This patch checks this is true, which fixes crashes on fuzzed
bitstreams.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-06-27 20:38:34 +08:00
Frank Plowman
c917c423e0 lavc/vvc: Don't discard return codes
Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-06-27 20:36:13 +08:00
Martin Storsjö
75e3b81f75 avcodec/vvcdec: Fix compiling with MSVC 2022 17.8 and older
Versions of MSVC older than 17.9 error out here with the following
error:

    src/libavcodec/vvc/filter.c(815): error C2059: syntax error: '}'
    src/libavcodec/vvc/filter.c(832): error C2065: 'all_zero_bs': undeclared identifier
    src/libavcodec/vvc/filter.c(836): error C2065: 'all_zero_bs': undeclared identifier

This was a regression from 5b9320b209.

Signed-off-by: Martin Storsjö <martin@martin.st>
2024-06-27 11:06:23 +03:00
Nuo Mi
23531c9776 avcodec/vvcdec: alf, support virtual boundaries
see https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9503377

passed files:
    GDR_A_ERICSSON_2.bit
    GDR_B_NOKIA_2.bit
    GDR_C_NOKIA_2.bit
    VIRTUAL_A_MediaTek_3.bit
    VIRTUAL_B_MediaTek_3.bit
2024-06-25 19:32:17 +08:00
Nuo Mi
3bac5ba861 avcodec/vvcdec: sao, support virtual boundaries 2024-06-25 19:32:17 +08:00
Nuo Mi
17c8b63781 avcodec/vvcdec: deblock, support virtual boundaries 2024-06-25 19:32:17 +08:00
Nuo Mi
80ded9cfe7 avcodec/vvcdec: ps, derive virtual boundaries 2024-06-25 19:32:17 +08:00
Nuo Mi
fc731d8b39 avcodec/vvcdec: misc, constify ALFParams 2024-06-25 19:32:17 +08:00
Nuo Mi
9b532ace02 avcodec/vvcdec: misc, remove unused ALFParams.applied 2024-06-25 19:32:17 +08:00
Nuo Mi
98b6b90601 avcodec/vvcdec: refact out alf_get_edges 2024-06-25 19:32:17 +08:00
Nuo Mi
45d0ba8313 avcodec/vvcdec: misc, reformat ff_vvc_sao_filter 2024-06-25 19:32:17 +08:00
Nuo Mi
a69b07dc79 avcodec/vvcdec: refact, fix naming convention of x0, y0 for sao
it's mismatched with the ff_vvc_sao_filter function declaration
2024-06-25 19:32:17 +08:00
Nuo Mi
ccf1d4172c avcodec/vvcdec: refact out sao_extends_edges 2024-06-25 19:32:17 +08:00
Nuo Mi
200862b144 avcodec/vvcdec: refact out sao_get_edges 2024-06-25 19:32:17 +08:00
Nuo Mi
5b9320b209 avcodec/vvcdec: refact, unify ff_vvc_deblock_{horizontal, vertical} 2024-06-25 19:32:17 +08:00
Nuo Mi
16e410aaac avcodec/vvcdec: misc, use POS to simplify filter code 2024-06-25 19:32:17 +08:00
Nuo Mi
a6ea542a34 avcodec/vvcdec: refact, unify {horizontal, vertical}_bs, {horizontal, vertical}_p, {horizontal, vertical}_q 2024-06-25 19:32:17 +08:00
Nuo Mi
cc89832d56 avcodec/vvcdec: refact, unify vvc_deblock_bs_chroma_{horizontal, vertical} 2024-06-25 19:32:17 +08:00
Nuo Mi
0a5bbcf0f3 avcodec/vvcdec: refact, unify vvc_deblock_bs_luma_{horizontal, vertical} 2024-06-25 19:32:17 +08:00
Nuo Mi
cb6538e9a1 avcodec/vvcdec: refact, unify vvc_deblock_subblock_bs_{horizontal, vertical} 2024-06-25 19:32:17 +08:00
James Almer
1b9af306da avcodec: use the renamed av_zero_extend
Signed-off-by: James Almer <jamrial@gmail.com>
2024-06-13 20:36:09 -03:00
Frank Plowman
d72a5fe719 lavc/vvc: Prevent overflow in chroma QP derivation
On the top of p. 112 in VVC (09/2023):

It is a requirement of bitstream conformance that the values of
qpInVal[ i ][ j ] and qpOutVal[ i ][ j ] shall be in the range
of −QpBdOffset to 63, inclusive for i in the range of 0 to
numQpTables − 1, inclusive, and j in the range of 0 to
sps_num_points_in_qp_table_minus1[ i ] + 1, inclusive.

Additionally, don't discard the return code from sps_chroma_qp_table.

Signed-off-by: Frank Plowman <post@frankplowman.com>
2024-06-12 20:48:08 +08:00
Andreas Rheinhardt
a0ff31e740 avcodec/vvc/inter: Don't return void
Returning a void is not allowed by the spec. Just return instead.

Reviewed-by: Nuo Mi <nuomi2021@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2024-06-11 02:43:14 +02:00
Nuo Mi
f68f40736f avcodec/vvcdec: support mv wraparound
A 360 video specific tool
see https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9503377

passed files:
    DMVR_A_Huawei_3.bit
    WRAP_D_InterDigital_4.bit
    WRAP_A_InterDigital_4.bit
    WRAP_B_InterDigital_4.bit
    WRAP_C_InterDigital_4.bit
    ERP_A_MediaTek_3.bit
2024-06-08 17:45:55 +08:00
Nuo Mi
685174069f avcodec/vvcdec: misc, reindent inter.c 2024-06-08 17:45:55 +08:00