Commit graph

49203 commits

Author SHA1 Message Date
Clément Bœsch
1574475033 avcodec/proresenc_anatoliy: rework encode_codeword() prototype
This matches the function of the same name in proresenc_kostya.
2024-01-10 14:08:00 +01:00
Clément Bœsch
1832bd7838 avcodec/proresenc_anatoliy: shuffle encode_codeword() code to match Kostya encoder
Code is functionally identical, it's just rename of variables, cosmetics
and branch logic shuffling.
2024-01-10 14:08:00 +01:00
Clément Bœsch
3885d2493d avcodec/proresenc_anatoliy: use FRAME_ID defined in proresdata.h 2024-01-10 14:08:00 +01:00
Clément Bœsch
d6e0fb7c92 avcodec/proresenc_kostya: simplify quantization matrix bytestream writing 2024-01-10 14:08:00 +01:00
Clément Bœsch
cbee015867 avcodec/proresenc_kostya: fix chroma quantisation matrix in frame header
Most of the time the quantisation matrices are the same, it only matters
with the proxy profile.
2024-01-10 14:08:00 +01:00
Clément Bœsch
631fa19ee0 avcodec/proresenc_kostya: save a few operations in DC encoding
This matches the logic from proresenc_anatoliy.
2024-01-10 14:08:00 +01:00
Clément Bœsch
f06f2cf16a avcodec/proresenc_anatoliy: move DC codebook LUT to shared proresdata
This is going to be shared with proresenc_kostya in the upcoming commit.
2024-01-10 14:08:00 +01:00
Clément Bœsch
9f547e2f15 avcodec/proresenc_anatoliy: remove duplicated define
This is already defined in proresdata.h
2024-01-10 14:08:00 +01:00
Clément Bœsch
c35733006a avcodec/proresenc_kostya: remove one LUT indirection for run/level to codebook mapping
This is following the same logic as proresenc_anatoliy.
2024-01-10 14:08:00 +01:00
Clément Bœsch
3ba52f18e4 avcodec/proresenc_anatoliy: move run/lev to codebook LUT to shared proresdata
This is going to be shared with proresenc_kostya in the upcoming commit.
2024-01-10 14:08:00 +01:00
Clément Bœsch
e940baa65b avcodec/proresenc_kostya: remove redundant codebook assignments
This is already assigned at declaration.
2024-01-10 14:08:00 +01:00
Clément Bœsch
e453efcfbc avcodec/proresenc_kostya: remove unused plane factor variables 2024-01-10 14:08:00 +01:00
Clément Bœsch
2ac88c1362 avcodec/proresenc_kostya: remove an unnecessary parenthesis level in MAKE_CODE() macro 2024-01-10 14:08:00 +01:00
Marton Balint
363b3ec98a all: use av_channel_layout_describe_bprint instead of av_channel_layout_describe in a few places
Where an AVBPrint buffer is used later anyway.

Signed-off-by: Marton Balint <cus@passwd.hu>
2024-01-07 22:47:22 +01:00
James Almer
b95ccfcada avcodec/vvc_thread: don't use an anonymous union
Should fix compilation with old GCC.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-01-06 23:28:03 -03:00
Nuo Mi
02d600c568 vvcdec: add TODO for combining transform, lmcs_scale_chroma, and add_residual
Thanks for the suggestion from Lynne.
2024-01-07 09:01:04 +08:00
Nuo Mi
26769024d1 avcodec/vvcdec: decode extradata to support container formats
For example:
wget https://www.elecard.com/storage/video/NovosobornayaSquare_1920x1080.mp4
./ffplay NovosobornayaSquare_1920x1080.mp4
2024-01-07 08:58:43 +08:00
Sam James
2f24f10d9c libavcodec: fix -Wint-conversion in vulkan
FIx warnings (soon to be errors in GCC 14, already so in Clang 15):
```
src/libavcodec/vulkan_av1.c: In function ‘vk_av1_create_params’:
src/libavcodec/vulkan_av1.c:183:43: error: initialization of ‘long long unsigned int’ from ‘void *’ makes integer from pointer without a cast [-Wint-conversion]
  183 |         .videoSessionParametersTemplate = NULL,
      |                                           ^~~~
src/libavcodec/vulkan_av1.c:183:43: note: (near initialization for ‘(anonymous).videoSessionParametersTemplate’)
```

Use Vulkan's VK_NULL_HANDLE instead of bare NULL.

Fix Trac ticket #10724.

Was reported downstream in Gentoo at https://bugs.gentoo.org/919067.

Signed-off-by: Sam James <sam@gentoo.org>
2024-01-06 22:38:55 +01:00
Clément Bœsch
9109273e3b avcodec/proresenc: fix alpha plane encoding bitstream
These functions encode a slice of alpha (1 to 8 macroblocks) which are
expected to be encoded as a repeated sequence of "[diff][run-1]", where
diff is the running difference of the alpha value and run is how many
times that value is expected to be duplicated (within the limit of a
grand total of 2048 unpacked samples, corresponding to a slice of 8 MB).

Even when run==0 (the run variable semantic is actually "run minus 1"),
there is always a diff previously encoded that needs a counter of at
least 1. This means we need to call put_alpha_run() unconditionally at
the end of the bitstream to account for the last running diff.

This commit fixes glitchy playbacks on QuickTime with M2 and M3 hardware
(but not M1 for some mysterious reason) with files generated with
commands such as:

  ffmpeg -f lavfi -i testsrc2=d=5:s=912x320,chromakey -c:v prores_aw -profile:v 4    -y aw.mov
  ffmpeg -f lavfi -i testsrc2=d=5:s=912x320,chromakey -c:v prores_ks -profile:v 4444 -y ks.mov

The glitch expresses itself deterministically as blinking black
rectangles on random frames (for example on frame 21, 54, 71, 79, ...).

Even with the proresdec from FFmpeg, overreads actually happens while
reading the run-minus-1 value (around val = get_bits(gb, 4) in
unpack_alpha()). This doesn't seem to cause any particular issue because
it simply overreads into the next slice, and because the decoder is
resilient, but it's still a problem.

The investigation leading to this fix was made possible because of paid
work for Jitter (https://jitter.video).

Fixes ticket #10255.
2024-01-06 17:29:59 +01:00
Clément Bœsch
2142141a16 avcodec/proresenc: make transparency honored in mov/QT
In the mov muxer (in mov_write_video_tag()), bits_per_coded_sample will
be written under certain conditions and is required to be 32 for the
transparency to be honored in QuickTime.

prores_kostya already has this setting but prores_anatoliy and
prores_videotoolbox didn't.
2024-01-06 17:29:59 +01:00
Wu Jianhua
94949d4770 avcodec/d3d12va_decode: don't change the resource state if the referenced frame is the same as the current frame
This commit removes the follow warning and error:

D3D12 WARNING: ID3D12CommandList::ResourceBarrier: Called on the same subresource(s) of
Resource(0x000002236E0E00D0:'Unnamed ID3D12Resource Object') in separate Barrier Descs
which is inefficient and likely unintentional. Desc[0] and Desc[1] on (subresource :
4294967295). [RESOURCE_MANIPULATION WARNING #1008: RESOURCE_BARRIER_DUPLICATE_SUBRESOURCE_TRANSITIONS]

D3D12 ERROR: ID3D12CommandList::ResourceBarrier: Before state (0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT])
of resource (0x000002236E0E00D0:'Unnamed ID3D12Resource Object') (subresource: 0) specified
by transition barrier does not match with the state (0x20000: D3D12_RESOURCE_STATE_VIDEO_DECODE_WRITE)
specified in the previous call to ResourceBarrier [RESOURCE_MANIPULATION ERROR #527:
RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]

Tested-by: Tong Wu <tong1.wu@intel.com>
Signed-off-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-05 11:08:17 +08:00
Tong Wu
270cd14bbb avcodec/dxva2(h264|mpeg2|vc1): use av_assert0 instead of assert
Signed-off-by: Tong Wu <tong1.wu@intel.com>
2024-01-05 11:06:57 +08:00
Tong Wu
0f01581ccd avcodec/d3d12va_decode|dxva2: add a warning to replace assertion
Previous assertion was not useful. Now a warning is added to replace it.
For get_surface_index, we should return a zero index in case the index is not found.
But a warning is necessary to notify.

Signed-off-by: Tong Wu <tong1.wu@intel.com>
2024-01-05 11:06:57 +08:00
Tong Wu
56c671c3b0 avcodec/d3d12va_h264: replace assert with av_assert0
Signed-off-by: Tong Wu <tong1.wu@intel.com>
2024-01-05 11:06:57 +08:00
Tong Wu
83e0fcbe03 avcodec/d3d12va_decode: delete an empty line and fix a fuction alignment
Signed-off-by: Tong Wu <tong1.wu@intel.com>
2024-01-05 11:06:57 +08:00
Tong Wu
d18ed2ddb5 avcodec/d3d12va_vp9: fix vp9 max_num_refs value
Previous max_num_refs was based on pp.frame_refs plus 1 and it could possibly
reaches the size limit. Actually it should be the size of pp.ref_frame_map
plus 1.

Signed-off-by: Tong Wu <tong1.wu@intel.com>
2024-01-05 11:06:57 +08:00
Zhao Zhili
33698ef891 avcodec/mpegutils: print axis in debug_info2
For example,

./ffmpeg -nostats -threads 1 -debug qp \
	-export_side_data +venc_params \
	-i reinit-small_420_9-to-small_420_8.h264 \
	-frames 2 \
	-f null -

New frame, type: B
    0       64      128     192
  0 313131313131313131313131313129
 16 292929292929292929292929292929
 32 323232323232323232323232323232
 48 323232323232323232323232323232
 64 323232323232323232323232323232
 80 323232323232323232323232323232
 96 323232323030303030303030303030
112 303030303030303030303030303030
128 303030303030303030303030303028
144 313131312929292929292929292929
160 292929292929292929292929292929
176 292929292929292929292929292931
192 312831312631313131312730283131

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-01-04 17:35:11 +08:00
Zhao Zhili
bd48c08f80 avcodec/mpegutils: make debug_info2 thread safe
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-01-04 17:33:56 +08:00
Zhao Zhili
7f900a737f avcodec/videotoolbox: specify color range for hw frame ctx
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-01-04 17:33:38 +08:00
Zhao Zhili
0f824d792d avcodec/hevc_parser: fix missing zero_byte at frame beginning
The start code is matched against 0x000001, zero_byte was treated
as last byte of last frame rather than the beginning of next frame.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-01-05 01:14:33 +08:00
Zhao Zhili
b7ac1f9856 avcodec/hevc_mp4toannexb_bsf: use HEVCNALUnitType instead of integer literal
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2024-01-05 01:14:33 +08:00
Nuo Mi
301ed950d1 vvcdec: add vvc decoder
vvc decoder plug-in to avcodec.
split frames into slices/tiles and send them to vvc_thread for further decoding
reorder and wait for the frame decoding to be done and output the frame

Features:
    + Support I, P, B frames
    + Support 8/10/12 bits, chroma 400, 420, 422, and 444 and range extension
    + Support VVC new tools like MIP, CCLM, AFFINE, GPM, DMVR, PROF, BDOF, LMCS, ALF
    + 295 conformace clips passed
    - Not support RPR, IBC, PALETTE, and other minor features yet

Performance:
    C code FPS on an i7-12700K (x86):
        BQTerrace_1920x1080_60_10_420_22_RA.vvc      93.0
        Chimera_8bit_1080P_1000_frames.vvc          184.3
        NovosobornayaSquare_1920x1080.bin           191.3
        RitualDance_1920x1080_60_10_420_32_LD.266   150.7
        RitualDance_1920x1080_60_10_420_37_RA.266   170.0
        Tango2_3840x2160_60_10_420_27_LD.266         33.7

    C code FPS on a M1 Mac Pro (ARM):
        BQTerrace_1920x1080_60_10_420_22_RA.vvc     58.7
        Chimera_8bit_1080P_1000_frames.vvc          153.3
        NovosobornayaSquare_1920x1080.bin           150.3
        RitualDance_1920x1080_60_10_420_32_LD.266   105.0
        RitualDance_1920x1080_60_10_420_37_RA.266   133.0
        Tango2_3840x2160_60_10_420_27_LD.266        21.7

    Asm optimizations still working in progress. please check
    https://github.com/ffvvc/FFmpeg/wiki#performance-data for the latest

Contributors (based on code merge order):
    Nuo Mi <nuomi2021@gmail.com>
    Xu Mu <toxumu@outlook.com>
    Frank Plowman <post@frankplowman.com>
    Shaun Loo <shaunloo10@gmail.com>
    Wu Jianhua <toqsxw@outlook.com>

Thank you for reporting issues and providing performance reports:
    Łukasz Czech <lukaszcz18@wp.pl>
    Xu Fulong <839789740@qq.com>

Thank you for providing review comments:
    Ronald S. Bultje <rsbultje@gmail.com>
    James Almer <jamrial@gmail.com>
    Andreas Rheinhardt <andreas.rheinhardt@outlook.com>

Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:12 +08:00
Nuo Mi
e7ef457d6b vvcdec: add CTU thread logical
This is the main entry point for the CTU (Coding Tree Unit) decoder.
The code will divide the CTU decoder into several stages.
It will check the stage dependencies and run the stage decoder.

Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:12 +08:00
Nuo Mi
07f75d5e02 vvcdec: add CTU parser
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:12 +08:00
Nuo Mi
b49575f4cf vvcdec: add dsp init and inv transform
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:12 +08:00
Nuo Mi
02c1455b44 vvcdec: add LMCS, Deblocking, SAO, and ALF filters
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:12 +08:00
Nuo Mi
c05ba94ce8 vvcdec: add intra prediction
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:12 +08:00
Nuo Mi
2592cc1f96 vvcdec: add inv transform 1d
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:11 +08:00
Nuo Mi
ea49c83bad vvcdec: add inter prediction
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:11 +08:00
Nuo Mi
603d0bd171 vvcdec: add motion vector decoder
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:11 +08:00
Nuo Mi
c1a3d17491 vvcdec: add reference management
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:11 +08:00
Nuo Mi
976d3b7d69 vvcdec: add cabac decoder
add Context-based Adaptive Binary Arithmetic Coding (CABAC) decoder

Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 23:15:06 +08:00
Nuo Mi
e97a5bbb13 vvcdec: add parameter parser for sps, pps, ph, sh
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 16:31:59 +08:00
Nuo Mi
49db9fc171 vvcdec: add vvc_data
Co-authored-by: Xu Mu <toxumu@outlook.com>
Co-authored-by: Frank Plowman <post@frankplowman.com>
Co-authored-by: Shaun Loo <shaunloo10@gmail.com>
Co-authored-by: Wu Jianhua <toqsxw@outlook.com>
2024-01-03 16:31:59 +08:00
James Almer
85b8d59ec7 avcodec/d3d12va_mpeg2: change the type for the ID3D12Resource_Map input data argument
Fixes -Wincompatible-pointer-types warnings.

Reviewed-by: Wu Jianhua <toqsxw@outlook.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2024-01-01 09:46:41 -03:00
James Almer
e9722735fa avcodec/d3d12va_mpeg2: remove unused variables
Reviewed-by: Wu Jianhua <toqsxw@outlook.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2024-01-01 09:46:06 -03:00
Michael Niedermayer
e063c1d079
avcodec/mpegvideo_enc: Use ptrdiff_t for stride
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-12-30 21:50:05 +01:00
Michael Niedermayer
a066b8a809
avcodec/mpegvideo_enc: Dont copy beyond the image
Fixes: out of array access
Fixes: tickets/10754/poc17ffmpeg

Discovered by Zeng Yunxiang.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-12-30 21:50:04 +01:00
Michael Niedermayer
bf1159774b
avcodec/vaapi_encode: Avoid double AVERRORS
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-12-29 21:36:02 +01:00
Michael Niedermayer
850ab8f6da
avcodec/jpegxl_parser: Check get_vlc2()
Fixes: shift exponent -1 is negative
Fixes: 63889/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-6009343056936960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-12-29 19:21:26 +01:00