The current logic for detecting frames that are too small for the
algorithm does not account for chroma sub-sampling, and so a sample
where the luma plane is large enough, but the chroma planes are not
will not be rejected. In that event, a heap overflow will occur.
This change adjusts the logic to consider the chroma planes and makes
the change to all three bwdif implementations.
Fixes#10688
Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at>
Reviewed-by: Thomas Mundt <tmundt75@gmail.com>
Signed-off-by: Philip Langdale <philipl@overt.org>
(cherry picked from commit 737ede405b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1500281 Out-of-bounds write
Fixes: CID1500331 Out-of-bounds write
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5fe8bf4aa5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1551694 Use after free (false positive based on assuming that out == in and one is freed and one used)
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c296d4fdec)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1403227 Division or modulo by float zero
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25cb66369e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3d5f03bbc8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID 1403229 Dereference after null check
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 98ae1ad7cf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6c50482951)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bb04235d72)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access:
Fixes: tickets/10745/poc12ffmpeg
Found-by: Li Zeyuan and Zeng Yunxiang.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7deaca71b3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9f4c5bd7d2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9d1ba698d2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: tickets/10744/poc11ffmpeg
Found-by: Li Zeyuan and Zeng Yunxiang.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a88b06f9ee)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: tickets/10746/poc13ffmpeg
Found-by: Zeng Yunxiang
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 50f0f8c53c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The code works in steps of 2 lines and lacks support for odd height
Implementing odd height support is better but for now this fixes the
out of array access
Fixes: out of array access
Fixes: tickets/10702/poc6ffmpe
Found-by: Zeng Yunxiang
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e4d2666bdc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1403233
The second of the 2 changes may be unneeded but will help coverity
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dd6040675e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fbd22504c4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: issues with non trivial linesize
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d353909e77)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d740782701)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -1
Fixes: Ticket8270
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 06af6e101b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: floating point division by 0
Fixes: Ticket 8269
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a3917c02c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: floating point division by 0
Fixes: undefined behavior in handling NaN
Fixes: Ticket 8268
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3d500e62f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: Ticket8240
Fixes: CVE-2020-22021
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7971f62120)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bb08ee0c6fb7bdebd37cbf00aefed206909e8f78)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Found-by: Jeremy Leconte <jleconte@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1cf96ce269)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr,
filter-3xbr, filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4294dc3589)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fa21194326)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: signed integer overflow: 1562273630 * 17 cannot be represented in type 'int'
Fixes: Ticket8323
Found-by: Suhwan
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c0ca0f244)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
A score of 0 is possible
Fixes: Ticket8500
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dfc4714886)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
score could be 1.0 which lead to uninitialized values
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6ff2474e02)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
With floats we cannot represent all 32bit integer dimensions
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8813b1a98)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
