Both AV_PIX_FMT_GRAY8 and AV_PIX_FMT_GRAY16 use unsigned values,
not signed ones. The fact that the input might be signed
in some cases in the original format doesn't change this.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The PGX decoder accesses the lines via code like
(PIXEL*)frame->data[0] + i*frame->linesize[0]/sizeof(PIXEL)
where PIXEL is a macro parameter. This code has issues with negative
linesizes, because the type of sizeof(PIXEL) is size_t, so
that on common systems i*linesize/sizeof(PIXEL) will
always be an unsigned type that is very large in case linesize is
negative. This happens to work*, but it is undefined behaviour
and e.g. leads to "src/libavcodec/pgxdec.c:114:1: runtime error:
addition of unsigned offset to 0x7efe9c2b7040 overflowed to 0x7efe9c2b6040"
errors from UBSAN.
Fix this by using (PIXEL*)(frame->data[0] + i*frame->linesize[0]).
This is allowed because linesize has to be suitably aligned.
*: Converting a negative int to size_t works by adding SIZE_MAX + 1
to the number, so that the result is off by (SIZE_MAX + 1) /
sizeof(PIXEL). Converting the pointer arithmetic (performed on PIXELs)
back to ordinary pointers is tantamount to multiplying by sizeof(PIXEL),
so that the result is off by SIZE_MAX + 1; but SIZE_MAX + 1 == 0
for the underlying pointers.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
These checks were (most likely) added to check for overreads
as the bytestream2_get_* functions return 0 in this case.
Yet this is not necessary anymore as we now have an explicit check
for the size. Should the input contain a real \0, pgx_get_number()
will error out lateron.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
We have already checked that there is data to be read.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Each of the three calls to pgx_get_number() consumes at least two bytes.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This is possible, because every given FFCodec has to implement
exactly one of these. Doing so decreases sizeof(FFCodec) and
therefore decreases the size of the binary.
Notice that in case of position-independent code the decrease
is in .data.rel.ro, so that this translates to decreased
memory consumption.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This increases type-safety by avoiding conversions from/through void*.
It also avoids the boilerplate "AVFrame *frame = data;" line
for non-subtitle decoders.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, codec.h contains both public and private parts
of AVCodec. This exposes the internals of AVCodec to users
and leads them into the temptation of actually using them
and forces us to forward-declare structures and types that
users can't use at all.
This commit changes this by adding a new structure FFCodec to
codec_internal.h that extends AVCodec, i.e. contains the public
AVCodec as first member; the private fields of AVCodec are moved
to this structure, leaving codec.h clean.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Given that the AVCodec.next pointer has now been removed, most of the
AVCodecs are not modified at all any more and can therefore be made
const (as this patch does); the only exceptions are the very few codecs
for external libraries that have a init_static_data callback.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Fixes: left shift of negative value -121
Fixes: 23911/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGX_fuzzer-4986800258154496
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This patch makes the pgx decoder select the correct
byte order instead of selecting big endian format for
16 bit images.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
