This was calling atoi() on `p + offset`, which is nonsense (p may point to
the start of the cache-control string, which does not necessarilly coincide
with the location of the max-age value). Based on the code, the intent
was clearly to parse the value *after* the matched substring.
The retry path restores this offset, but the failure path does not. This
is especially important for the case of the continuation handler in
http_read_stream(), which may result in subsequent loop iterations (after
repeated failures to read additional data) seeking to the wrong offset.
This (arbitrarily) returns -1, which happens to be AVERROR(EPERM) on my
machine. Return the more descriptive AVERORR(EIO) instead.
Also add a log message to explain what's going on.
It has come to my attention that a way to limit the request range size
would be useful in general, for reasons beyond just speeding up initial
header parsing.
This patch generalizez -initial_request_size to -request_size. I decided
to continue allowing both options to be used simultaneously, so users can
e.g. set -request_size to something large like 10 MiB, but still use a smaller
size for initial header parsing (e.g. 256 KiB).
Fixes: https://github.com/mpv-player/mpv/issues/8655
Sometimes, HTTP sources require a lot of seeking during probing / header
parsing (especially for formats like MXF). Currently, we need to completely
tear down and re-establish the connection most times this happens, which puts
a lot of stress on the network stack and also results in transmission of
possibly many unnecessary bytes.
This patch adds an option to allow FFmpeg to request partial ranges during
the initialization stage. This is done until the initial request size is fully
read, after which we fall back to the normal behavior (i.e. infinite streaming
via an unbounded request).
The usefulness of this is limited without also specifying -multiple_requests 1,
since otherwise there is little point to requesting partial ranges to begin
with. (However, it is semantically independent, so we keep it that way.)
When the previous reply was a partial response (e.g. due to a seek to the
end of the file), and the remaining data from that partial response is
below the short seek size threshold, we can serve this seek by just draining
that data and re-using the existing connection.
This can currently only happen when using keep-alive connections
(-multiple_requests 1) and seeking from the end of the file to somewhere
else, in which case the file's tail can be drained and the connection re-used.
Under other circumstances, however, we still need to force a reconnection,
because we do not yet send partial range requests. (This will be changed in the
following commit)
We need to take special care not to regress the existing fallback logic
for when `http_open_cnx` fails, so here is a quick case analysis:
non-drain path:
- identical to the current
soft drain fails: (ffurl_read error path)
- s->hd = old_hd = NULL
- http_open_cnx() always opens a new connection
- on failure, old buffer is restored and s->hd remains NULL
soft drain succeeds, http_open_cnx() fails:
- s->hd is set to NULL by http_open_cnx() failure path
- old_hd was never set, so remains NULL
- old buffer is restored, s->hd remains NULL
In either case, the outcome that any (previously valid) buffer is left as-is,
the offset is unchanged, and the connection ends up closed (s->hd == NULL).
This is okay to do after the previous change to http_buf_read, which allows
it to internally re-open the connection if needed.
If the Content-Range indicates a smaller range than what we expected,
we should send a new request for the remainder before attempting to read
more.
Again, this commit is theoretically non-functional on its own, since any
conforming HTTP server should give us the entire range we asked for in the
first place, but it is semantically independent from and prepares us for the
following changes.
This could conceivably happen currently if the user tries reading more
bytes after the last chunk has already been received. In this case,
we currently segfault - but simply returning AVERROR(EIO) seems more
reasonable and lets the higher end retry the connection in this case.
In the event that the range returned is smaller than the true filesize, we
should only expect to receive that many bytes - not the entire rest of the
file.
This commit is theoretically non-functional on its own, since any conforming
HTTP server will always return us the full file range, but I wanted to split
it off from the subsequent changes in order to make review easier.
This fails to consider the case of whence == SEEK_END and the resulting
offset happening to exactly match the current position.
Reorder the check to compute the target position first, then compare.
Possible since 222127418b.
Reviewed-by: Kacper Michajłow <kasper93@gmail.com>
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Fixes: out of array access
Fixes: zeropath/off-by-one-one-byte
Found-by: Joshua Rogers <joshua@joshua.hu>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
When using a literal IPv6 address as hostname, it can contain a Zone ID
especially in the case of link-local addresses. Sending this to the
server in the Host header is not useful to the server and in some cases
servers refuse such requests.
To prevent any such issues, strip the Zone ID from the address if it's
an IPv6 address. This also removes it for the Cookies lookup.
Based on a patch by: Daniel N Pettersson <danielnp@axis.com>
0. WHIP Version 3.
1. The WHIP muxer has been renamed and refined,
with improved logging context and error messages for SSL, DTLS, and RTC.
2. Magic numbers have been replaced with macros and extracted to functions,
and log levels have been altered for better clarity.
3. DTLS curve list has been updated,
and SRTP profile names have been refined for FFmpeg and OpenSSL.
4. ICE STUN magic number has been refined,
and RTP payload types have been updated based on Chrome's definition.
5. Fixed frame size has been refined to rtc->audio_par->frame_size,
and h264_mp4toannexb is now used to convert MP4/ISOM to annexb.
6. OPUS timestamp issue has been addressed,
and marker setting has been corrected after utilizing BSF.
7. DTLS handshake and ICE handling have been optimized for improved performance,
with a single handshake timeout and server role to prevent ARQ.
8. Consolidated ICE request/response handling and DTLS handshake into a single function,
and fixed OpenSSL build errors to work with Pion.
9. Merge TLS & DTLS implementation, shared BIO callbacks, read, write,
print_ssl_error, openssl_init_ca_key_cert,
init_bio_method function and shared same data structure
10. Modify configure that whip is enabled only dtls is
enabled(just support openssl for now) to fix build error
Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: yangrtc <yangrtc@aliyun.com>
Co-authored-by: cloudwebrtc <duanweiwei1982@gmail.com>
Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: Steven Liu <lq@chinaffmpeg.org>
Co-authored-by: Jun Zhao <barryjzhao@tencent.com>
Signed-off-by: Jack Lau <jacklau1222@qq.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
This can happen for HLS with AES-128 at the middle of m3u8, so old
protocol is https while new protocol is crypt+https.
And change the log level from ERROR to INFO when protocol/host/port
don't match. User should prepared for this function to fail and
retry with new connection.
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Currently, a prematurely broken connection normally leads to the same
EOF, as a completed successful transfer. However, enabling reconnect
changes this logic, and leads to the return of EIO.
This patch unifies that logic, leading to the return of EIO for premature
disconnect, regardless of setting of "reconnect".
429 and 503 codes can, and often do (e.g. all Google Cloud
Storage URLs can), return a Retry-After header with the error,
indicating how long to wait, asd either a date, or in seconds,
before retrying again. If it is not respected by, for example,
using our default backoff stratetgy instead, chances of success
are very unlikely.
Some references:
* https://datatracker.ietf.org/doc/html/rfc6585
* https://datatracker.ietf.org/doc/html/rfc7231#section-7.1.3
This adds an AVOption to respect that header.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
That is what it actually does, and it will be needed for more
than the Expiry header soon.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
The existing option only allows users to set the max delay for a
single attempt, rather than the total allowed delay, which is both
pretty unintitive, and only applicable when exponential backoff is
used.
The default for this option is set to 256, which is just above the
effective total delay accomplished by the the existing
reconnect_delay_max default of 120.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Many "bad" HTTP codes like 429 and 503 may include important info in
their headers.
Also, in general, there is no purpose in bailing here.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
There are lots of files that don't need it: The number of object
files that actually need it went down from 2011 to 884 here.
Keep it for external users in order to not cause breakages.
Also improve the other headers a bit while just at it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Makes it robust against adding fields before it, which will be useful in
following commits.
Majority of the patch generated by the following Coccinelle script:
@@
typedef AVOption;
identifier arr_name;
initializer list il;
initializer list[8] il1;
expression tail;
@@
AVOption arr_name[] = { il, { il1,
- tail
+ .unit = tail
}, ... };
with some manual changes, as the script:
* has trouble with options defined inside macros
* sometimes does not handle options under an #else branch
* sometimes swallows whitespace
Unnecessary since acf63d5350;
also avoids relocations.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
AVCodec is only ever used as an incomplete type (i.e. via a pointer
to an AVCodec) in avformat.h and it is not really part of the core
of avformat.h or libavformat; almost none of our internal users
make use of it (and none make use of hwcontext.h, which is implicitly
included). So switch to use struct AVCodec, but continue to include
codec.h for external users for compatibility.
Also, do the same for AVFrame and frame.h, which is implicitly included
by codec.h (via lavu/hwcontext.h).
Also, remove an unnecessary inclusion of <time.h>.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Content-Type can include charset and boundary which is not a part of
mime type and shouldn't be copied as such.
Fixes HLS playback when the Content-Type includes additional fields.
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
The path attribute in the Set-Cookie header is optional but treated by ffmpeg as being compulsory.
Signed-off-by: Michael J. Walsh <mjfwalsh@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
This is needed to get LIBAVFORMAT_VERSION, used as part of the user agent.
Fixes a recent regression.
Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: James Almer <jamrial@gmail.com>
1. getenv() is replaced with getenv_utf8() across libavformat.
2. New versions of AviSynth+ are now called with UTF-8 filenames.
3. Old versions of AviSynth are still using ANSI strings,
but MAX_PATH limit on filename is removed.
Signed-off-by: Martin Storsjö <martin@martin.st>
This avoids unnecessary rebuilds of most source files if only the
list of enabled components has changed, but not the other properties
of the build, set in config.h.
Signed-off-by: Martin Storsjö <martin@martin.st>
Using Range allows for getting the full file size from the
Content-Range header in the response, even if the server sends
back the response using chunked Transfer-Encoding, which does not
allow using Content-Length.
When Transfer-Encoding:chunked is used, the client must ignore a
Content-Length header, if present. However, it should not ignore a
Content-Range header, which also includes the full size of the
entity.
av_dict_set() with AV_DICT_DONT_STRDUP_VAL takes ownership
of the string it is passed to as val; this includes freeing it
on error.
Fixes Coverity issue #1497468.
Reviewed-by: Eran Kornblau <eran.kornblau@kaltura.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
