Reading from an incoming request body after the request handler aborts
with a panic can cause a panic, becuse http.Server does not (contrary
to its documentation) close the request body in this case.
Always close the incoming request body in ReverseProxy.ServeHTTP to
ensure that any in-flight outgoing requests using the body do not
read from it.
Updates #46866
Fixes CVE-2021-36221
Change-Id: I310df269200ad8732c5d9f1a2b00de68725831df
Reviewed-on: https://go-review.googlesource.com/c/go/+/333191
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
The current implementation uses UTF-aware functions
like strings.EqualFold and strings.ToLower.
This could, in some cases, cause http smuggling.
Change-Id: I0e76a993470a1e1b1b472f4b2859ea0a2b22ada0
Reviewed-on: https://go-review.googlesource.com/c/go/+/308009
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Roberto Clapis <roberto@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Finish up a prior TODO by making ReverseProxy flush immediately
if Content-Length is -1, which is a case that can occur if for
example we have a streamed response, or chunked encoding, or when
the body's length wasn't known.
Fixes#41642
Change-Id: I30babaaf3e14837b99e3ecdc562a0a0e50c579bf
GitHub-Last-Rev: efc019a9fe
GitHub-Pull-Request: golang/go#41858
Reviewed-on: https://go-review.googlesource.com/c/go/+/260637
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
When forwarding a request, a SingleHostReverseProxy appends the
request's path to the target URL's path. However, if certain path
elements are encoded, (such as %2F for slash in either the request or
target path), simply joining the URL.Path elements is not sufficient,
since the field holds the decoded path.
Since 87a605, the RawPath field was added which holds a decoding
hint for the URL. When joining URL paths, this decoding hint needs
to be taken into consideration.
As an example, if the target URL.Path is /a/b, and URL.RawPath
is /a%2Fb, joining the path with /c should result in /a/b/c
in URL.Path, and /a%2Fb/c in RawPath.
The added joinURLPath function combines the two URL's Paths,
while taking into account escaping, and replaces the previously used
singleJoiningSlash in NewSingleHostReverseProxy.
Fixes#35908
Change-Id: I45886aee548431fe4031883ab1629a41e35f1727
GitHub-Last-Rev: 7be6b8d421
GitHub-Pull-Request: golang/go#36378
Reviewed-on: https://go-review.googlesource.com/c/go/+/213257
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Ensures that a canceled client request for Switching Protocols
(e.g. h2c, Websockets) will cause the underlying connection to
be terminated.
Adds a goroutine in handleUpgradeResponse in order to select on
the incoming client request's context and appropriately cancel it.
Fixes#35559
Change-Id: I1238e18fd4cce457f034f78d9cdce0e7f93b8bf6
GitHub-Last-Rev: 3629c78493
GitHub-Pull-Request: golang/go#38021
Reviewed-on: https://go-review.googlesource.com/c/go/+/224897
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Everywhere else is using "cancellation"
The reasoning is mentioned in 170060
> Though there is variation in the spelling of canceled,
> cancellation is always spelled with a double l.
>
> Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/
Change-Id: Ifc97c6785afb401814af77c377c2e2745ce53c5a
GitHub-Last-Rev: 05edd7477d
GitHub-Pull-Request: golang/go#38662
Reviewed-on: https://go-review.googlesource.com/c/go/+/230200
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The net/url error in question now quotes the URL, so update the expected
output string.
While at it, also update a comment in httputil, though that doesn't
affect any test.
Fixes#33910.
Change-Id: I0981f528b24337c2952ef60c0db3b7ff72d72110
Reviewed-on: https://go-review.googlesource.com/c/go/+/192078
Run-TryBot: Daniel Martà <mvdan@mvdan.cc>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In Go1.12 and below, the logic in ReverseProxy.ServeHTTP would always
allocate request.Header even if it were not present in the incoming request.
CL 174324 added http.Request.Clone and re-factors ReverseProxy.ServeHTTP
to use the new Clone method. However, the new Clone logic is not equivalent
to the former logic. We preserve former semantics by explicitly allocating
the Header map if nil.
Fixes#33142
Change-Id: I356f94a915dd9779584ce3fe31e56e5474b9ad37
Reviewed-on: https://go-review.googlesource.com/c/go/+/186437
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
In the reverseproxy, replace use (Header).Get, which returns only one value
of a multiple value header, with using the Header map directly. Also fixes
corresponding tests which hid the bug, and adds more tests.
Fixes#30303
Change-Id: Ic9094b5983043460697748759f6dfd95fc111db7
GitHub-Last-Rev: b41038143f
GitHub-Pull-Request: golang/go#30687
Reviewed-on: https://go-review.googlesource.com/c/go/+/166298
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A regression was introduced in CL 137335 (5440bfc) that caused FlushInterval
to not be honored until the first Write() call was encountered. This change
starts the flush timer as part of setting up the maxLatencyWriter.
Fixes#31125Fixes#31126
Change-Id: I75325bd926652922219bd1457b2b00ac6d0d41b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/170066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix unannounced trailers when body is empty and without announced trailers.
Fixes#29031
Change-Id: If49951a42fe56d4be4436a999627db4c2678659d
GitHub-Last-Rev: 3469adc8f5
GitHub-Pull-Request: golang/go#29032
Reviewed-on: https://go-review.googlesource.com/c/151898
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
* Rewrite the flushing code to not use a persistent goroutine, which
also simplifies testing.
* Define the meaning of a negative flush interval. Its meaning doesn't
change, but now it's locked in, and then we can use it to optimize
the performance of the non-buffered case to avoid use of an AfterFunc.
* Support (internal-only) special casing of FlushInterval values per
request/response.
* For now, treat Server-Sent Event responses as unbuffered. (or rather,
immediately flushed from the buffer per-write)
Fixes#27816
Change-Id: Ie0f975c997daa3db539504137c741a96d7022665
Reviewed-on: https://go-review.googlesource.com/c/137335
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This permits specifying an ErrorHandler to customize the RoundTrip
error handling if the backend fails to return a response.
Fixes#22700Fixes#21255
Change-Id: I8879f0956e2472a07f584660afa10105ef23bf11
Reviewed-on: https://go-review.googlesource.com/77410
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Prior to the fix to #23643, the ReverseProxy didn't panic with
ErrAbortHandler when the copy to a client failed.
During Go 1.11 beta testing, we found plenty of code using
ReverseProxy in tests that were unprepared for a panic.
Change the behavior to only panic when running under the http.Server
that'll handle the panic.
Updates #23643
Change-Id: Ic1fa8405fd54c858ce8c797cec79d006833a9f7d
Reviewed-on: https://go-review.googlesource.com/122819
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For functions TestClonesRequestHeaders and TestReverseProxy_PanicBodyError,
I made changes to update the log verbosity.
Fixes#25634
Change-Id: I2a0ef70a8191cfb1a0005949345be722fb4ab62e
Reviewed-on: https://go-review.googlesource.com/115296
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously when RoundTrip returned a non-nil error, the proxy returned a
StatusBadGateway error, instead of first calling ModifyResponse. This
commit first calls ModifyResponse, whether or not the error returned
from RoundTrip is nil.
Also closes response body when ModifyResponse returns an error. See #22658.
Fixes#21255
Change-Id: I5b5bf23a69ae5608f87d4ece756a1b4985ccaa9c
Reviewed-on: https://go-review.googlesource.com/54030
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We used to do it sometimes as an optimization, but the optimization is
flawed: in all non-contrived cases we need to deep clone the map
anyway. So do it always, which both simplifies the code but also fixes
the X-Forward-For value leaking to the caller's Request, as well as
modifications from the optional Director func.
Fixes#18327
Change-Id: I0c86d10c557254bf99fdd988227dcb15f968770b
Reviewed-on: https://go-review.googlesource.com/46716
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Despite the previously known behavior of Request.WithContext
shallow copying a request, usage of the request inside server.ServeHTTP
mutates the request's URL. This CL implements deep copying of the URL.
Fixes#20068
Change-Id: I86857d7259e23ac624d196401bf12dde401c42af
Reviewed-on: https://go-review.googlesource.com/41308
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Trailers that are not announced in the Trailer must be passed on to
the downstream client.
Rather than iterate over each and find missing trailer values,
this re-adds all trailers to the headers if there is a disparity
between the number of announced trailers and the final number.
This fixes#20437
Change-Id: I867e85f45feff68616a9a9bd6f65f12d73825eb7
Reviewed-on: https://go-review.googlesource.com/43712
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
After merging https://go-review.googlesource.com/c/34639/,
it was pointed out to me that a lot of tests under net/http
could use the new functionality to simplify and unify testing.
Using the httptest.Server provided Client removes the need to
call CloseIdleConnections() on all Transports created, as it
is automatically called on the Transport associated with the
client when Server.Close() is called.
Change the transport used by the non-TLS
httptest.Server to a new *http.Transport rather than using
http.DefaultTransport implicitly. The TLS version already
used its own *http.Transport. This change is to prevent
concurrency problems with using DefaultTransport implicitly
across several httptest.Server's.
Add tests to ensure the httptest.Server.Client().Transport
RoundTripper interface is implemented by a *http.Transport,
as is now assumed across large parts of net/http tests.
Change-Id: I9f9d15f59d72893deead5678d314388718c91821
Reviewed-on: https://go-review.googlesource.com/37771
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Adds ModifyResponse, an optional func to ReverseProxy
that modifies a response in the backend, right before
the headers of the response are written to the internal
response writer.
If ModifyResponse returns an error, the proxy returns
a StatusBadGateway error.
Fixes#14237.
Change-Id: I8e03139e34dea0084512ccbd8cc49e941bf9fb5d
Reviewed-on: https://go-review.googlesource.com/32356
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Hop-by-hop headers (explicitly mentioned in RFC 2616) were already
removed from the response. This removes the custom hop-by-hop
headers listed in the "Connection" header of the response.
Updates #16875
Change-Id: I6b8f261d38b8d72040722f3ded29755ef0303427
Reviewed-on: https://go-review.googlesource.com/28810
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We were already making a copy of the map before removing
hop-by-hop headers. This commit does the same for proxied
headers mentioned in the "Connection" header.
A test is added to ensure request headers are not modified.
Updates #16875
Change-Id: I85329d212787958d5ad818915eb0538580a4653a
Reviewed-on: https://go-review.googlesource.com/28493
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
The http.Transport's retry can't retry requests with non-nil
bodies. When cloning an incoming server request into an outgoing
client request, nil out the Body field if the ContentLength is 0. (For
server requests, Body is always non-nil, even for GET, HEAD, etc.)
Also, don't use the deprecated CancelRequest and use Context instead.
And don't set Proto, ProtoMajor, ProtoMinor. Those are ignored in
client requests, which was probably a later documentation
clarification.
Fixes#16036
Updates #16696 (remove useless Proto lines)
Change-Id: I70a869e9bd4bf240c5838e82fb5aa695a539b343
Reviewed-on: https://go-review.googlesource.com/28412
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
RFC 2616, section 14.10 says:
>>>
HTTP/1.1 proxies MUST parse the Connection header field before a message
is forwarded and, for each connection-token in this field, remove any
header field(s) from the message with the same name as the
connection-token. Connection options are signaled by the presence of a
connection-token in the Connection header field, not by any
corresponding additional header field(s), since the additional header
field may not be sent if there are no parameters associated with that
connection option.
<<<
The same requirement was included in RFC 7230, section 6.1.
Fixes#16875
Change-Id: I57ad4a4a17775537c8810d0edd7de1604317b5fa
Reviewed-on: https://go-review.googlesource.com/27970
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If client does not provided User-Agent header, do not set default one
used by net/http package when doing request to backend.
Fixes#15524
Change-Id: I9a46bb3b7ec106bc7c3071e235b872d279994d67
Reviewed-on: https://go-review.googlesource.com/23089
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
libcurl sends this (despite never being standardized), and the Google
GFE rejects it with a 400 bad request (but only when over http2?).
So nuke it.
Change-Id: I3fc95523d50f33a0e23bb26b9195f70ab0aed0f4
Reviewed-on: https://go-review.googlesource.com/19184
Reviewed-by: Chris Broadfoot <cbro@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
RFC Errata 4522 (http://www.rfc-editor.org/errata_search.php?eid=4522)
notes that RFC 2616 had a typo in a list of headers that the
httputil.ReverseProxy code copied. Fix the typo in our code.
Fixes#14174
Change-Id: Ifc8f18fd58a6508a02a23e54ff3c473f03e521d3
Reviewed-on: https://go-review.googlesource.com/19133
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The CloseNotifier implementation and documentation was
substantially changed in https://golang.org/cl/17750 but it was a bit
too aggressive.
Issue #13666 highlighted that in addition to breaking external
projects, even the standard library (httputil.ReverseProxy) didn't
obey the new rules about not using CloseNotifier until the
Request.Body is fully consumed.
So, instead of fixing httputil.ReverseProxy, dial back the rules a
bit. It's now okay to call CloseNotify before consuming the request
body. The docs now say CloseNotifier may wait to fire before the
request body is fully consumed, but doesn't say that the behavior is
undefined anymore. Instead, we just wait until the request body is
consumed and start watching for EOF from the client then.
This CL also adds a test to ReverseProxy (using a POST request) that
would've caught this earlier.
Fixes#13666
Change-Id: Ib4e8c29c4bfbe7511f591cf9ffcda23a0f0b1269
Reviewed-on: https://go-review.googlesource.com/18144
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Fixes#7782Fixes#9554
Updates #7237 (original metabug, before we switched to specific bugs)
Updates #11932 (plan9 still doesn't have net I/O deadline support)
Change-Id: I96f311b88b1501d884ebc008fd31ad2cf1e16d75
Reviewed-on: https://go-review.googlesource.com/15941
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
