Commit graph

283 commits

Author SHA1 Message Date
Bryan C. Mills
6760f20ef5 net/http: scale rstAvoidanceDelay to reduce test flakiness
As far as I can tell, some flakiness is unavoidable in tests
that race a large client request write against a server's response
when the server doesn't read the full request.
It does not appear to be possible to simultaneously ensure that
well-behaved clients see EOF instead of ECONNRESET and also prevent
misbehaving clients from consuming arbitrary server resources.
(See RFC 7230 §6.6 for more detail.)

Since there doesn't appear to be a way to cleanly eliminate
this source of flakiness, we can instead work around it:
we can allow the test to adjust the hard-coded delay if it
sees a plausibly-related failure, so that the test can retry
with a longer delay.

As a nice side benefit, this also allows the tests to run more quickly
in the typical case: since the test will retry in case of spurious
failures, we can start with an aggressively short delay, and only back
off to a longer one if it is really needed on the specific machine
running the test.

Fixes #57084.
Fixes #51104.
For #58398.

Change-Id: Ia4050679f0777e5eeba7670307a77d93cfce856f
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-race,gotip-linux-amd64-race,gotip-windows-amd64-race
Reviewed-on: https://go-review.googlesource.com/c/go/+/527196
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-09-13 20:57:25 +00:00
Jonathan Amsterdam
545e4f38e0 net/http: ServeMux handles extended patterns
Modify ServeMux to handle patterns with methods and wildcards.

Remove the map and list of patterns. Instead patterns
are registered and matched using a routing tree.

We also reorganize the code around "trailing-slash redirection,"
the feature whereby a trailing slash is added to a path
if it doesn't match an existing one. The existing code
checked the map of paths twice, but searching the tree
twice would be needlessly expensive. The rewrite
searches the tree once, and then again only if a
trailing-slash redirection is possible.

There are a few omitted features in this CL, indicated
with TODOs.

Change-Id: Ifaef59f6c8c7b7131dc4a5d0f101cc22887bdc74
Reviewed-on: https://go-review.googlesource.com/c/go/+/528039
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2023-09-13 20:07:22 +00:00
Daniel Martí
088f953fd0 all: add a few more godoc links
Over the past few months as I read the standard library's documentation
I kept finding spots where godoc links would have helped me.
I kept adding to a stash of changes to fix them up bit by bit.

The stash has grown big enough by now, and we're nearing a release,
so I think it's time to merge to avoid git conflicts or bit rot.

Note that a few sentences are slightly reworded,
since "implements the Fooer interface" can just be "implements [Fooer]"
now that the link provides all the context needed to the user.

Change-Id: I01c31d3d3ff066d06aeb44f545f8dd0fb9a8d998
Reviewed-on: https://go-review.googlesource.com/c/go/+/508395
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-08-07 06:58:18 +00:00
Jes Cok
2b82d70f85 net/http: declare publicErr as a constant
Do the same as the code above: "case err == errTooLarge", declare
publicErr as a constant to avoid runtime calls.

Change-Id: I50a9951232c70eff027b0da86c0bbb8bea51acbe
GitHub-Last-Rev: 71d4458ded
GitHub-Pull-Request: golang/go#60884
Reviewed-on: https://go-review.googlesource.com/c/go/+/504456
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Olif Oftimis <oftimisolif@gmail.com>
2023-07-20 16:57:26 +00:00
Al Cutter
3c8b7a9551 net/http: check RemoteAddr isn't nil before dereferencing
RemoteAddr can return nil in some cases, this fix prevents a panic.

I chatted with @neild about this beforehand, but what's happening in our
case is that a connection comes in to the HTTP server which is then
immediately closed (we discovered this issue by accident using nmap).
The network implementation that we're using (it happens to be gVisor
via its gonet adaptor) is returning nil from RemoteAddr(), presumably
as there is no remote at that point.

But, ultimately, since RemoteAddr returns an interface it is always
possible for it to return nil, and indeed conn.RemoteAddr in this file
does exactly that if the conn is not ok.

Change-Id: Ibe67ae6e30b68e2776df5ee2911bf5f1dc539641
GitHub-Last-Rev: ff3505d1d0
GitHub-Pull-Request: golang/go#60823
Reviewed-on: https://go-review.googlesource.com/c/go/+/503656
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-16 18:24:26 +00:00
Jorropo
543e601c11 net/http: second do not force the Content-Length header if nilled
This is a second round of CL 469095 which has been fixed after
the issue discovered in the revert CL 495017.

The issue was a missing res.Body.Close() in the newly added test.

Change-Id: Ifd9d8458022e59f4486397443a2862d06383e990
Reviewed-on: https://go-review.googlesource.com/c/go/+/495115
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Jorropo <jorropo.pgm@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-05-24 22:39:33 +00:00
Austin Clements
7213f2e720 Revert "net/http: do not force the Content-Length header if nilled"
This reverts CL 469095.

The newly added TestDisableContentLength is failing on all longtest
builders.

Change-Id: Id307df61c7bf80691d9c276e8d200eebf6d4a59c
Reviewed-on: https://go-review.googlesource.com/c/go/+/495017
Auto-Submit: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-15 21:31:16 +00:00
Damien Neil
268d2f7cf2 net/http: handle WriteHeader(101) as a non-informational header
Prior to Go 1.19 adding support for sending 1xx informational headers
with ResponseWriter.WriteHeader, WriteHeader(101) would send a 101
status and disable further writes to the response. This behavior
was not documented, but is intentional: Writing to the response
body explicitly checks to see if a 101 status has been sent before
writing.

Restore the pre-1.19 behavior when writing a 101 Switching Protocols
header: The header is sent, no subsequent headers are sent, and
subsequent writes to the response body fail.

For #59564

Change-Id: I72c116f88405b1ef5067b510f8c7cff0b36951ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/485775
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-15 17:35:28 +00:00
Laurent Senta
949d0f4f99 net/http: do not force the Content-Length header if nilled
According to the ResponseWriter documentation:

  To suppress automatic response headers (such as "Date"), set
  their value to nil.

In some cases, this documentation is incorrect: chunkWriter writes
a Content-Length header even if the value was set to nil. Meaning
there is no way to suppress this header.

This patch replaces the empty string comparison with a call to
`header.has` which takes into account nil values as expected.
This is similar to the way we handle the "Date" header.

Change-Id: Ie10d54ab0bb7d41270bc944ff867e035fe2bd0c5
GitHub-Last-Rev: e0616dd463
GitHub-Pull-Request: golang/go#58578
Reviewed-on: https://go-review.googlesource.com/c/go/+/469095
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
Run-TryBot: Jorropo <jorropo.pgm@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-05-15 17:07:02 +00:00
Damien Neil
457fd1d52d net/http: support full-duplex HTTP/1 responses
Add support for concurrently reading from an HTTP/1 request body
while writing the response.

Normally, the HTTP/1 server automatically consumes any remaining
request body before starting to write a response, to avoid deadlocking
clients which attempt to write a complete request before reading the
response.

Add a ResponseController.EnableFullDuplex method which disables this
behavior.

For #15527
For #57786

Change-Id: Ie7ee8267d8333e9b32b82b9b84d4ad28ab8edf01
Reviewed-on: https://go-review.googlesource.com/c/go/+/472636
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2023-03-07 22:52:18 +00:00
Damien Neil
133e0bca0b net/http: remove warning when parsing a query containing a semicolon
It's been years since the behavior here was changed, and there's
little point in continuing to warn users of it.

Fixes #49399

Change-Id: I95f64ca14cacb64ebe78296593b1cc3d837e6b77
Reviewed-on: https://go-review.googlesource.com/c/go/+/470315
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-02-22 21:08:59 +00:00
cui fliter
117fe2e08d all: fix some comments
Change-Id: I16ec916b47de2f417b681c8abff5a1375ddf491b
Reviewed-on: https://go-review.googlesource.com/c/go/+/468055
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-02-14 16:27:39 +00:00
Damien Neil
b02d5d325a Revert "io: allocate copy buffers from a pool"
This reverts CL 456555.

Reason for revert: This seems too likely to exercise race conditions
in code where a Write call continues to access its buffer after
returning. The HTTP/2 ResponseWriter is one such example.

Reverting this change while we think about this some more.

For #57202

Change-Id: Ic86823f81d7da410ea6b3f17fb5b3f9a979e3340
Reviewed-on: https://go-review.googlesource.com/c/go/+/467095
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-10 16:34:44 +00:00
Damien Neil
7d7fd6d362 io: allocate copy buffers from a pool
CopyBuffer allocates a 32k buffer when no buffer is available.
Allocate these buffers from a sync.Pool.

This removes an optimization where the copy buffer size was
reduced when the source is a io.LimitedReader (including the
case of CopyN) with a limit less than the default buffer size.
This change could cause a program which only uses io.Copy
with sources with a small limit to allocate unnecessarily
large buffers. Programs which care about the transient
buffer allocation can avoid this by providing their own buffer.

name           old time/op    new time/op    delta
CopyNSmall-10     165ns ± 7%     117ns ± 7%  -29.19%  (p=0.001 n=7+7)
CopyNLarge-10    7.33µs ±34%    4.07µs ± 2%  -44.52%  (p=0.001 n=7+7)

name           old alloc/op   new alloc/op   delta
CopyNSmall-10    2.20kB ±12%    1.20kB ± 4%  -45.24%  (p=0.000 n=8+7)
CopyNLarge-10     148kB ± 9%      81kB ± 4%  -45.26%  (p=0.000 n=8+7)

name           old allocs/op  new allocs/op  delta
CopyNSmall-10      2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=8+8)
CopyNLarge-10      2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=8+8)

For #57202

Change-Id: I2292226da9ba1dc09a2543f5d74fe5da06080d49
Reviewed-on: https://go-review.googlesource.com/c/go/+/456555
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Thomas Austad <thomas.austad@gmail.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-01-31 17:44:33 +00:00
Oleksii Sholik
5ad799ef8a net/http: fix missing word in ServeMux doc
Change-Id: I324cedfcdfa1d7eac7be43dc89febb584359c8e3
GitHub-Last-Rev: 7724cd84f3
GitHub-Pull-Request: golang/go#57799
Reviewed-on: https://go-review.googlesource.com/c/go/+/462039
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
2023-01-27 00:42:16 +00:00
Russ Cox
213495a4a6 internal/godebug: export non-default-behavior counters in runtime/metrics
Allow GODEBUG users to report how many times a setting
resulted in non-default behavior.

Record non-default-behaviors for all existing GODEBUGs.

Also rework tests to ensure that runtime is in sync with runtime/metrics.All,
and generate docs mechanically from metrics.All.

For #56986.

Change-Id: Iefa1213e2a5c3f19ea16cd53298c487952ef05a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/453618
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-19 22:26:43 +00:00
Russ Cox
ea4631cc0c internal/godebug: define more efficient API
We have been expanding our use of GODEBUG for compatibility,
and the current implementation forces a tradeoff between
freshness and efficiency. It parses the environment variable
in full each time it is called, which is expensive. But if clients
cache the result, they won't respond to run-time GODEBUG
changes, as happened with x509sha1 (#56436).

This CL changes the GODEBUG API to provide efficient,
up-to-date results. Instead of a single Get function,
New returns a *godebug.Setting that itself has a Get method.
Clients can save the result of New, which is no more expensive
than errors.New, in a global variable, and then call that
variable's Get method to get the value. Get costs only two
atomic loads in the case where the variable hasn't changed
since the last call.

Unfortunately, these changes do require importing sync
from godebug, which will mean that sync itself will never
be able to use a GODEBUG setting. That doesn't seem like
such a hardship. If it was really necessary, the runtime could
pass a setting to package sync itself at startup, with the
caveat that that setting, like the ones used by runtime itself,
would not respond to run-time GODEBUG changes.

Change-Id: I99a3acfa24fb2a692610af26a5d14bbc62c966ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/449504
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-14 15:19:57 +00:00
Damien Neil
fd0c0db4a4 net/http: add ResponseController and per-handler timeouts
The ResponseController type provides a discoverable interface
to optional methods implemented by ResponseWriters.

	c := http.NewResponseController(w)
	c.Flush()

vs.

	if f, ok := w.(http.Flusher); ok {
		f.Flush()
	}

Add the ability to control per-request read and write deadlines
via the ResponseController SetReadDeadline and SetWriteDeadline
methods.

For #54136

Change-Id: I3f97de60d4c9ff150cda559ef86c6620eee665d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/436890
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
2022-11-10 18:18:03 +00:00
cuiweixie
2b45aebe14 net/http: using strings.CutPrefix replace strings.HasPrefix and strings.TrimPrefix
Change-Id: I0b7b6e4e9d2539e4fcb5c08430ba5a74733fad3c
Reviewed-on: https://go-review.googlesource.com/c/go/+/435136
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: xie cui <523516579@qq.com>
2022-09-29 21:28:08 +00:00
cuiweixie
7db923fe56 net/http: convert Server.disableKeepAlives to atomic type
Change-Id: I87526520b519554ea344288cc0f0940d7b182e21
Reviewed-on: https://go-review.googlesource.com/c/go/+/428815
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2022-09-19 22:51:53 +00:00
cui fliter
b244b89d56 net/http: convert allowQuerySemicolonsInUse to atomic type
For #53821

Change-Id: I135783bd5472011f6a74d2f5ee34ce96ff49ad2b
GitHub-Last-Rev: 4da2d67ff3
GitHub-Pull-Request: golang/go#54863
Reviewed-on: https://go-review.googlesource.com/c/go/+/428336
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-09-09 18:42:57 +00:00
Eliza Weisman
4e7e7ae140 net/http: don't time out idle server connections after ReadHeaderTimeout
Consistently wait for idle connections to become readable before
starting the ReadHeaderTimeout timer. Previously, connections with no
idle timeout skipped directly to reading headers, so the
ReadHeaderTimeout also included time spent idle.

Fixes #54784

Change-Id: Iff1a876f00311d03dfa0fbef5b577506c62f7c41
GitHub-Last-Rev: 09332743ad
GitHub-Pull-Request: golang/go#54785
Reviewed-on: https://go-review.googlesource.com/c/go/+/426895
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2022-09-05 07:17:56 +00:00
Alexander Yastrebov
8a86b94aef net/http: remove unused doneChan
The https://golang.org/cl/43230 removed use of `getDoneChan`.

Change-Id: I33390c0e3aea6d98367363773ebe39d9c1f64ae9
GitHub-Last-Rev: fe1e4154ea
GitHub-Pull-Request: golang/go#53172
Reviewed-on: https://go-review.googlesource.com/c/go/+/409538
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: hopehook <hopehook@qq.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-24 17:12:07 +00:00
Alexander Yastrebov
05ff045dfe net/http: add Server.DisableOptionsHandler for custom handling of OPTIONS *
Fixes #41773

Change-Id: I432ad5410d5e3bb0aff3a6e0eea6906ab1b214e2
GitHub-Last-Rev: 57d1ee249d
GitHub-Pull-Request: golang/go#49014
Reviewed-on: https://go-review.googlesource.com/c/go/+/356410
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: hopehook <hopehook@qq.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-15 18:57:35 +00:00
Ludi Rehak
f3c39a83a3 all: replace hand-rolled atomicBool types with atomic.Bool
Two packages construct atomic booleans from atomic integers.
Replace these implementations with the new atomic.Bool type.
Indeed, these packages were the impetus for the new atomic.Bool
type, having demonstrated a need to access boolean values
atomically.

Change-Id: I6a0314f8e7d660984a6daf36a62ed05a0eb74b2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/411400
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2022-08-12 16:41:47 +00:00
Abirdcfly
469a87d7a5 net/http: remove unused method needsSniff
Change-Id: Ia8c67d148fc7e71cac8954ccce6fffb8a6f423df
GitHub-Last-Rev: 9441b8f0c5
GitHub-Pull-Request: golang/go#54036
Reviewed-on: https://go-review.googlesource.com/c/go/+/419322
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Javad Rajabzadeh <ja7ad@live.com>
Reviewed-by: hopehook <hopehook@qq.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2022-08-09 18:11:37 +00:00
Ludi Rehak
adac8acf88 net/http: change conn.curReq type to atomic.Pointer[response]
Use the newly added atomic.Pointer[T] type for atomically
loading and storing type *T pointers. This has the advantage of
avoiding runtime type assertions required by its predecessor,
atomic.Value.

To fix build failures uncovered by TryBots (caused by "panic:
unaligned 64-bit atomic operation"), also change conn.curState to
type atomic.Uint64 so that it is 64-bit aligned.

Change-Id: I6024d12cd581adfdccc01be7eb0faa7482036614
Reviewed-on: https://go-review.googlesource.com/c/go/+/420901
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2022-08-08 16:01:12 +00:00
Damien Neil
180bcad33d net/http: wait for listeners to exit in Server.Close and Shutdown
Avoid race conditions when a new connection is accepted just after
Server.Close or Server.Shutdown is called by waiting for the
listener goroutines to exit before proceeding to clean up active
connections.

No test because the mechanism required to trigger the race condition
reliably requires such tight coupling to the Server internals that
any test would be quite fragile in the face of reasonable refactorings.

Fixes #48642
Updates #33313, #36819

Change-Id: I109a93362680991bf298e0a95637595dcaa884af
Reviewed-on: https://go-review.googlesource.com/c/go/+/409537
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-07-08 17:00:50 +00:00
Kévin Dunglas
770e0e584a net/http: allow sending 1xx responses
Currently, it's not possible to send informational responses such as
103 Early Hints or 102 Processing.

This patch allows calling WriteHeader() multiple times in order
to send informational responses before the final one.

If the status code is in the 1xx range, the current content of the header map
is also sent. Its content is not removed after the call to WriteHeader()
because the headers must also be included in the final response.

The Chrome and Fastly teams are starting a large-scale experiment to measure
the real-life impact of the 103 status code.
Using Early Hints is proposed as a (partial) alternative to Server Push,
which are going to be removed from Chrome:
https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYLvmQUBY/m/21anpFhxAQAJ

Being able to send this status code from servers implemented using Go would
help to see if implementing it in browsers is worth it.

Fixes #26089
Fixes #36734
Updates #26088

Change-Id: Ib7023c1892c35e8915d4305dd7f6373dbd00a19d
GitHub-Last-Rev: 06d749d345
GitHub-Pull-Request: golang/go#42597
Reviewed-on: https://go-review.googlesource.com/c/go/+/269997
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-17 16:09:37 +00:00
Alexander Yastrebov
c14ed5b37c Revert "net/http: close accepted connection"
This reverts CL 353714.

The change closes accepted connection also in graceful shutdown which
breaks the fix for #33313 (and apparent duplicate #36819).

The proper fix should close accepted connection only if server is closed
but not in graceful shutdown.

Updates #48642

Change-Id: I2f7005f3f3037e6563745731bb2693923b654004
GitHub-Last-Rev: f6d885aa37
GitHub-Pull-Request: golang/go#52823
Reviewed-on: https://go-review.googlesource.com/c/go/+/405454
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-11 23:44:16 +00:00
Alexander Yastrebov
1ce7fcf139 net/http: close accepted connection
Fixes #48642

Change-Id: I7bf00517bea43dcf38e15b778818a3a3f6ffe23e
GitHub-Last-Rev: a0e8b80f18
GitHub-Pull-Request: golang/go#48753
Reviewed-on: https://go-review.googlesource.com/c/go/+/353714
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2022-05-09 21:24:34 +00:00
João Penteado
ac01de5446 net/http: optimize StatusText implementation
The current implementation, although more succinct, relies on a runtime
lookup to a "constant" unexported map (which also needs to be
initialized at runtime).

The proposed implementation is able to be optimized by the compiler at
build-time, resulting in *much* more efficient instructions.
Additionally, unused string literals may even be removed altogether
from the generated binary in some cases.

This change is fully backwards-compatible behavior-wise with the
existing implementation.

Change-Id: I36450320aacff5b322195820552f2831d4fecd52
GitHub-Last-Rev: e2058f132e
GitHub-Pull-Request: golang/go#49811
Reviewed-on: https://go-review.googlesource.com/c/go/+/367201
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2022-04-15 17:03:50 +00:00
Russ Cox
19309779ac all: gofmt main repo
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]

Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.

For #51082.

Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-11 16:34:30 +00:00
Daniel Martí
3e7ffb862f all: consistently use US spelling of present participles
It has been agreed that we should prefer the US spelling of words like
"canceling" over "cancelling"; for example, see https://go.dev/cl/14526.

Fix a few occurrences of the "canceling" inconsistency, as well as:

* signaling
* tunneling
* marshaling

Change-Id: I99f3ba0a700a9f0292bc6c1b110af31dd05f1ff0
Reviewed-on: https://go-review.googlesource.com/c/go/+/398734
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-08 13:44:41 +00:00
Russ Cox
81431c7aa7 all: replace `` and '' with “ (U+201C) and ” (U+201D) in doc comments
go/doc in all its forms applies this replacement when rendering
the comments. We are considering formatting doc comments,
including doing this replacement as part of the formatting.
Apply it to our source files ahead of time.

For #51082.

Change-Id: Ifcc1f5861abb57c5d14e7d8c2102dfb31b7a3a19
Reviewed-on: https://go-review.googlesource.com/c/go/+/384262
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-05 17:52:29 +00:00
Russ Cox
7d87ccc860 all: fix various doc comment formatting nits
A run of lines that are indented with any number of spaces or tabs
format as a <pre> block. This commit fixes various doc comments
that format badly according to that (standard) rule.

For example, consider:

	// - List item.
	//   Second line.
	// - Another item.

Because the - lines are unindented, this is actually two paragraphs
separated by a one-line <pre> block. This CL rewrites it to:

	//  - List item.
	//    Second line.
	//  - Another item.

Today, that will format as a single <pre> block.
In a future release, we hope to format it as a bulleted list.

Various other minor fixes as well, all in preparation for reformatting.

For #51082.

Change-Id: I95cf06040d4186830e571cd50148be3bf8daf189
Reviewed-on: https://go-review.googlesource.com/c/go/+/384257
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01 18:18:01 +00:00
Reilly Watson
b5e06545b3 net/http: fix link to ResponseWriter trailer example
The links to this example were using an underscore separator
instead of a hyphen, and incorrect casing.

Fixes #50279

Change-Id: I35d76a8a78cd708b7505ff1a70f7dacddaf43efd
Reviewed-on: https://go-review.googlesource.com/c/go/+/373514
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2021-12-21 03:54:36 +00:00
Russ Cox
2580d0e08d all: gofmt -w -r 'interface{} -> any' src
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.

Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.

A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.

Fixes #49884.

Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13 18:45:54 +00:00
Damien Neil
b69b2f63d6 net/http: do not send Transfer-Encoding: identity in responses
Server handlers may set a "Transfer-Encoding: identity" header on
responses to disable chunking, but this header should not be sent
on the wire.

Fixes #49194.

Change-Id: I46a9e3b8ff9d93edd7d1c34d264fc309fa322ad5
Reviewed-on: https://go-review.googlesource.com/c/go/+/359176
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-11-12 23:35:31 +00:00
Carl Johnson
55e6e825d4 net/http: add MaxBytesHandler
Fixes #39567

Change-Id: I226089b678a6a13d7ce69f360a23fc5bd297d550
GitHub-Last-Rev: 6435fd5881
GitHub-Pull-Request: golang/go#48104
Reviewed-on: https://go-review.googlesource.com/c/go/+/346569
Trust: Damien Neil <dneil@google.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2021-11-09 18:23:16 +00:00
Charlie Getzen
4c7cafdd03 net/http: distinguish between timeouts and client hangups in TimeoutHandler
Fixes #48948

Change-Id: I411e3be99c7979ae289fd937388aae63d81adb59
GitHub-Last-Rev: 14abd7e4d7
GitHub-Pull-Request: golang/go#48993
Reviewed-on: https://go-review.googlesource.com/c/go/+/356009
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-11-05 21:18:28 +00:00
Brad Fitzpatrick
a59e33224e net/netip: add new IP address package
Co-authored-by: Alex Willmer <alex@moreati.org.uk> (GitHub @moreati)
Co-authored-by: Alexander Yastrebov <yastrebov.alex@gmail.com>
Co-authored-by: David Anderson <dave@natulte.net> (Tailscale CLA)
Co-authored-by: David Crawshaw <crawshaw@tailscale.com> (Tailscale CLA)
Co-authored-by: Dmytro Shynkevych <dmytro@tailscale.com> (Tailscale CLA)
Co-authored-by: Elias Naur <mail@eliasnaur.com>
Co-authored-by: Joe Tsai <joetsai@digital-static.net> (Tailscale CLA)
Co-authored-by: Jonathan Yu <jawnsy@cpan.org> (GitHub @jawnsy)
Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> (Tailscale CLA)
Co-authored-by: Maisem Ali <maisem@tailscale.com> (Tailscale CLA)
Co-authored-by: Manuel Mendez (Go AUTHORS mmendez534@...)
Co-authored-by: Matt Layher <mdlayher@gmail.com>
Co-authored-by: Noah Treuhaft <noah.treuhaft@gmail.com> (GitHub @nwt)
Co-authored-by: Stefan Majer <stefan.majer@gmail.com>
Co-authored-by: Terin Stock <terinjokes@gmail.com> (Cloudflare CLA)
Co-authored-by: Tobias Klauser <tklauser@distanz.ch>

Fixes #46518

Change-Id: I0041f9e1115d61fa6e95fcf32b01d9faee708712
Reviewed-on: https://go-review.googlesource.com/c/go/+/339309
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
2021-11-02 01:28:01 +00:00
Brad Fitzpatrick
b59467e036 net/http: also use Server.ReadHeaderTimeout for TLS handshake deadline
Fixes #48120

Change-Id: I72e89af8aaf3310e348d8ab639925ce0bf84204d
Reviewed-on: https://go-review.googlesource.com/c/go/+/355870
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2021-10-14 22:15:24 +00:00
Russ Cox
4d8db00641 all: use bytes.Cut, strings.Cut
Many uses of Index/IndexByte/IndexRune/Split/SplitN
can be written more clearly using the new Cut functions.
Do that. Also rewrite to other functions if that's clearer.

For #46336.

Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448
Reviewed-on: https://go-review.googlesource.com/c/go/+/351711
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-06 15:53:04 +00:00
Damien Neil
2a463a22ce net/http: close request body after recovering from a handler panic
When recovering from a panic in a HTTP handler, close the request body
before closing the *conn, ensuring that the *conn's bufio.Reader is safe
to recycle.

Fixes #46866.

Change-Id: I3fe304592e3b423a0970727d68bc1229c3752939
Reviewed-on: https://go-review.googlesource.com/c/go/+/329922
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-09-02 16:59:57 +00:00
Filippo Valsorda
d77f4c0c5c net/http: improve some server docs
Change-Id: I04662a08e07c49f629f9067a89bf453e697d44dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/327813
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2021-06-15 20:59:42 +00:00
Filippo Valsorda
e4e7807d24 net/http: add AllowQuerySemicolons
Fixes #45973

Change-Id: I6cbe05f5d1d3c324900c74314b0ea0e12524d7f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/326309
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-09 16:59:02 +00:00
Katie Hockman
e6dda19888 net/url: reject query values with semicolons
Semicolons are no longer valid separators, so
net/url.ParseQuery will now return an error
if any part of the query contains a semicolon.

net/http.(*Request).ParseMultipartForm has been
changed to fall through and continue parsing
even if the call to (*Request).ParseForm fails.

This change also includes a few minor refactors
to existing tests.

Fixes #25192

Change-Id: Iba3f108950fb99b9288e402c41fe71ca3a2ababd
Reviewed-on: https://go-review.googlesource.com/c/go/+/325697
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-06-09 15:44:42 +00:00
Damien Neil
831f9376d8 net/http: fix ResponseWriter.ReadFrom with short reads
CL 249238 changes ResponseWriter.ReadFrom to probe the source with
a single read of sniffLen bytes before writing the response header.
If the source returns less than sniffLen bytes without reaching
EOF, this can cause Content-Type and Content-Length detection to
fail.

Fix ResponseWrite.ReadFrom to copy a full sniffLen bytes from
the source as a probe.

Drop the explicit call to w.WriteHeader; writing the probe will
trigger a WriteHeader call.

Consistently use io.CopyBuffer; ReadFrom has already acquired a
copy buffer, so it may as well use it.

Fixes #44953.

Change-Id: Ic49305fb827a2bd7da4764b68d64b797b5157dc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/301449
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-06-04 17:33:24 +00:00
Roland Shoemaker
b584230889 net/http: use relative path in Location redirect
If the cleaned path did not match the requested path, ServeMux.Handler
would return a Location header which reflected the hostname in the
request, possibly leading to an incorrect redirect. Instead the
Location header should be relative, like the other cases in
ServeMux.Handler.

Change-Id: I2c220d925e708061bc128f0bdc96cca7a32764d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/313950
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-05-03 17:34:10 +00:00