net/http: make NewRequest set empty Body nil, don't peek Read Body in Transport

This CL makes NewRequest set Body nil for known-zero bodies, and makes
the http1 Transport not peek-Read a byte to determine whether there's
a body.

Background:

Many fields of the Request struct have different meanings for whether
they're outgoing (via the Transport) or incoming (via the Server).

For outgoing requests, ContentLength and Body are documented as:

	// Body is the request's body.
	//
	// For client requests a nil body means the request has no
	// body, such as a GET request. The HTTP Client's Transport
	// is responsible for calling the Close method.
	Body io.ReadCloser

	// ContentLength records the length of the associated content.
	// The value -1 indicates that the length is unknown.
	// Values >= 0 indicate that the given number of bytes may
	// be read from Body.
	// For client requests, a value of 0 with a non-nil Body is
	// also treated as unknown.
	ContentLength int64

Because of the ambiguity of what ContentLength==0 means, the http1 and
http2 Transports previously Read the first byte of a non-nil Body when
the ContentLength was 0 to determine whether there was an actual body
(with a non-zero length) and ContentLength just wasn't populated, or
it was actually empty.

That byte-sniff has been problematic and gross (see #17480, #17071)
and was removed for http2 in a previous commit.

That means, however, that users doing:

    req, _ := http.NewRequest("POST", url, strings.NewReader(""))

... would not send a Content-Length header in their http2 request,
because the size of the reader (even though it was known, being one of
the three common recognized types from NewRequest) was zero, and so
the HTTP Transport thought it was simply unset.

To signal explicitly-zero vs unset-zero, this CL changes NewRequest to
signal explicitly-zero by setting the Body to nil, instead of the
strings.NewReader("") or other zero-byte reader.

This CL also removes the byte sniff from the http1 Transport, like
https://golang.org/cl/31326 did for http2.

Updates #17480
Updates #17071

Change-Id: I329f02f124659bf7d8bc01e2c9951ebdd236b52a
Reviewed-on: https://go-review.googlesource.com/31445
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

src/net/http/request.go

@@ -742,6 +742,15 @@ func NewRequest(method, urlStr string, body io.Reader) (*Request, error) {
 			req.ContentLength = int64(v.Len())
 		case *strings.Reader:
 			req.ContentLength = int64(v.Len())
+		default:
+			req.ContentLength = -1 // unknown
+		}
+		// For client requests, Request.ContentLength of 0
+		// means either actually 0, or unknown. The only way
+		// to explicitly say that the ContentLength is zero is
+		// to set the Body to nil.
+		if req.ContentLength == 0 {
+			req.Body = nil
 		}
 	}
 
@@ -1216,49 +1225,14 @@ func (r *Request) isReplayable() bool {
 	return false
 }
 
-// bodyAndLength reports the request's body and content length, with
-// the difference from r.ContentLength being that 0 means actually
-// zero, and -1 means unknown.
-func (r *Request) bodyAndLength() (body io.Reader, contentLen int64) {
-	body = r.Body
-	if body == nil {
-		return nil, 0
+// outgoingLength reports the Content-Length of this outgoing (Client) request.
+// It maps 0 into -1 (unknown) when the Body is non-nil.
+func (r *Request) outgoingLength() int64 {
+	if r.Body == nil {
+		return 0
 	}
 	if r.ContentLength != 0 {
-		return body, r.ContentLength
+		return r.ContentLength
 	}
-
-	// Don't try to sniff the request body if,
-	// * they're using a custom transfer encoding (or specified
-	//   chunked themselves)
-	// * they're not using HTTP/1.1 and can't chunk anyway (even
-	//   though this is basically irrelevant, since this package
-	//   only sends minimum 1.1 requests)
-	// * they're sending an "Expect: 100-continue" request, because
-	//   they might get denied or redirected and try to use the same
-	//   body elsewhere, so we shoudn't consume it.
-	if len(r.TransferEncoding) != 0 ||
-		!r.ProtoAtLeast(1, 1) ||
-		r.Header.Get("Expect") == "100-continue" {
-		return body, -1
-	}
-
-	// Test to see if it's actually zero or just unset.
-	var buf [1]byte
-	n, err := io.ReadFull(body, buf[:])
-	if err != nil && err != io.EOF {
-		return errorReader{err}, -1
-	}
-
-	if n == 1 {
-		// Oh, guess there is data in this Body Reader after all.
-		// The ContentLength field just wasn't set.
-		// Stich the Body back together again, re-attaching our
-		// consumed byte.
-		// TODO(bradfitz): switch to stitchByteAndReader
-		return io.MultiReader(bytes.NewReader(buf[:]), body), -1
-	}
-
-	// Body is actually empty.
-	return nil, 0
+	return -1
 }