This reverts CL 254537.
Reason for revert: Reason for revert: The recommended way to check for a type of error is errors.As. API changes should also start with a proposal.
Change-Id: I07c37428575e99c80b17525833a61831d10963bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/254857
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Allows users to check:
errors.Is(err, &UnmarshalTypeError{})
errors.Is(err, &UnmarshalFieldError{})
errors.Is(err, &InvalidUnmarshalError{})
errors.Is(err, &UnsupportedValueError{})
errors.Is(err, &MarshalerError{})
which is the recommended way of checking for kinds of errors.
SyntaxError.Is was implemented in CL 253037.
As and Unwrap relevant methods will be added in future CLs.
Change-Id: I1f8a503b8fdc0f3afdfe9669a91f3af8d960e028
GitHub-Last-Rev: 930cda5384
GitHub-Pull-Request: golang/go#41360
Reviewed-on: https://go-review.googlesource.com/c/go/+/254537
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
golang.org/cl/193604 fixed one bug when one encodes a string with the
",string" option: if SetEscapeHTML(false) is used, we should not be
using HTML escaping for the inner string encoding. The CL correctly
fixed that.
The CL also tried to speed up this edge case. By avoiding an entire new
call to Marshal, the new Issue34127 benchmark reduced its time/op by
45%, and lowered the allocs/op from 3 to 2.
However, that last optimization wasn't correct:
Since Go 1.2 every string can be marshaled to JSON without error
even if it contains invalid UTF-8 byte sequences. Therefore
there is no need to use Marshal again for the only reason of
enclosing the string in double quotes.
JSON string encoding isn't just about adding quotes and taking care of
invalid UTF-8. We also need to escape some characters, like tabs and
newlines.
The new code failed to do that. The bug resulted in the added test case
failing to roundtrip properly; before our fix here, we'd see an error:
invalid use of ,string struct tag, trying to unmarshal "\"\b\f\n\r\t\"\\\"" into string
If you pay close attention, you'll notice that the special characters
like tab and newline are only encoded once, not twice. When decoding
with the ",string" option, the outer string decode works, but the inner
string decode fails, as we are now decoding a JSON string with unescaped
special characters.
The fix we apply here isn't to go back to Marshal, as that would
re-introduce the bug with SetEscapeHTML(false). Instead, we can use a
new encode state from the pool - it results in minimal performance
impact, and even reduces allocs/op further. The performance impact seems
fair, given that we need to check the entire string for characters that
need to be escaped.
name old time/op new time/op delta
Issue34127-8 89.7ns ± 2% 100.8ns ± 1% +12.27% (p=0.000 n=8+8)
name old alloc/op new alloc/op delta
Issue34127-8 40.0B ± 0% 32.0B ± 0% -20.00% (p=0.000 n=8+8)
name old allocs/op new allocs/op delta
Issue34127-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8)
Instead of adding another standalone test, we convert an existing
"string tag" test to be table-based, and add another test case there.
One test case from the original CL also had to be amended, due to the
same problem - when escaping '<' due to SetEscapeHTML(true), we need to
end up with double escaping, since we're using ",string".
Fixes#38173.
Change-Id: I2b0df9e4f1d3452fff74fe910e189c930dde4b5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/226498
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This updates the link to a newer image.
Change-Id: Ibdfe8c57d9217a325bcfde98cb6f952ca63d588a
GitHub-Last-Rev: f5970ba395
GitHub-Pull-Request: golang/go#36938
Reviewed-on: https://go-review.googlesource.com/c/go/+/217297
Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Otherwise we'd panic with a stack overflow.
Most programs are in control of the data being encoded and can ensure
there are no cycles, but sometimes it's not that simple. For example,
running a user's html template with script tags can easily result in
crashes if the user can find a pointer cycle.
Adding the checks via a map to every ptrEncoder.encode call slowed down
the benchmarks below by a noticeable 13%. Instead, only start doing the
relatively expensive pointer cycle checks if we're many levels of
pointers deep in an encode state.
A threshold of 1000 is small enough to capture pointer cycles before
they're a problem (the goroutine stack limit is currently 1GB, and I
needed close to a million levels to reach it). Yet it's large enough
that reasonable uses of the json encoder only see a tiny 1% slow-down
due to the added ptrLevel field and check.
name old time/op new time/op delta
CodeEncoder-8 2.34ms ± 1% 2.37ms ± 0% +1.05% (p=0.000 n=10+10)
CodeMarshal-8 2.42ms ± 1% 2.44ms ± 0% +1.10% (p=0.000 n=10+10)
name old speed new speed delta
CodeEncoder-8 829MB/s ± 1% 820MB/s ± 0% -1.04% (p=0.000 n=10+10)
CodeMarshal-8 803MB/s ± 1% 795MB/s ± 0% -1.09% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
CodeEncoder-8 43.1kB ± 8% 42.5kB ±10% ~ (p=0.989 n=10+10)
CodeMarshal-8 1.99MB ± 0% 1.99MB ± 0% ~ (p=0.254 n=9+6)
name old allocs/op new allocs/op delta
CodeEncoder-8 0.00 0.00 ~ (all equal)
CodeMarshal-8 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Finally, add a few tests to ensure that the code handles the edge cases
properly.
Fixes#10769.
Change-Id: I73d48e0cf6ea140127ea031f2dbae6e6a55e58b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/187920
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
If we marshal a non-pointer struct field whose type implements Marshaler with
a non-pointer receiver, then we avoid an allocation if we take the address of
the field before casting it to an interface.
name old time/op new time/op delta
EncodeMarshaler-8 104ns ± 1% 92ns ± 2% -11.72% (p=0.001 n=7+7)
name old alloc/op new alloc/op delta
EncodeMarshaler-8 36.0B ± 0% 4.0B ± 0% -88.89% (p=0.000 n=8+8)
name old allocs/op new allocs/op delta
EncodeMarshaler-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=8+8)
Test coverage already looks good enough for this change. TestRefValMarshal
already covers all possible combinations of value & pointer receivers on
value and pointer struct fields.
Change-Id: I6fc7f72396396d98f9a90c3c86e813690f41c099
Reviewed-on: https://go-review.googlesource.com/c/go/+/203608
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
1. Change mapencode.encode to use fmt.Error rather than MarshalerError.
MarshalerError refer to MarshalJSON, but mapencode.encode does not use that.
2. Add sourceFunc field to MarshalerError to record the name of the function
that creates the error, so that the Error method can report it correctly.
Fixes#29753
Change-Id: I186c2fac8470ae2f9e300501de3730face642230
Reviewed-on: https://go-review.googlesource.com/c/go/+/184119
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Add quotes when marshaling a json.Number with the string option
set via a struct tag. This ensures that the resulting json
can be unmarshaled into the source struct without error.
Fixes#34268
Change-Id: Ide167d9dec77019554870b5957b37dc258119d81
GitHub-Last-Rev: dde81b7120
GitHub-Pull-Request: golang/go#34269
Reviewed-on: https://go-review.googlesource.com/c/go/+/195043
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Since Go 1.2 every string can be marshaled to JSON without error even if it
contains invalid UTF-8 byte sequences. Therefore there is no need to use
Marshal again for the only reason of enclosing the string in double quotes.
Not using Marshal here also removes the error check as there has not been a
way for Marshal to fail anyway.
name old time/op new time/op delta
Issue34127-4 360ns ± 3% 200ns ± 3% -44.56% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Issue34127-4 56.0B ± 0% 40.0B ± 0% -28.57% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Issue34127-4 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.008 n=5+5)
Fixes#34154
Change-Id: Ib60dc11980f9b20d8bef2982de7168943d632263
GitHub-Last-Rev: 9b0ac1d4c5
GitHub-Pull-Request: golang/go#34127
Reviewed-on: https://go-review.googlesource.com/c/go/+/193604
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reset is already performed when retrieving from pool
Change-Id: Ia810dd18d3e55a1565a5ad435a00d1e46724576c
GitHub-Last-Rev: d9df74a4ae
GitHub-Pull-Request: golang/go#34195
Reviewed-on: https://go-review.googlesource.com/c/go/+/194338
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
This change adds a a check in the encodeWithString.resolve method
to ensure that a reflect.Value with kind Ptr is not nil before
the type assertion to TextMarshaler.
If the value is nil, the method returns a nil error, and the map key
encodes to an empty string.
Fixes#33675
Change-Id: I0a04cf690ae67006f6a9c5f8cbb4cc99d236bca8
GitHub-Last-Rev: 6c987c9084
GitHub-Pull-Request: golang/go#33700
Reviewed-on: https://go-review.googlesource.com/c/go/+/190697
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
The decoder called this function to check numbers being decoded into a
json.Number. However, these can't be quoted as strings, so the tokenizer
has already verified they are valid JSON numbers.
Verified this by adding a test with such an input. As expected, it
produces a syntax error, not the fmt.Errorf - that line could never
execute.
Since the only remaining non-test caller of isvalidnumber is in
encode.go, move the function there.
This change should slightly reduce the amount of work when decoding into
json.Number, though that isn't very common nor part of any current
benchmarks.
Change-Id: I67a1723deb3d18d5b542d6dd35f3ae56a43f23eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/184817
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It wasn't obeyed in the case where the MarshalJSON method uses a pointer
receiver, and the encoder grabs the address of a value to find that
method. addrMarshalerEncoder is the function that does this work, but it
ignored opts.escapeHTML.
Here's the before and after of the added test case, which was failing
before the fix. Now the two cases are correct and consistent.
{"NonPtr":"<str>","Ptr":"\u003cstr\u003e"}
{"NonPtr":"<str>","Ptr":"<str>"}
Fixes#32896.
Change-Id: Idc53077ece074973558bd3bb5ad036380db0d02c
Reviewed-on: https://go-review.googlesource.com/c/go/+/184757
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Caleb Spare <cespare@gmail.com>
Add Unwrap methods to types which wrap an underlying error:
"encodinc/csv".ParseError
"encoding/json".MarshalerError
"net/http".transportReadFromServerError
"net".OpError
"net".DNSConfigError
"net/url".Error
"os/exec".Error
"signal/internal/pty".PtyError
"text/template".ExecError
Add os.ErrTemporary. A case could be made for putting this error
value in package net, since no exported error types in package os
include a Temporary method. However, syscall errors returned from
the os package do include this method.
Add Is methods to error types with a Timeout or Temporary method,
making errors.Is(err, os.Err{Timeout,Temporary}) equivalent to
testing the corresponding method:
"context".DeadlineExceeded
"internal/poll".TimeoutError
"net".adrinfoErrno
"net".OpError
"net".DNSError
"net/http".httpError
"net/http".tlsHandshakeTimeoutError
"net/pipe".timeoutError
"net/url".Error
Updates #30322
Updates #29934
Change-Id: I409fb20c072ea39116ebfb8c7534d493483870dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/170037
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Make explicit that Compact does HTML escaping.
Fixes#30357.
Change-Id: I4648f8f3e907d659db977d07253f716df6e07d7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/173417
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In the common case, structs have a handful of fields and most inputs
match struct field names exactly.
The previous code would do a linear search over the fields, stopping at
the first exact match, and otherwise using the first case insensitive
match.
This is unfortunate, because it means that for the common case, we'd do
a linear search with bytes.Equal. Even for structs with only two or
three fields, that is pretty wasteful.
Worse even, up until the exact match was found via the linear search,
all previous fields would run their equalFold functions, which aren't
cheap even in the simple case.
Instead, cache a map along with the field list that indexes the fields
by their name. This way, a case sensitive field search doesn't involve a
linear search, nor does it involve any equalFold func calls.
This patch should also slightly speed up cases where there's a case
insensitive match but not a case sensitive one, as then we'd avoid
calling bytes.Equal on all the fields. Though that's not a common case,
and there are no benchmarks for it.
name old time/op new time/op delta
CodeDecoder-8 11.0ms ± 0% 10.6ms ± 1% -4.42% (p=0.000 n=9+10)
name old speed new speed delta
CodeDecoder-8 176MB/s ± 0% 184MB/s ± 1% +4.62% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.28MB ± 0% 2.28MB ± 0% ~ (p=0.725 n=10+10)
name old allocs/op new allocs/op delta
CodeDecoder-8 76.9k ± 0% 76.9k ± 0% ~ (all equal)
Updates #28923.
Change-Id: I9929c1f06c76505e5b96914199315dbdaae5dc76
Reviewed-on: https://go-review.googlesource.com/c/go/+/172918
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If a for loop has a simple condition and begins with a simple
"if x { break; }"; we can simply add "!x" to the loop's condition.
While at it, simplify a few assignments to use the common pattern
"x := staticDefault; if cond { x = otherValue(); }".
Finally, simplify a couple of var declarations.
Change-Id: I413982c6abd32905adc85a9a666cb3819139c19f
Reviewed-on: https://go-review.googlesource.com/c/go/+/165342
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I had been finding these over a year or so, but none were big enough
changes to warrant CLs. They're a handful now, so clean them all up in a
single commit.
The smaller bodies get a bit simpler, but most importantly, the larger
bodies get unindented.
Change-Id: I5707a6fee27d4c9ff9efd3d363af575d7a4bf2aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/165340
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Given the following types:
type S2 struct{ Field string }
type S struct{ *S2 }
Marshalling a value of type T1 should result in "{}", as there's no way
to access any value of T2.Field. This is how Go 1.10 and earlier
versions behave.
However, in the recent refactor golang.org/cl/125417 I broke this logic.
When the encoder found an anonymous struct pointer field that was nil,
it no longer skipped the embedded fields underneath it. This can be seen
in the added test:
--- FAIL: TestAnonymousFields/EmbeddedFieldBehindNilPointer (0.00s)
encode_test.go:430: Marshal() = "{\"Field\":\"\\u003c*json.S2 Value\\u003e\"}", want "{}"
The human error was a misplaced label, meaning we weren't actually
skipping the right loop iteration. Fix that.
Change-Id: Iba8a4a77d358dac73dcba4018498fe4f81afa263
Reviewed-on: https://go-review.googlesource.com/131376
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Encoders like map and array can use the much cheaper "i > 0" check to
see if we're not writing the first element. However, since struct fields
support omitempty, we need to keep track of that separately.
This is much more expensive - after calling the field encoder itself,
and retrieving the field via reflection, this branch was the third most
expensive piece of this field loop.
Instead, hoist the branch logic outside of the loop. The code doesn't
get much more complex, since we just delay the writing of each byte
until the next iteration. Yet the performance improvement is noticeable,
even when the struct types in CodeEncoder only have 2 and 7 fields,
respectively.
name old time/op new time/op delta
CodeEncoder-4 5.39ms ± 0% 5.31ms ± 0% -1.37% (p=0.010 n=4+6)
name old speed new speed delta
CodeEncoder-4 360MB/s ± 0% 365MB/s ± 0% +1.39% (p=0.010 n=4+6)
Updates #5683.
Change-Id: I2662cf459e0dfd68e56fa52bc898a417e84266c2
Reviewed-on: https://go-review.googlesource.com/131401
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A few encoder struct types, such as map and slice, only encapsulate
other prepared encoder funcs. Using pointer receivers has no advantage,
and makes calling these methods slightly more expensive.
Not a huge performance win, but certainly an easy one. The struct types
used in the benchmark below contain one slice field and one pointer
field.
name old time/op new time/op delta
CodeEncoder-4 5.48ms ± 0% 5.39ms ± 0% -1.66% (p=0.010 n=6+4)
name old speed new speed delta
CodeEncoder-4 354MB/s ± 0% 360MB/s ± 0% +1.69% (p=0.010 n=6+4)
Updates #5683.
Change-Id: I9f78dbe07fcc6fbf19a6d96c22f5d6970db9eca4
Reviewed-on: https://go-review.googlesource.com/131400
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some WriteByte('\\') calls can be deduplicated.
fillField is used in two occasions, but it is unnecessary when adding
fields to the "next" stack, as those aren't used for the final encoding.
Inline the func with its only remaining call.
Finally, unindent a default-if block.
The performance of the encoder is unaffected:
name old time/op new time/op delta
CodeEncoder-4 6.65ms ± 1% 6.65ms ± 0% ~ (p=0.662 n=6+5)
Change-Id: Ie55baeab89abad9b9f13e9f6ca886a670c30dba9
Reviewed-on: https://go-review.googlesource.com/122461
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This function was only used in a single place - in the field encoding
loop within the struct encoder.
Inlining the function call manually lets us get rid of the call
overhead. But most importantly, it lets us simplify the logic afterward.
We no longer need to use reflect.Value{} and !fv.IsValid(), as we can
skip the field immediately.
The two factors combined (mostly just the latter) give a moderate speed
improvement to this hot loop.
name old time/op new time/op delta
CodeEncoder-4 6.01ms ± 1% 5.91ms ± 1% -1.66% (p=0.002 n=6+6)
name old speed new speed delta
CodeEncoder-4 323MB/s ± 1% 328MB/s ± 1% +1.69% (p=0.002 n=6+6)
Updates #5683.
Change-Id: I12757c325a68abb2856026cf719c122612a1f38e
Reviewed-on: https://go-review.googlesource.com/125417
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
structEncoder had two slices - the list of fields, and a list containing
the encoder for each field. structEncoder.encode then looped over the
fields, and indexed into the second slice to grab the field encoder.
However, this makes it very hard for the compiler to be able to prove
that the two slices always have the same length, and that the index
expression doesn't need a bounds check.
Merge the two slices into one to completely remove the need for bounds
checks in the hot loop.
While at it, don't copy the field elements when ranging, which greatly
speeds up the hot loop in structEncoder.
name old time/op new time/op delta
CodeEncoder-4 6.18ms ± 0% 5.56ms ± 0% -10.08% (p=0.002 n=6+6)
name old speed new speed delta
CodeEncoder-4 314MB/s ± 0% 349MB/s ± 0% +11.21% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
CodeEncoder-4 93.2kB ± 0% 62.1kB ± 0% -33.33% (p=0.002 n=6+6)
Updates #5683.
Change-Id: I0dd47783530f439b125e084aede09dda172eb1e8
Reviewed-on: https://go-review.googlesource.com/125416
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If the encoded bytes fit in the bootstrap array encodeState.scratch, use
that instead of allocating a new byte slice.
Also tweaked the Encoding vs Encoder heuristic to use the length of the
encoded bytes, not the length of the input bytes. Encoding is used for
allocations of up to 1024 bytes, as we measured 2048 to be the point
where it no longer provides a noticeable advantage.
Also added some benchmarks. Only the first case changes in behavior.
name old time/op new time/op delta
MarshalBytes/32-4 420ns ± 1% 383ns ± 1% -8.69% (p=0.002 n=6+6)
MarshalBytes/256-4 913ns ± 1% 915ns ± 0% ~ (p=0.580 n=5+6)
MarshalBytes/4096-4 7.72µs ± 0% 7.74µs ± 0% ~ (p=0.340 n=5+6)
name old alloc/op new alloc/op delta
MarshalBytes/32-4 112B ± 0% 64B ± 0% -42.86% (p=0.002 n=6+6)
MarshalBytes/256-4 736B ± 0% 736B ± 0% ~ (all equal)
MarshalBytes/4096-4 7.30kB ± 0% 7.30kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
MarshalBytes/32-4 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.002 n=6+6)
MarshalBytes/256-4 2.00 ± 0% 2.00 ± 0% ~ (all equal)
MarshalBytes/4096-4 2.00 ± 0% 2.00 ± 0% ~ (all equal)
Updates #5683.
Change-Id: I5fa55c27bd7728338d770ae7c0756885ba9a5724
Reviewed-on: https://go-review.googlesource.com/122462
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Struct field names are static, so we can run HTMLEscape on them when
building each struct type encoder. Then, when running the struct
encoder, we can select either the original or the escaped field name to
write directly.
When the encoder is not escaping HTML, using the original string works
because neither Go struct field names nor JSON tags allow any characters
that would need to be escaped, like '"', '\\', or '\n'.
When the encoder is escaping HTML, the only difference is that '<', '>',
and '&' are allowed via JSON struct field tags, hence why we use
HTMLEscape to properly escape them.
All of the above lets us encode field names with a simple if/else and
WriteString calls, which are considerably simpler and faster than
encoding an arbitrary string.
While at it, also include the quotes and colon in these strings, to
avoid three WriteByte calls in the loop hot path.
Also added a few tests, to ensure that the behavior in these edge cases
is not broken. The output of the tests is the same if this optimization
is reverted.
name old time/op new time/op delta
CodeEncoder-4 7.12ms ± 0% 6.14ms ± 0% -13.85% (p=0.004 n=6+5)
name old speed new speed delta
CodeEncoder-4 272MB/s ± 0% 316MB/s ± 0% +16.08% (p=0.004 n=6+5)
name old alloc/op new alloc/op delta
CodeEncoder-4 91.9kB ± 0% 93.2kB ± 0% +1.43% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
CodeEncoder-4 0.00 0.00 ~ (all equal)
Updates #5683.
Change-Id: I6f6a340d0de4670799ce38cf95b2092822d2e3ef
Reviewed-on: https://go-review.googlesource.com/122460
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Fixes#18037
Change-Id: I20e27bcc013b00b726eb348daf5ca86b138ddcc2
Reviewed-on: https://go-review.googlesource.com/107598
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The previous type cache is quadratic in time in the situation where
new types are continually encountered. Now that it is possible to dynamically
create new types with the reflect package, this can cause json to
perform very poorly.
Switch to sync.Map which does well when the cache has hit steady state,
but also handles occasional updates in better than quadratic time.
benchmark old ns/op new ns/op delta
BenchmarkTypeFieldsCache/MissTypes1-8 14817 16202 +9.35%
BenchmarkTypeFieldsCache/MissTypes10-8 70926 69144 -2.51%
BenchmarkTypeFieldsCache/MissTypes100-8 976467 208973 -78.60%
BenchmarkTypeFieldsCache/MissTypes1000-8 79520162 1750371 -97.80%
BenchmarkTypeFieldsCache/MissTypes10000-8 6873625837 16847806 -99.75%
BenchmarkTypeFieldsCache/HitTypes1000-8 7.51 8.80 +17.18%
BenchmarkTypeFieldsCache/HitTypes10000-8 7.58 8.68 +14.51%
The old implementation takes 12 minutes just to build a cache of size 1e5
due to the quadratic behavior. I did not bother benchmark sizes above that.
Change-Id: I5e6facc1eb8e1b80e5ca285e4dd2cc8815618dad
Reviewed-on: https://go-review.googlesource.com/76850
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rather than only ignoring runtime.Error panics, which are a very
narrow set of possible panic values, switch it such that the json
package only captures panic values that have been properly wrapped
in a jsonError struct. This ensures that only intentional panics
originating from the json package are captured.
Fixes#23012
Change-Id: I5e85200259edd2abb1b0512ce6cc288849151a6d
Reviewed-on: https://go-review.googlesource.com/94019
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL reverts CL 76851 and takes a different approach to #21357.
The changes in encode.go and encode_test.go are reverts that
rolls back the changed behavior in CL 76851 where
embedded pointers to unexported struct types were
unilaterally ignored in both marshal and unmarshal.
Instead, these fields are handled as before with the exception that
it returns an error when Unmarshal is unable to set an unexported field.
The behavior of Marshal is now unchanged with regards to #21357.
This policy maintains the greatest degree of backwards compatibility
and avoids silently discarding data the user may have expected to be present.
Fixes#21357
Change-Id: I7dc753280c99f786ac51acf7e6c0246618c8b2b1
Reviewed-on: https://go-review.googlesource.com/82135
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It was added in CL 79995. It is unnecessarily confusing.
Change-Id: Ib8ff35b9f71b54ff99d2d6e0534c7128e1f4345a
Reviewed-on: https://go-review.googlesource.com/80035
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 60410 fixes a bug in reflect that allows assignments to an embedded
field of a pointer to an unexported struct type.
This breaks the json package because unmarshal is now unable to assign
a newly allocated struct to such fields.
In order to be consistent in the behavior for marshal and unmarshal,
this CL changes both marshal and unmarshal to always ignore
embedded pointers to unexported structs.
Fixes#21357
Change-Id: If62ea11155555e61115ebb9cfa5305caf101bde5
Reviewed-on: https://go-review.googlesource.com/76851
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In #10909, it was decided that "Deprecated:" is a magic string for
tools (e.g., #17056 for godoc) to detect deprecated identifiers.
Use those convention instead of custom written prose.
Change-Id: Ia514fc3c88fc502e86c6e3de361c435f4cb80b22
Reviewed-on: https://go-review.googlesource.com/70110
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
CL 60410 fixes the compiler such that reflect.StructField.PkgPath
is non-empty if and only if the field is unexported.
Given that property, we can cleanup the logic in the json encoder
to avoid parsing the field name to detect export properties.
Updates #21122
Change-Id: Ic01b9c4ca76386774846b742b0c1b9b948f53e7c
Reviewed-on: https://go-review.googlesource.com/65550
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Make arguments semantics clear without the need to look for
json.Indent documentation.
Change-Id: If9adfe9f477a30d426ae83790b0f2578c0a809b7
Reviewed-on: https://go-review.googlesource.com/61670
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of these are return values that were part of a receiving parameter,
so they're still accessible.
A few others are not, but those have never had a use.
Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that
the tool should also warn about unused result parameters.
Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b
Reviewed-on: https://go-review.googlesource.com/55910
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
https://golang.org/cl/33773 fixes the JSON marshaler to avoid serializing
embedded fields on unexported types of non-struct types. However, Go allows
embedding pointer to types, so the check for whether the field is a non-struct
type must first dereference the pointer to get at the underlying type.
Furthermore, due to a edge-case in the behavior of StructField.PkgPath not
being a reliable indicator of whether the field is unexported (see #21122),
we use our own logic to determine whether the field is exported or not.
The logic in this CL may be simplified depending on what happens in #21122.
Fixes#21121
Updates #21122
Change-Id: I8dfd1cdfac8a87950df294a566fb96dfd04fd749
Reviewed-on: https://go-review.googlesource.com/50711
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Marshal must process unexported embedded fields of struct type,
looking for exported fields in those structs. However, it must
not process unexported embedded fields of non-struct type.
For example, consider:
type t1 struct {
X int
}
type t2 int
type T struct {
t1
t2
}
When considering T, Marshal must process t1 to find t1.X.
Marshal must not process t2, but it was. Fix that.
Fixes#18009
Change-Id: I62ba0b65ba30fd927990e101a26405a9998787a3
Reviewed-on: https://go-review.googlesource.com/33773
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Struct fields can be suppressed in JSON serialization by "-" tags, but
that doesn't preclude generation of "-" object keys.
Document and verify the mechanism for doing so.
Change-Id: I7f60e1759cfee15cb7b2447cd35fab91c5b004e6
Reviewed-on: https://go-review.googlesource.com/21204
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
