The fully streaming UnmarshalJSONFrom method and UnmarshalFromFunc
introduce an edge case where they can encounter EOF in the stream,
where it should be reported upstream as EOF rather than
ErrUnexpectedEOF or be wrapped within a SemanticError.
This is not possible with other unmarshal methods since the
"json" package would read the appropriate JSON value
before calling the custom method or function.
To avoid custom unmarshal methods from encountering EOF,
check whether the stream is already at EOF for top-level values
before calling the custom method.
Also, when wrapping EOF within a SemanticError, convert it
to ErrUnexpectedEOF to better indicate that this is unexpected.
Fixes#75802
Change-Id: I001396734b7e95b5337f77b71326284974ee730a
Reviewed-on: https://go-review.googlesource.com/c/go/+/710877
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
When EOF is encountered within jsontext.Decoder stream without
starting to parse any token, UnmarshalDecode should report EOF,
rather than converting it into ErrUnexpectedEOF.
This fixes a regression introduced by https://go.dev/cl/689919.
This change only affects code compiled under goexperiment.jsonv2.
Fixes#74835
Change-Id: I7e8e57ab11b462c422c538503ed8c6b91ead53bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/692175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
When operating under v1 semantics in the v2 implementation,
a extra data error should take precedence over any semantic error
that could theoretically occur within the value itself.
This change only affects code compiled under goexperiment.jsonv2.
Fixes#74614
Change-Id: I055a606b053fa66b0c766ae205487b8290109285
Reviewed-on: https://go-review.googlesource.com/c/go/+/689919
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In the event that the input is just JSON whitespace,
the underlying jsontext.Decoder treats this as an empty stream
and reports io.EOF.
The logic in unmarshalFull simply casted io.EOF as io.ErrUnexpectedEOF,
which is inconsistent with how all other io.ErrUnexpectedEOF are reported,
which are wrapped within a jsontext.SyntacticError.
Do the same thing for consistency.
We add a v1 test (without goexperiment.jsonv2) to verify that
the behavior is identical to how v1 has always behaved.
We add a v1in2 test (with goexperiment.jsonv2) to verify that
the v1in2 behavior correctly replicates historical v1 behavior.
We also fix a faulty check in v1 Decoder.Decode,
where it tried to detect errUnexpectedEnd and
return an unwrapped io.ErrUnexpectedEOF error.
This is the exact semantic that v1 has always done
in streaming Decoder.Decode (but not non-streaming Unmarshal).
There is a prior bug reported in #25956 about this inconsistency,
but we aim to preserve historical v1 behavior to reduce
the probability of churn when v1 is re-implemented in terms of v2.
Fixes#74548
Change-Id: Ibca52c3699ff3c09141e081c85f853781a86ec8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/687115
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Based on the discussion in #71631, it is hotly contested
whether the default JSON representation for a Go time.Duration
should be the time.Duration.String format or
a particular profile of ISO 8601.
Regardless of the default, it seems clear that we should
at least support ISO 8601 if specified via a format flag.
Note that this CL does not alter the default representation.
Unfortunately, ISO 8601 is a large and evolving standard
with many optional extensions and optional restrictions.
Thus, the term "ISO 8601 duration" unfortunately does not
resolve to a particular grammar, nor one that is stable.
However, there is precedence that we can follow in this matter.
JSON finds its heritage in JavaScript and
JavaScript is adding a Temporal.Duration type whose default
JSON representation is ISO 8601.
There is a well-specified grammar for their particular
profile of ISO 8601, which is documented at:
https://tc39.es/proposal-temporal/#prod-Duration
This particular CL adds support for ISO 8601 according to
the exact same grammar that JavaScript uses.
While Temporal.Duration is technically still a proposal,
it is already in stage 3 of the TC39 proposal process
(i.e., "no changes to the proposal are expected"
and "has been recommended for implementation")
and therefore close to final adoption.
One major concern with ISO 8601 is that it supports
nominal date units like years, months, weeks, and days
that do not have an accurate meaning without being
anchored to a particular point in time and place on Earth.
Fortunately, JavaScript (by default) avoids producing
Temporal.Duration values with nominal units unless
arithmetic in JavaScript explicitly sets a largestUnits
value that is larger than "hours". In the Go implementation,
we support syntactically parsing the full ISO 8601 grammar
(according to JavaScript), but semantically report an error if
nominal units are present. This ensures that ISO 8601 durations
remain accurate so long as they only use the accurate units
of hours, minutes, or seconds.
Updates #71631
Change-Id: I983593662f2150461ebc486a5acfeb72f0286939
Reviewed-on: https://go-review.googlesource.com/c/go/+/682403
Reviewed-by: Daniel Martà <mvdan@mvdan.cc>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This imports the proposed new v2 JSON API implemented in
github.com/go-json-experiment/json as of commit
d3c622f1b874954c355e60c8e6b6baa5f60d2fed.
When GOEXPERIMENT=jsonv2 is set, the encoding/json/v2 and
encoding/jsontext packages are visible, the encoding/json
package is implemented in terms of encoding/json/v2, and
the encoding/json package include various additional APIs.
(See #71497 for details.)
When GOEXPERIMENT=jsonv2 is not set, the new API is not
present and the encoding/json package is unchanged.
The experimental API is not bound by the Go compatibility
promise and is expected to evolve as updates are made to
the json/v2 proposal.
The contents of encoding/json/internal/jsontest/testdata
are compressed with zstd v1.5.7 with the -19 option.
Fixes#71845
For #71497
Change-Id: Ib8c94e5f0586b6aaa22833190b41cf6ef59f4f01
Reviewed-on: https://go-review.googlesource.com/c/go/+/665796
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
