Stowage/go
2
0
Fork 0
mirror of https://github.com/golang/go.git synced 2025-12-08 06:10:04 +00:00
Code Issues Projects Releases Packages Wiki Activity

encoding/json: remove legacy option to EscapeInvalidUTF8

Browse source 
In the presence of invalid UTF-8, the AllowInvalidUTF8 option
allows such bytes to be present, but silently mangles them
using the Unicode replacement character.

The v2 default is to emit the replacement character verbatim
(which is valid UTF-8 and exactly what it is for).

However, the v1 behavior has historically been to emit
the escaped form of the replacement character.
This behavior was introduced in https://go.dev/cl/11211045
where the documentation says that it is:

    replacing invalid bytes with the Unicode replacement rune U+FFFD

but the implementation actually replaces it with
the escaped form of the Unicode replacement rune.
Given that the documentation differs from the implementation,
the actual behavior is likely an oversight.

Given how esoteric of behavior this is,
we change the v1in2 behavior to avoid the unnecesary escaping
and drop support for EscapeInvalidUTF8.

This does not violate the Go compatibility agreement since
we do not document what the exact syntactic output is.
Also, there has already been prior precedence for changing the output:

* [encoding/json: encode \b and \f as '\b' and '\f' in JSON strings](https://go.dev/cl/521675)
* [encoding/json: encode \n in strings as "\n", not "\u000A"](https://go.dev/cl/4678046)
* [encoding/json: encode \t as \t instead of \u0009](https://go.dev/cl/162340043)
* [encoding/json: use standard ES6 formatting for numbers during marshal](https://go.dev/cl/30371)

Fixes #74551

Change-Id: Ib59a873c44713d302f1f6ab103ffba2520d63276
Reviewed-on: https://go-review.googlesource.com/c/go/+/687116
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This commit is contained in:
Joe Tsai 2025-07-09 17:08:49 -07:00 • committed by Gopher Robot
parent a0a99cb22b
commit 1e48ca7020
5 changed files with 9 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

4
src/encoding/json/internal/jsonflags/flags.go
View file

@ -52,7 +52,6 @@ const (
AllowInvalidUTF8 |
EscapeForHTML |
EscapeForJS |
EscapeInvalidUTF8 |
PreserveRawStrings |
Deterministic |
FormatNilMapAsNull |
@ -77,7 +76,7 @@ const (
WhitespaceFlags = AnyWhitespace | Indent | IndentPrefix
// AnyEscape is the set of flags related to escaping in a JSON string.
AnyEscape = EscapeForHTML | EscapeForJS | EscapeInvalidUTF8
AnyEscape = EscapeForHTML | EscapeForJS
// CanonicalizeNumbers is the set of flags related to raw number canonicalization.
CanonicalizeNumbers = CanonicalizeRawInts | CanonicalizeRawFloats
@ -97,7 +96,6 @@ const (
ReorderRawObjects // encode only
EscapeForHTML // encode only
EscapeForJS // encode only
EscapeInvalidUTF8 // encode only; only exposed in v1
Multiline // encode only
SpaceAfterColon // encode only
SpaceAfterComma // encode only