The Go programming language https://go.dev
Find a file
Joe Tsai 1e48ca7020 encoding/json: remove legacy option to EscapeInvalidUTF8
In the presence of invalid UTF-8, the AllowInvalidUTF8 option
allows such bytes to be present, but silently mangles them
using the Unicode replacement character.

The v2 default is to emit the replacement character verbatim
(which is valid UTF-8 and exactly what it is for).

However, the v1 behavior has historically been to emit
the escaped form of the replacement character.
This behavior was introduced in https://go.dev/cl/11211045
where the documentation says that it is:

    replacing invalid bytes with the Unicode replacement rune U+FFFD

but the implementation actually replaces it with
the escaped form of the Unicode replacement rune.
Given that the documentation differs from the implementation,
the actual behavior is likely an oversight.

Given how esoteric of behavior this is,
we change the v1in2 behavior to avoid the unnecesary escaping
and drop support for EscapeInvalidUTF8.

This does not violate the Go compatibility agreement since
we do not document what the exact syntactic output is.
Also, there has already been prior precedence for changing the output:

* [encoding/json: encode \b and \f as '\b' and '\f' in JSON strings](https://go.dev/cl/521675)
* [encoding/json: encode \n in strings as "\n", not "\u000A"](https://go.dev/cl/4678046)
* [encoding/json: encode \t as \t instead of \u0009](https://go.dev/cl/162340043)
* [encoding/json: use standard ES6 formatting for numbers during marshal](https://go.dev/cl/30371)

Fixes #74551

Change-Id: Ib59a873c44713d302f1f6ab103ffba2520d63276
Reviewed-on: https://go-review.googlesource.com/c/go/+/687116
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2025-07-11 14:27:20 -07:00
.github .github: update language change proposal template 2024-08-08 19:02:29 +00:00
api testing: add Output method to TB 2025-06-02 09:09:57 -07:00
doc cmd/go: disable support for multiple vcs in one module 2025-07-08 11:30:38 -07:00
lib cmd/dist: test encoding/json/... with GOEXPERIMENT=jsonv2 2025-06-23 18:21:07 -07:00
misc misc/linkcheck: remove unused tool 2025-03-20 04:38:55 -07:00
src encoding/json: remove legacy option to EscapeInvalidUTF8 2025-07-11 14:27:20 -07:00
test cmd/compile/internal/escape: evaluate any side effects when rewriting with literals 2025-06-26 10:24:19 -07:00
.gitattributes all: treat all files as binary, but check in .bat with CRLF 2020-06-08 15:31:43 +00:00
.gitignore runtime,internal: move runtime/internal/sys to internal/runtime/sys 2024-07-23 19:05:35 +00:00
codereview.cfg codereview.cfg: add codereview.cfg for master branch 2021-02-19 18:44:53 +00:00
CONTRIBUTING.md doc: normalize proposal-process links 2023-03-29 22:00:27 +00:00
go.env cmd/go: additional doc-inspired tests and bug fixes 2023-06-06 19:18:46 +00:00
LICENSE LICENSE: update per Google Legal 2024-08-09 14:54:31 +00:00
PATENTS LICENSE: separate, change PATENTS text 2010-12-06 16:31:59 -05:00
README.md README: fix CC BY license name 2024-07-22 17:45:27 +00:00
SECURITY.md SECURITY.md: update the Reporting a Vulnerability link 2023-09-22 21:17:24 +00:00

The Go Programming Language

Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.

Gopher image Gopher image by Renee French, licensed under Creative Commons 4.0 Attribution license.

Our canonical Git repository is located at https://go.googlesource.com/go. There is a mirror of the repository at https://github.com/golang/go.

Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.

Download and Install

Binary Distributions

Official binary distributions are available at https://go.dev/dl/.

After downloading a binary release, visit https://go.dev/doc/install for installation instructions.

Install From Source

If a binary distribution is not available for your combination of operating system and architecture, visit https://go.dev/doc/install/source for source installation instructions.

Contributing

Go is the work of thousands of contributors. We appreciate your help!

To contribute, please read the contribution guidelines at https://go.dev/doc/contribute.

Note that the Go project uses the issue tracker for bug reports and proposals only. See https://go.dev/wiki/Questions for a list of places to ask questions about the Go language.