| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Windows UTF-16 strings can contain unpaired surrogates, which can't be
decoded into a valid UTF-8 string. This file defines a set of functions
that can be used to encode and decode potentially ill-formed UTF-16
strings by using the
[the WTF-8 encoding](https://simonsapin.github.io/wtf-8/).
WTF-8 is a strict superset of UTF-8, i.e. any string that is
well-formed in UTF-8 is also well-formed in WTF-8 and the content
is unchanged. Also, the conversion never fails and is lossless.
The benefit of using WTF-8 instead of UTF-8 when decoding a UTF-16
string is that the conversion is lossless even for ill-formed
UTF-16 strings. This property allows to read an ill-formed UTF-16
string, convert it to a Go string, and convert it back to the same
original UTF-16 string.
Fixes #59971
Change-Id: Id6007f6e537844913402b233e73d698688cd5ba6
Reviewed-on: https://go-review.googlesource.com/c/go/+/493036
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Paul Hampson <Paul.Hampson@Pobox.com>
|
|
|
It turns out that the proc thread update function doesn't actually
allocate new memory for its arguments and instead just copies the
pointer values into the preallocated memory. Since we were allocating
that memory as []byte, the garbage collector didn't scan it for pointers
to Go allocations and freed them. We _could_ fix this by requiring that
all users of this use runtime.KeepAlive for everything they pass to the
update function, but that seems harder than necessary. Instead, we can
just do the allocation as []unsafe.Pointer, which means the GC can
operate as intended and not free these from beneath our feet. In order
to ensure this remains true, we also add a test for this.
Fixes #44662.
Change-Id: Ib392ba8ceacacec94b11379919c8179841cba29f
Reviewed-on: https://go-review.googlesource.com/c/go/+/297389
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
