delta/go-git.git/src/encoding, branch dev.types

encoding/binary: replace constant literals with named constant (cleanup)

2020-08-25T16:06:28+00:00

Follow-up on https://golang.org/cl/247120.

Brought to my attention by Luke McCoy.

Change-Id: I4530c96fb164d23b0ce5311f2cecb1964f2dea74
Reviewed-on: https://go-review.googlesource.com/c/go/+/249837
Reviewed-by: Katie Hockman

encoding/asn1: speed up marshal by reducing allocations

2020-08-18T17:46:40+00:00

Replace strings.Split by strings.IndexByte and explicit
slicing to avoid the allocation of the return slice
of strings.Split.

name     old time/op    new time/op    delta
Marshal    43.3µs ± 1%    36.7µs ± 1%  -15.23%  (p=0.000 n=9+9)

name     old alloc/op   new alloc/op   delta
Marshal    10.7kB ± 0%     9.2kB ± 0%  -13.96%  (p=0.000 n=10+10)

name     old allocs/op  new allocs/op  delta
Marshal       444 ± 0%       366 ± 0%  -17.57%  (p=0.000 n=10+10)

Change-Id: I9e727defa23f7e5fc684f246de0136fe28cf8d25
Reviewed-on: https://go-review.googlesource.com/c/go/+/231738
TryBot-Result: Gobot Gobot 
Reviewed-by: Brad Fitzpatrick

encoding/binary: read at most MaxVarintLen64 bytes in ReadUvarint

2020-08-06T17:24:10+00:00

This CL ensures that ReadUvarint consumes only a limited
amount of input (instead of an unbounded amount).

On some inputs, ReadUvarint could read an arbitrary number
of bytes before deciding to return an overflow error.
After this CL, ReadUvarint returns that same overflow
error sooner, after reading at most MaxVarintLen64 bytes.

Fix authored by Robert Griesemer and Filippo Valsorda.

Thanks to Diederik Loerakker, Jonny Rhea, Raúl Kripalani,
and Preston Van Loon for reporting this.

Fixes #40618
Fixes CVE-2020-16845

Change-Id: Ie0cb15972f14c38b7cf7af84c45c4ce54909bb8f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/812099
Reviewed-by: Filippo Valsorda 
Reviewed-on: https://go-review.googlesource.com/c/go/+/247120
Run-TryBot: Katie Hockman 
TryBot-Result: Gobot Gobot 
Reviewed-by: Alexander Rakoczy

Revert "encoding/json: don't reuse slice elements when decoding"

2020-07-02T22:08:11+00:00

This reverts https://golang.org/cl/191783.

Reason for revert: Broke too many programs which depended on the previous
behavior, even when it was the opposite of what the documentation said.

We can attempt to fix the original issue again for 1.16, while keeping
those programs in mind.

Fixes #39427.

Change-Id: I7a7f24b2a594c597ef625aeff04fff29aaa88fc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/240657
Run-TryBot: Dmitri Shuralyov 
TryBot-Result: Gobot Gobot 
Reviewed-by: Emmanuel Odeke 
Reviewed-by: Ian Lance Taylor

Revert "encoding/xml: fix reserved namespace check to be case-insensitive"

2020-06-29T21:33:02+00:00

This reverts CL 203417.

Reason for revert: This change changes uses of tags like "XMLSchema-instance" without any recourse.

For #35151
Fixes #39876

Change-Id: I4c85c8267a46b3748664b5078794dafffb42aa26
Reviewed-on: https://go-review.googlesource.com/c/go/+/240179
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Gobot Gobot 
Reviewed-by: Emmanuel Odeke 
Reviewed-by: Andrew Bonventre

encoding/json: revert "avoid work when unquoting strings, take 2"

2020-06-15T16:10:18+00:00

This reverts golang.org/cl/190659 and golang.org/cl/226218, minus the
regression tests in the latter.

The original work happened in golang.org/cl/151157, which was reverted
in golang.org/cl/190909 due to a crash found by fuzzing.

We tried a second time in golang.org/cl/190659, which shipped with Go
1.14. A bug was found, where strings would be mangled in certain edge
cases. The fix for that was golang.org/cl/226218, which was backported
into Go 1.14.4.

Unfortunately, a second regression was just reported in #39555, which is
a similar case of strings getting mangled when decoding under certain
conditions. It would be possible to come up with another small patch to
fix that edge case, but instead, let's just revert the entire
optimization, as it has proved to do more harm than good. Moreover, it's
hard to argue or prove that there will be no more such regressions.

However, all the work wasn't for nothing. First, we learned that the way
the decoder unquotes tokenized strings isn't simple; initially, we had
wrongly assumed that each string was unquoted exactly once and in order.
Second, we have gained a number of regression tests which will be useful
to prevent the same mistakes in the future, including the test cases we
add in this CL.

Fixes #39555.

Change-Id: I66a6919c2dd6d9789232482ba6cf3814eaa70f61
Reviewed-on: https://go-review.googlesource.com/c/go/+/237838
Run-TryBot: Daniel Martí 
TryBot-Result: Gobot Gobot 
Reviewed-by: Andrew Bonventre

encoding/xml: only initialize nil struct fields when decoding

2020-05-28T22:48:53+00:00

fieldInfo.value used to initialize nil anonymous struct fields if they
were encountered. This behavior is wanted when decoding, but not when
encoding. When encoding, the value should never be modified, and these
nil fields should be skipped entirely.

To fix the bug, add a bool argument to the function which tells the
code whether we are encoding or decoding.

Finally, add a couple of tests to cover the edge cases pointed out in
the original issue.

Fixes #27240.

Change-Id: Ic97ae4bfe5f2062c8518e03d1dec07c3875e18f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/196809
Run-TryBot: Daniel Martí 
TryBot-Result: Gobot Gobot 
Reviewed-by: Emmanuel Odeke

Revert "encoding/json: reuse values when decoding map elements"

2020-05-28T22:17:10+00:00

This reverts golang.org/cl/179337.

Reason for revert: broke a few too many reasonably valid Go programs.
The previous behavior was perhaps less consistent, but the docs were
never very clear about when the decoder merges with existing values,
versus replacing existing values altogether.

Fixes #39149.

Change-Id: I1c1d857709b8398969fe421aa962f6b62f91763a
Reviewed-on: https://go-review.googlesource.com/c/go/+/234559
Run-TryBot: Daniel Martí 
TryBot-Result: Gobot Gobot 
Reviewed-by: Andrew Bonventre

encoding/asn1: document what Unmarshal returns in rest

2020-05-23T06:47:06+00:00

Specifically, this change documents the behavior of Unmarshal when a
SEQUENCE contains trailing elements.

For context Unmarshal treats trailing elements of a SEQUENCE that do not
have matching struct fields as valid, as this is how ASN.1 structures
are typically extended. This can be somewhat confusing as you might
expect those elements to be appended to rest, but rest is really only
for trailing data unrelated to the structure being parsed (i.e. if you
append a second sequence to b, it would be returned in rest).

Fixes #35680

Change-Id: Ia2c68b2f7d8674d09e859b4b7f9aff327da26fa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/233537
Run-TryBot: Emmanuel Odeke 
TryBot-Result: Gobot Gobot 
Reviewed-by: Katie Hockman

encoding/json: reuse values when decoding map elements

2020-05-08T21:19:55+00:00

When we decode into a struct, each input key-value may be decoded into
one of the struct's fields. Particularly, existing data isn't dropped,
so that some sub-fields can be decoded into without zeroing all other
data.

However, decoding into a map behaved in the opposite way. Whenever a
key-value was decoded, it completely replaced the previous map element.
If the map contained any non-zero data in that key, it's dropped.

Instead, try to reuse the existing element value if possible. If the map
element type is a pointer, and the value is non-nil, we can decode
directly into it. If it's not a pointer, make a copy and decode into
that copy, as map element values aren't addressable.

This means we have to parse and convert the map element key before the
value, to be able to obtain the existing element value. This is fine,
though. Moreover, reporting errors on the key before the value follows
the input order more closely.

Finally, add a test to explore the four combinations, involving pointer
and non-pointer, and non-zero and zero values. A table-driven test
wasn't used, as each case required different checks, such as checking
that the non-nil pointer case doesn't end up with a different pointer.

Fixes #31924.

Change-Id: I5ca40c9963a98aaf92f26f0b35843c021028dfca
Reviewed-on: https://go-review.googlesource.com/c/go/+/179337
Run-TryBot: Daniel Martí 
TryBot-Result: Gobot Gobot 
Reviewed-by: Ian Lance Taylor