diff options
author | Nick Vatamaniuc <vatamane@gmail.com> | 2023-05-04 02:31:38 -0400 |
---|---|---|
committer | Nick Vatamaniuc <nickva@users.noreply.github.com> | 2023-05-05 12:56:06 -0400 |
commit | bb633a7120b099304344e9d380ea6c3cb2d289b2 (patch) | |
tree | c1557652bdfdd3487afdcc3832a29446928cd218 /LICENSE | |
parent | 4083f7b1b8497c6eb1d6e9415787675513492328 (diff) | |
download | couchdb-bb633a7120b099304344e9d380ea6c3cb2d289b2.tar.gz |
Import xxHash
https://cyan4973.github.io/xxHash/
It's a reasonable replacment for MD5
* It's fast: about the speed of memcpy [1]
* Has a 128 bit variant, so its output is the same size as MD5's.
* It's not cryptographic. So it won't require any replacing again in a few years.
* It's a single header file. So it's easy to update and build.
We need only the 128 bit variant so the NIF only implements that API call at
the moment. To avoid blocking the schedulers on large inputs, the NIF will
swtich to using dirty CPU schedulers if the input size is greater than 1MB.
Benchmarking on an 8 year-old laptop, 1MB block can be hashed in about 40-50
microseconds.
As the first use case replace MD5 in ETag generation.
[1] The speedup compared to MD5:
```
> Payload = crypto:strong_rand_bytes(1024*1024*100).
<<3,24,111,1,194,207,162,224,207,181,240,217,215,218,218,
205,158,34,105,37,113,104,124,155,61,3,179,30,67,...>>
> timer:tc(fun() -> erlang:md5(Payload) end).
{712241,
<<236,134,158,103,156,236,124,91,106,251,186,60,167,244,
30,53>>}
> timer:tc(fun() -> crypto:hash(md5, Payload) end).
{190945,
<<236,134,158,103,156,236,124,91,106,251,186,60,167,244,
30,53>>}
> timer:tc(fun() -> exxhash:xxhash128(Payload) end).
{9952,
<<24,239,152,98,18,100,83,212,174,157,72,241,149,121,161,
122>>}
```
(First element of the tuple is time in microseconds).
Diffstat (limited to 'LICENSE')
-rw-r--r-- | LICENSE | 26 |
1 files changed, 26 insertions, 0 deletions
@@ -2267,3 +2267,29 @@ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +xxHash Library +Copyright (c) 2012-2021 Yann Collet +All rights reserved. + +BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php) + +Redistribution and use in source and binary forms, with or without modification, +are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, this + list of conditions and the following disclaimer in the documentation and/or + other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |