| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| |
| | |
remove-md5-entirely
|
| | |
| |
| |
| | |
Part of a series of changes to expunge MD5 entirely.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Check xxhash first, since it's faster [1], and if that fails, check the slower
md5 version.
Bump a stats counter to indicate if there are still any md5 checksums found
during normal cluster operation.
[1]
Comparison of hashing a 4KB block (units are microseconds).
```
(node1@127.0.0.1)20> f(T), {T, ok} = timer:tc(fun() -> lists:foreach(fun (_) -> do_nothing_overhead end, lists:seq(1, 1000000)) end), (T/1000000.0).
0.167425
(node1@127.0.0.1)21> f(T), {T, ok} = timer:tc(fun() -> lists:foreach(fun (_) -> exxhash:xxhash128(B) end, lists:seq(1, 1000000)) end), (T/1000000).
0.770687
(node1@127.0.0.1)22> f(T), {T, ok} = timer:tc(fun() -> lists:foreach(fun (_) -> crypto:hash(md5, B) end, lists:seq(1, 1000000)) end), (T/1000000).
6.205445
```
|
| | |
| |
| |
| | |
Avoid leaking checksumming details into couch_bt_engine.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
https://cyan4973.github.io/xxHash/
It's a reasonable replacment for MD5
* It's fast: about the speed of memcpy [1]
* Has a 128 bit variant, so its output is the same size as MD5's.
* It's not cryptographic. So it won't require any replacing again in a few years.
* It's a single header file. So it's easy to update and build.
We need only the 128 bit variant so the NIF only implements that API call at
the moment. To avoid blocking the schedulers on large inputs, the NIF will
swtich to using dirty CPU schedulers if the input size is greater than 1MB.
Benchmarking on an 8 year-old laptop, 1MB block can be hashed in about 40-50
microseconds.
As the first use case replace MD5 in ETag generation.
[1] The speedup compared to MD5:
```
> Payload = crypto:strong_rand_bytes(1024*1024*100).
<<3,24,111,1,194,207,162,224,207,181,240,217,215,218,218,
205,158,34,105,37,113,104,124,155,61,3,179,30,67,...>>
> timer:tc(fun() -> erlang:md5(Payload) end).
{712241,
<<236,134,158,103,156,236,124,91,106,251,186,60,167,244,
30,53>>}
> timer:tc(fun() -> crypto:hash(md5, Payload) end).
{190945,
<<236,134,158,103,156,236,124,91,106,251,186,60,167,244,
30,53>>}
> timer:tc(fun() -> exxhash:xxhash128(Payload) end).
{9952,
<<24,239,152,98,18,100,83,212,174,157,72,241,149,121,161,
122>>}
```
(First element of the tuple is time in microseconds).
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
Add new report logging mechanism to log a map of key/value pairs
---------
Co-authored-by: ILYA Khlopotov <iilyak@apache.org>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* FIX NOUVEAU DOCS - MISSING PARAMETER
The Nouveau docs contain guidance on how to code definsively
for handling docs with missing attributes. All of the code
blocks in this section are missing the first parameter
which indicates the data type to be indexed by Lucene.
* FIX NOUVEAU DOCS - SWAP query= for q=
In some places in the Nouveau API examples, there was a
`query=` parameter, when it should be `q=`.
|
| | | |
|
| | |
| |
| |
| |
| | |
The original text said that something that takes 16 hex digits can be represented with just 4 digits (in an hypothetical base62 encoding).
I believe that was a typo since 16 hex digits encode a 8-byte sequence that will require (8/3)*4 = 11 digits in base64 (without padding).
|
| |\ \
| |/
| | |
fix ken_server:nouveau_updated
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We can drop a compat nouveau_maps module. Later we can check the code and see
if we can replace any maps:map/2 with maps:foreach/2 perhaps.
In smoosh_persist, no need to check for file:delete/2. Later we should probably
make the delete in couch_file do the same thing to avoid going through the file
server.
`sha_256_512_supported/0` has been true for a while but the check had been
broken, the latest crypto module is `crypto:mac/3,4` so we can re-enable these
tests.
ML discussion: https://lists.apache.org/thread/7nxm16os8dl331034v126kb73jmb7j3x
|
| |\
| |
| | |
finish partitioned support for nouveau
|
| |/ |
|
| |\
| |
| | |
remove afterburner
|
| |/
|
|
| |
https://github.com/FasterXML/jackson-modules-base/tree/jackson-modules-base-2.13.3/blackbird#readme
|
| |\ |
|
| |/ |
|
| |\
| |
| | |
enable nouveau in CI
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
|
|
|
| |
After the previous fix, now the flakiness moved on to the next line.
Remove the extra assertion to avoid it generating flaky tests. The main
assertion is already checked above that we get a crash.
|
| |
|
|
|
|
| |
It's designed to crash and exit but depending when it does it exactly it may
generate different errors. Add a few more clauses. Hopefully we don't have to
completely remove it or comment it out.
|
| |\
| |
| | |
declare dependency on nouveau
|
| |/ |
|
| | |
|
| | |
|
| |\
| |
| | |
simplify analyzer factory
|
| |/ |
|
| |\
| |
| | |
remove unused facet property
|
| |/
|
|
| |
(all fields are indexed for faceting)
|
| |\ |
|
| |/
|
|
| |
makes it easier for the future 9->10 migration via rebuild.
|
| |
|
| |
Nouveau - a new (experimental) full-text indexing feature for Apache CouchDB, using Lucene 9. Requires Java 11 or higher (19 is preferred).
|
| |
|
|
|
|
|
| |
Sending GET requests targeting paths under the `/{db}/_index`
endpoint, e.g. `/{db}/_index/something`, cause an internal error.
Change the endpoint's behavior to gracefully return HTTP 405
"Method Not Allowed" instead to be consistent with others.
|
| | |
|
