From c7dc17fc0f9341f61be1a1318468409249310316 Mon Sep 17 00:00:00 2001 From: yoav-steinberg Date: Mon, 13 Dec 2021 20:16:25 +0100 Subject: Fix possible int overflow when hashing an sds. (#9916) This caused a crash when adding elements larger than 2GB to a set (same goes for hash keys). See #8455. Details: * The fix makes the dict hash functions receive a `size_t` instead of an `int`. In practice the dict hash functions call siphash which receives a `size_t` and the callers to the hash function pass a `size_t` to it so the fix is trivial. * The issue was recreated by attempting to add a >2gb value to a set. Appropriate tests were added where I create a set with large elements and check basic functionality on it (SADD, SCARD, SPOP, etc...). * When I added the tests I also refactored a bit all the tests code which is run under the `--large-memory` flag. This removed code duplication for the test framework's `write_big_bulk` and `write_big_bulk` code and also takes care of not allocating the test frameworks helper huge string used by these tests when not run under `--large-memory`. * I also added the _violoations.tcl_ unit tests to be part of the entire test suite and leaned up non relevant list related tests that were in there. This was done in this PR because most of the _violations_ tests are "large memory" tests. --- src/dict.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'src/dict.c') diff --git a/src/dict.c b/src/dict.c index 4d064f548..1420055e7 100644 --- a/src/dict.c +++ b/src/dict.c @@ -83,11 +83,11 @@ uint8_t *dictGetHashFunctionSeed(void) { uint64_t siphash(const uint8_t *in, const size_t inlen, const uint8_t *k); uint64_t siphash_nocase(const uint8_t *in, const size_t inlen, const uint8_t *k); -uint64_t dictGenHashFunction(const void *key, int len) { +uint64_t dictGenHashFunction(const void *key, size_t len) { return siphash(key,len,dict_hash_function_seed); } -uint64_t dictGenCaseHashFunction(const unsigned char *buf, int len) { +uint64_t dictGenCaseHashFunction(const unsigned char *buf, size_t len) { return siphash_nocase(buf,len,dict_hash_function_seed); } -- cgit v1.2.1